PhD Proposal: Privacy-Preserving Machine Learning: Theory And Efficient Methods For Differential Privacy And Homomorphic Encryption

The rapid growth of machine learning (ML) has led to its increasing adoption in applications involving sensitive and private data. However, ML systems face privacy risks both during outsourced training, when servers may infer information about training data, and after deployment, when released models may leak sensitive information. This thesis investigates privacy-preserving ML through two complementary techniques: differential privacy (DP) and fully homomorphic encryption (FHE), focusing on protecting data during model training and preventing privacy leakage from released models.
For pre-training protection, this work studies DP synthetic data generation based on low-order marginal preservation. We establish theoretical excess-risk bounds for downstream learning, characterizing the relationship between marginal approximation error and predictive utility. We also develop lower bounds and evaluate the practical performance of marginal-based DP synthetic data, demonstrating its effectiveness for preserving downstream model utility under privacy constraints.
For during-training protection, this work combines FHE and DP to enable private model training on encrypted data. We establish convergence guarantees for polynomially approximated gradients and analyze the impact of Gaussian DP noise. To improve efficiency, we develop an FHE-compatible DP training algorithm that bounds the optimization domain and controls approximation error without conventional gradient clipping, reducing computational overhead while maintaining rigorous privacy guarantees.
Building on these results, this thesis proposes two research directions. First, we aim to extend FHE-based private training beyond smooth convex objectives to general and non-convex models, including deep neural networks, with theoretical guarantees for convergence, privacy, and efficiency. Second, we investigate whether inherent noise and approximation errors in FHE can be leveraged to reduce explicit DP noise while preserving rigorous privacy guarantees and improving model utility.
Overall, this work aims to develop efficient and theoretically grounded privacy-preserving ML methods by integrating statistical learning theory, differential privacy, and cryptographic techniques.