Privacy-Preserving Federated Learning

• Privacy-preserving federated learning is a collaborative learning paradigm that protects sensitive information by integrating differential privacy, secure multiparty computation, and encryption techniques.
• It utilizes methods such as gradient noise addition, secure aggregation protocols, and trusted execution environments to defend against both honest-but-curious and advanced adversaries.
• The approach balances privacy, utility, and efficiency, offering trade-offs in accuracy versus computational overhead and communication costs while ensuring robust data confidentiality.

Privacy-preserving federated learning (PPFL) refers to the suite of technical and theoretical approaches that enable collaborative machine learning across multiple silos or clients while maintaining strong guarantees on the confidentiality of raw data, individual model contributions, and often the content of intermediate updates. PPFL targets not only classic honest-but-curious adversaries but also sophisticated collusion, Byzantine, and inference attacks. Research in this area draws from differential privacy, secure multiparty computation (SMPC), homomorphic encryption, secure aggregation, trusted execution environments, and advanced protocol design, producing a rich taxonomy of defenses with distinct privacy-utility-communication trade-offs.

1. Formal Foundations and Privacy Objectives

PPFL formalizes privacy guarantees in several rigorous senses—most notably differential privacy (DP), cryptographic simulation-based security, and information-theoretic indistinguishability. The most prevalent definition is $(\epsilon, \delta)$-DP:

• Differential Privacy: A randomized mechanism $\mathcal{A}$ is $(\epsilon,\delta)$-differentially private if, for all adjacent datasets $D, D'$ differing in a single record, and all measurable $S$, $$\Pr[\mathcal{A}(D) \in S] \leq e^\epsilon \Pr[\mathcal{A}(D') \in S] + \delta$$ (Ryu et al., 2022, Byrd et al., 2022, Fernandez et al., 2021, Piran et al., 2024).
• Oblivious DP: Strengthens standard distributed DP so that no coalition of $t < n$ clients or even $n-1$ colluders can invert or strip away all privacy noise from honest contributions, using protocols that blind the noise allocation and ensure residual DP noise under full collusion (Byrd et al., 2022).
• Cryptographic Security: Simulation-based definitions guarantee that an adversary, even observing all protocol state (minus private randomness), cannot tell the difference between real and ideal executions (Nie et al., 2024, Cai et al., 15 Jun 2025), relying on primitives such as Paillier encryption, Groth16 zk-SNARKs, and (decentralized) functional encryption.
• Attribute/Label Protection: In vertical FL, various threat models directly formalize the adversary’s ability to infer “partitioned attributes” or labels from exchanged representations, with formal privacy loss measured by minimal attack success or bounded recovery error (Zhang et al., 2021, Gu et al., 2024).

2. Methodological Classes of PPFL Approaches

Depending on the system and adversary model, PPFL is achieved by several non-mutually exclusive classes:

1. Differential Privacy Mechanisms
   • Gradient or Update Noise: Clip per-client updates to norm-bounded envelopes, then inject noise drawn from Laplace (Ryu et al., 2022, Byrd et al., 2022) or Gaussian distributions (Piran et al., 2024, Fernandez et al., 2021, Arazzi et al., 2023, Zhang et al., 2021). Privacy budget tracked with sequential or Rényi composition.
   • Cumulative Noise Minimization: In dynamic or lifelong FL (e.g., IoT), optimally monitor accumulated noise and inject only the additional noise needed per round, avoiding the exponential degradation of accuracy with naive DP (Piran et al., 2024).
   • DP in Hyperdimensional Computing: Symbolic vector representations (HD computing) naturally enable per-dimension DP, with closed-form sensitivity bounds and controllable privacy-utility tradeoff (Piran et al., 2024).
2. Secure Aggregation Protocols
   • Pairwise Masking and SMPC: Each client masks its update with pairwise shared random keys, such that only the aggregate can be revealed to the server. Resilient to client dropouts and avoids the need for trusted key management (Mugunthan et al., 2020, Fernandez et al., 2021).
   • Decentralized Aggregation/Block Design: Reduce privacy leakage by limiting which clients communicate in each aggregation round, ensuring that no adversary can solve for an individual update within a bounded number of iterations (Jeon et al., 2020).
3. Homomorphic and Functional Encryption
   • Homomorphic Encryption: Additively or multiplicatively homomorphic schemes (e.g. Paillier) enable the server to aggregate encrypted client updates without access to plaintext, supporting masking via jointly generated random vectors (Nie et al., 2024, Xu et al., 9 Jan 2025).
   • Functional Encryption (FE): Decrypts only the aggregate (e.g., weighted sum of updates) while hiding individual values; modern schemes support threshold decryption and verification, eliminating reliance on a central authority (Xu et al., 2022, Xu et al., 9 Jan 2025, Cai et al., 15 Jun 2025).
   • Verifiable FE: Incorporates zero-knowledge proofs that ciphertexts and functional key shares are well-formed, preventing malicious clients from forging or manipulating the aggregation process (Cai et al., 15 Jun 2025).
4. Trusted Execution Environments (TEEs)
   • Enclave-Based Training/Aggregation: Local and/or server-side TEEs (e.g., ARM TrustZone, Intel SGX) confidentially process model updates, hiding plaintext weights/gradients from OS and hypervisor adversaries. Layer-wise or greedy approaches address memory constraints (Mo et al., 2021).
5. Adaptive Obfuscation and Encodings
   • System Immersion and Random Matrix Coding: Treat FL as a dynamical system immersed in a higher-dimensional (encrypted) space via random coordinate transformations. These encodings are information-theoretically secure and maintain exact convergence, with optional DP-compliant noise for quantifiable privacy (Hayati et al., 2022, Hayati et al., 2024).
   • Passport-based Obfuscation: Insert adaptive, secret-keyed layers that obfuscate features or labels, yielding exponentially small recovery probability against model/gradient inversion (in both horizontal and vertical FL) (Gu et al., 2024).
6. Knowledge Distillation Paradigms
   • Offline, One-shot Distillation: Avoid transmission of parameters/gradients entirely; only distilled teacher logits and attention maps (on public data) are shared. This shrinks the attack surface and reduces communication overhead (Gong et al., 2022).

3. Protocols, Algorithms, and Implementation Patterns

The following abstract protocols typify the execution of PPFL under these various strategies:

Technique	Communication	Privacy Guarantee
Differential Privacy (DP)	Model/gradient + noise	$(\epsilon, \delta)$-DP w.r.t. local/global data (Ryu et al., 2022)
Secure Aggregation (SMPC/masking)	Masked updates; no plaintext	Aggregate-only recovery; strong input confusion (Fernandez et al., 2021, Mugunthan et al., 2020)
Homomorphic/Functional Encryption	Encrypted updates, partial decrypts	Only aggregate is ever revealed; hides all intermediates to threshold adversaries (Nie et al., 2024, Xu et al., 9 Jan 2025, Xu et al., 2022)
TEE Layerwise FL	Encrypted per-layer weights; attested channels	No exposure of gradients or weights outside enclaves (Mo et al., 2021)
System Immersion Random Coding	Encoded models/updates at each step	Information-theoretic secrecy; exact model recovery (Hayati et al., 2022, Hayati et al., 2024)
Adaptive Obfuscation (Passports)	Obfuscated features/labels	Exponential lower bounds on inversion probability (Gu et al., 2024)
Ensemble Attention Distillation	Only logit/attention vectors on public data	Public-only sharing, practical "signal-size" privacy (Gong et al., 2022)

Example: Differential Privacy Update

Let $z_p^{t+1}$ denote the update from client $p$ at round $t+1$. Then with gradient clipping and Laplace noise (as in (Ryu et al., 2022)):

$$z_p^{t+1} \leftarrow \mathrm{clip}(z_p^{t+1}, C) + \xi, \quad \xi \sim \mathrm{Lap}\left(0, \tfrac{\Delta}{\epsilon}\right)$$

where $\Delta$ is the sensitivity determined by the clipping norm and the ADMM penalty parameters.

4. Robustness against Advanced Adversaries

Recent work has moved beyond basic privacy guarantees to defend against stronger attack models:

• Collusion Resistance: Protocols such as oblivious distributed DP ensure that even $n-1$ colluding clients or server cannot reconstruct the honest party's data, via randomized noise sharing and permutation (Byrd et al., 2022).
• Byzantine/Poisoning-Resilience: Non-interactive zero-knowledge proofs (zk-SNARKs) coupled with similarity constraints (Euclidean, cosine) enforce that only realistically plausible (non-Byzantine) updates are accepted. Masking via jointly derived random vectors ensures that malicious clients cannot invert or tamper with others' updates (Nie et al., 2024, Cai et al., 15 Jun 2025).
• Functional Encryption Aggregators: Threshold (t-of-m) FE schemes (e.g., TAPFed) support multiple aggregators with up to $(t-1)$ malicious, blocking disaggregation, isolation, and replay attacks (Xu et al., 9 Jan 2025).
• Attribute and Label Leakage in VFL: Adversarial learning and forward-backward splitting defenders reduce privacy leakage of intermediate representations in vertically partitioned data without heavy dependence on DP noise or cryptographic primitives (Zhang et al., 2021).

5. Privacy–Utility–Efficiency Trade-offs

The efficiency and inferential utility of PPFL methods vary considerably by defensive primitive:

• Accuracy Preservation:
  • Information-theoretic/cryptographic schemes (random coding, secure aggregation, FE, TEE) introduce negligible or near-zero utility loss—any loss is due to encoding/decoding error or noise for DP extensions (Hayati et al., 2022, Hayati et al., 2024, Mugunthan et al., 2020, Xu et al., 9 Jan 2025, Cai et al., 15 Jun 2025).
  • DP-based mechanisms trade accuracy for privacy: e.g., in residential load forecasting, $\epsilon\approx4.2$ yields $\sim$3% higher MAPE than no privacy, and adaptive clipping can mitigate this (Fernandez et al., 2021). In hyperdimensional encoding FL, under strong DP ($\epsilon=10$, $\delta=1/(KLr)$), only $\approx$5% absolute accuracy loss is seen for aggressive privacy, versus 25–40% for baseline neural FL (Piran et al., 2024).
• Computation and Communication:
  • FE-based and random coding require matrix/vector transformations and modular exponentiations, but overheads are sublinear or linear in update dimension and number of clients (Hayati et al., 2024, Xu et al., 9 Jan 2025).
  • Modern zk-SNARKs (Nie et al., 2024, Cai et al., 15 Jun 2025) and TEE-based FL (Mo et al., 2021) incur nontrivial per-round or per-batch costs, but typically scale well on moderate hardware.
  • Communication cost is reduced in distillation or one-shot upload approaches (Gong et al., 2022), and sharply increased for multi-round FE/HE setups unless thresholding or bandwidth optimization strategies are adopted (Xu et al., 9 Jan 2025).
• Scalability & Decentralization:
  • Most protocols scale to hundreds–thousands of clients with commodity hardware (see e.g., APPFL's MPI+RDMA deployment (Ryu et al., 2022)), but cryptographic and SMPC protocols may require protocol-specific scaling techniques or batching for global FL at massive scale.

Defense Class	Privacy Guarantee	Utility Loss	Per-Round Overhead
Central DP	Statistical	Moderate	Low
Local/Distributed DP	Statistical	Higher (if noise)	Low
Secure Aggregation	Information-theoretic	Negligible	Medium (mask exchange)
Functional Encryption	Cryptographic/Ideal	Negligible	Medium–High (decryption, FE ops)
TEE-based	Hardware-protected	Negligible	Moderate (layerwise)
Adaptive Obfuscation	Statistical	Negligible	Low
Immersion/Coding	Information-theoretic	Zero (theoretical)	Low

6. Challenges and Future Directions

Recent research identifies open issues and forwards-looking themes:

• Robustness to Advanced Attacks: Community-driven GNN recommenders demonstrate that (ε,δ)-DP and neighbor aggregation can be subverted by carefully engineered pseudo-items and backdoors. Combinatory defenses including certified neighbor sampling, per-neighbor reputation, spectral anomaly detection, and stronger DP accounting are proposed (Arazzi et al., 2023).
• Decentralized Trust and Collusion: Functional encryption and protocol-level compliance checking can decentralize secure aggregation, support party churn, and enforce forced inclusion/participation (Xu et al., 2022, Xu et al., 9 Jan 2025).
• Composable Mechanisms and Hybridization: Combining system immersion, DP, and cryptographic masking can yield arbitrarily strong, composable privacy for arbitrary FL dynamical systems (Hayati et al., 2024).
• Scalable and Practical Implementations: Emphasis on one-shot or periodic distillation, progressive noise monitoring (FedHDPrivacy), and modular Python frameworks (APPFL, PrivacyFL) facilitate real-world, sector-specific deployments in domains such as biomedicine, IoT, smart grid, and manufacturing (Ryu et al., 2022, Piran et al., 2024, Mugunthan et al., 2020).
• Explainability and Personalization: HD computing and passport-based adaptive obfuscation offer explainable, fine-grained noise addition—potentially enabling personalized privacy budgets per client or feature (Piran et al., 2024, Gu et al., 2024).

7. Empirical Results: Summary Table

Method	Privacy Mechanism	Max Accuracy Loss	Communication Delta	Main Benchmark(s)	Reference
IIADMM (APPFL)	DP (Laplace)	negligible (under ε=3–10)	Same as FedAvg; gRPC ~10× slower than MPI	MNIST, CIFAR10, FEMNIST	(Ryu et al., 2022)
FedHDPrivacy	DP (advanced)	≤5% vs. no-noise	Not limiting	Smart manufacturing, IoT	(Piran et al., 2024)
Secure Aggregation	Masking/SMPC	negligible	+30%	Smart grid load forecasting	(Fernandez et al., 2021)
SecureD-FL	Combinatorial (info-theoretic)	<0.73%	Small peer-wise exchange	FEMNIST, CIFAR-10	(Jeon et al., 2020)
TEE (PPFL)	Hardware enclave	≈0	≈1× FedAvg; +15–20% client CPU	CIFAR-10, MNIST	(Mo et al., 2021)
TAPFed	Threshold FE	0	29–45% less than prior FE	MNIST, CIFAR-10	(Xu et al., 9 Jan 2025)
VFEFL	Verifiable FE	0 (fidelity)	Per-round: modern pairing cost	MNIST, FashionMNIST, CIFAR-10	(Cai et al., 15 Jun 2025)
FedAdOb	Adaptive obfusc.	<2%	<2% runtime overhead	MNIST, CIFAR-10, Criteo	(Gu et al., 2024)
FedAD (attention distil.)	Distillation	<2pp vs. centralized	<20GB (vs. 100–1000GB for FedAvg)	NIH CXR14, CheXpert, BraTS	(Gong et al., 2022)

• (Ryu et al., 2022) APPFL: Open-Source Software Framework for Privacy-Preserving Federated Learning
• (Piran et al., 2024) Privacy-Preserving Federated Learning with Differentially Private Hyperdimensional Computing
• (Jeon et al., 2020) Privacy-preserving Decentralized Aggregation for Federated Learning
• (Mo et al., 2021) PPFL: Privacy-preserving Federated Learning with Trusted Execution Environments
• (Hayati et al., 2024) Immersion and Invariance-based Coding for Privacy-Preserving Federated Learning
• (Byrd et al., 2022) Collusion Resistant Federated Learning with Oblivious Distributed Differential Privacy
• (Nie et al., 2024) Efficient Byzantine-Robust and Provably Privacy-Preserving Federated Learning
• (Xu et al., 9 Jan 2025) TAPFed: Threshold Secure Aggregation for Privacy-Preserving Federated Learning
• (Cai et al., 15 Jun 2025) VFEFL: Privacy-Preserving Federated Learning against Malicious Clients via Verifiable Functional Encryption
• (Gong et al., 2022) Federated Learning with Privacy-Preserving Ensemble Attention Distillation
• (Zhang et al., 2021) Privacy-Preserving Federated Learning on Partitioned Attributes
• (Xu et al., 2022) DeTrust-FL: Privacy-Preserving Federated Learning in Decentralized Trust Setting

This discipline continues to explore the intersection of statistical data analysis, cryptography, distributed systems, and adversarial robustness, with ongoing advancements targeting more rigorous, efficient, and flexible PPFL schemes across increasingly challenging data, device, and user populations.