Rényi Pufferfish Privacy with Gaussian-based Priors: From Single Gaussian to Mixture Model

Abstract: Rényi Pufferfish Privacy (RPP) provides a Rényi divergence-based privacy framework for correlated data, but existing $\infty$-Wasserstein mechanisms are often conservative and sacrifice data utility. We study Gaussian mechanisms for RPP under Gaussian and Gaussian-mixture priors. For single Gaussian priors, we derive the exact Rényi divergence after Gaussian perturbation, obtain a relaxed closed-form sufficient condition for $(α,ε)$-RPP, and characterize the monotonicity of the calibrated noise with respect to the privacy budget $ε$ and the Rényi order $α$. To handle more general non-Gaussian and multimodal priors, we approximate secret-conditioned outputs with Gaussian mixture models and introduce an optimal-transport-based sufficient condition for RPP. Experiments on three UCI datasets with statistical (\textsc{RAW}, \textsc{MEAN}) and model-output (\textsc{BNN}, \textsc{GP}) queries show that our prior-aware mechanisms consistently require less noise than a recent RPP additive-noise baseline, achieving an average noise reduction of 48.9\%. These results show that our mechanisms can substantially improve the privacy-utility trade-off under RPP.

• The paper formulates exact noise calibration conditions for achieving (α, ε)-Rényi Pufferfish Privacy using single Gaussian priors with closed-form solutions.
• It extends the mechanism to handle multimodal, real-world data by employing Gaussian mixture models and optimal-transport techniques.
• Empirical evaluations on UCI datasets demonstrate up to 48.9% noise reduction while maintaining robust privacy against adversaries exploiting data correlations.

Rényi Pufferfish Privacy with Gaussian-based Priors: From Single Gaussian to Mixture Model

Introduction and Motivation

This paper addresses the limitations of Differential Privacy (DP) in scenarios with correlated data by advancing mechanism design under Rényi Pufferfish Privacy (RPP). While DP relies on neighboring-database constructions and assumes independence between sensitive attributes and query outputs, this assumption does not hold in numerous real-world settings. Adversaries aware of correlations or prior information can conduct stronger inference attacks, compromising privacy. Pufferfish Privacy (PP) formalizes privacy guarantees that are robust to such correlations through bounding statistical distinguishability between outputs conditioned on sensitive secrets. RPP extends this further by utilizing the Rényi divergence to capture finer privacy risk granularity.

Existing RPP mechanisms, notably those leveraging infinite-Wasserstein ($W_\infty$) distances, are typically conservative, introducing excessive noise and harming data utility. The present work studies RPP with Gaussian and Gaussian mixture priors, both due to their prevalence in statistical modeling (e.g., Bayesian methods, generative models) and their utility for principled mechanism design. The authors derive new mechanisms for both unimodal (single Gaussian) and multimodal (Gaussian mixture) secrets, providing both exact and relaxed (closed-form and transport-based) sufficient conditions for $(\alpha, \epsilon)$-RPP, and demonstrate substantial utility improvements over state-of-the-art RPP noise mechanisms.

RPP with Single Gaussian Priors

The authors first consider the setting where the secret-conditioned priors are univariate Gaussians. They derive the exact Rényi divergence following the addition of Gaussian noise, yielding a mechanism that is parameterized by the prior means and variances, the noise variance, and $(\alpha, \epsilon)$.

The key condition for a randomized mechanism's output distribution to satisfy $(\alpha,\epsilon)$-RPP is:

$$\frac{\alpha (\mu_i - \mu_j)^2}{2 (\theta^2 + (1 - \alpha)\sigma_i^2 + \alpha\sigma_j^2)} + \frac{1}{2(1-\alpha)} \ln \frac{\theta^2 + (1-\alpha)\sigma_i^2 + \alpha\sigma_j^2}{(\theta^2 + \sigma_i^2)^{1-\alpha}(\theta^2 + \sigma_j^2)^\alpha} \leq \epsilon,$$

so that the calibrated noise $\theta^2$ is the minimal value for which this holds for all discriminative secret pairs.

The authors rigorously establish the monotonicity: The privacy loss is non-increasing in noise variance, so binary search provides an efficient root-finding procedure. For the symmetric case ($\sigma_i^2 = \sigma_j^2$) an analytic formula is derived, and for the general asymmetric setting a tight quadratic upper bound yields an explicit closed-form sufficient condition for calibration.

Monotonicity results prove that the required noise decreases strictly with the privacy budget $\epsilon$. The behavior with respect to Rényi order $\alpha$ is nontrivial and characterized: if mean differences dominate, noise requirements increase with $\alpha$; if variance differences dominate, they decrease.

Extension to Gaussian Mixture Priors

Recognizing that real-world data are often non-Gaussian or exhibit multimodality, the mechanism is generalized to Gaussian mixture priors. Each secret-conditioned distribution is approximated by a GMM, with the number of components selected using BIC-based model selection on empirical query samples.

Figure 2: Fitted GMM densities, with solid lines showing estimated mixtures and bars showing empirical distributions, on three real-world datasets (demographic, educational, medical).

To calibrate noise, the mechanism constructs an optimal-transport-based sufficient privacy condition. The component matching cost utilizes the 2-Wasserstein distance, and a closed-form for the Rényi divergence between output mixtures is derived using a hierarchical transport plan. The resulting sufficient condition ensures $(\alpha, \epsilon)$0-RPP for arbitrary multimodal priors. The complexity of calibration remains practical due to the GMM and transport structure.

Empirical Evaluation

The mechanisms are evaluated on three UCI datasets: Adult (demography), Heart Disease (medicine), and Student Performance (education). Released attributes are chosen to represent practical privacy risks (e.g., releasing education while protecting race or medical measures). Four types of queries are tested: direct (Raw), aggregate (Mean), and two model-output queries (Bayesian neural networks and Gaussian processes).

Across all tasks and privacy regimes, the proposed prior-aware mechanisms require substantially less Gaussian noise than the additive $(\alpha, \epsilon)$1-based baseline. Average noise reductions are quantified as 48.9%. Monotonicity results are empirically validated over sweeps in privacy budget and Rényi order, and the utility advantage persists for both unimodal and multimodal queries.

Theoretical and Practical Implications

This work provides a formally justified, practically efficient calibration rule for RPP in realistic statistical settings. Gaussian prior calibration closes the gap between strict theoretical privacy and practical data utility, without relaxing privacy guarantees. The GMM extension covers arbitrary continuous distributions relevant to Bayesian and deep generative modeling. The optimal-transport mechanism leverages structure in mixture models to maintain computational tractability.

From a theoretical perspective, the results characterize how privacy-utility trade-offs depend on secret-conditioned mean and variance, and provide analytic or near-analytic privacy calibrations, advancing understanding of privacy for correlated and structured data. Practically, these results suggest concrete approaches for privacy-preserving data release in domains where adversarial inference exploits distributional structure.

Future Directions

Potential future work includes:

• Extending calibration and utility bounds to multivariate/matrix-valued queries.
• Incorporating richer adversarial priors, including heavy tails or discrete-continuous mixtures.
• Analyzing privacy-utility composition and amplification properties.
• Adapting these results to private learning (e.g., federated learning) with structured model output spaces.

Conclusion

By developing new theoretical and computational tools for RPP with Gaussian and GMM priors, this paper advances both the privacy theory and utility of mechanisms for statistically correlated data. Closed-form and optimal-transport-based calibrations achieve strong privacy with lower noise, significantly improving over state-of-the-art baselines. These mechanisms are directly applicable to real-world settings where distributional structure cannot be ignored, setting a new benchmark for practical, robust data privacy under correlation-aware adversaries.