Quantifying Membership Privacy Risk Using Shadow Model Membership Inference Attack

Privacy regulations restrict sharing sensitive patient data across institutions, hindering collaborative medical research. Federated learning (FL) addresses this by keeping data local to the acquiring institution, but Membership Inference Attacks (MIAs) remain feasible even without direct access to training data. Differential privacy (DP) provides formal mathematical guarantees, yet translating its abstract ε parameter into concrete risk assessments that clinicians can interpret remains an open challenge. In this study, we quantify membership privacy risk from the attacker’s perspective by simulating a malicious FL participant who uses a fixed auxiliary data pool to attack target models via Shadow Model–based MIA. We instantiate this scenario in the EMBED breast cancer screening dataset, based on which we trained 300 ConvNeXt-Tiny models on binary classification without and with DP (ε ∈ {3, 5, 8, 10}) with data level variance. Non-DP models exhibit consistent membership leakage with a mean ROC-AUC of 0.661 across training configurations, with the rare malignant class substantially more vulnerable (malignant-only AUC up to 0.938). DP training collapses malignant-only AUC to 0.500–0.574 across all privacy budgets, with effective protection achieved at ε = 10. We further establish that 60 shadow models suffice for stable Likelihood Ratio Attack (LiRA) calibration, equivalent to ∼60× the computation of a single target model. Our results provide an empirically grounded visualization of membership privacy risk and demonstrate, based on stable LiRA calibration, that moderate privacy budgets offer meaningful protection against inference attacks in medical imaging FL.