Privacy by Design
in the machine learning pipeline
Workshop, University of Venice
2024-11-01
Machine Learning Pipeline
Machine Learning Pipeline Life Cycle
Reality is much messier. See Gelman and Loken (2013) for a discussion of the implications.
This workshop
- Work through the ML life cycle
- How could and have things gone wrong
- What tools do we have?
- What are their limitations?
- Interactive bits, every now and then
Snowball Sampling - Initial Recruitment
Snowball Sampling - Referral Round 1
Snowball Sampling - Referral Round 2
Snowball Sampling - Referral Round 3
Snowball Sampling
By using subject-driven sampling design, we can better explore the hard to reach target population while preserving the privacy of data subjects who do not want to be included in the study.
- Bonus: Also allows us to study community (network) structure if we are interested in that.
- Drawbacks:
- “isolated” nodes,
- partitioned graphs,
- adapting model fitting to non-uniform sampling.
Estimation from Randomised Response Data
Consider using a weighted coin, probability \(p\) of telling truth. Derive an expression for the probability of answering “Yes”.
\[\begin{align*} \Pr(\text{Yes}) &= \theta p + (1 - \theta)(1 - p) \\ & \approx \frac{\#\{yes\}}{\#\{subjects\}} \\ &= \bar y \end{align*}\]
Rearrange this expression to get a formula for \(\hat \theta\).
\[\hat \theta = \frac{\bar y - 1 + p}{2p -1}.\]
Randomised Response - Privacy Schematic
Encryption
Once we have gone to the effort of collecting data we don’t want to just leave it lying around for anyone to access.
\[ \text{Plain text} \overset{f}{\rightarrow} \text{Cipher Text}\]
\[ \text{Cipher text} \overset{f^{-1}}{\rightarrow} \text{Plain Text}\]
\[ f(\text{data}, \text{key})\] Many encryption schemes depending on the data to be encrypted and how the key is to be distributed.
Caesar Cipher
Have a go at decrypting the encrypted message f(?, 2 ) = JGNNQ YQTNF.
What are some benefits and drawbacks of this encryption scheme?
K-anonymity Example
K-anonymity Worked Example
Centralised Federation
Clustered Federated Learning
Personalised Federated Learning
Directions of Federation
Horizontal Federation
Vertical Federation
Privacy through modelling constraints
The core idea of differential privacy is that we want to ensure that no individual data point influences the model “too much”.
Strategies:
Add noise to individual data entries or summary statistics. Similar to robust regression (median modelling, heavy-tailed noise)
Combine goodness of fit and sensitivity penalty in loss function. Similar to penalised regression.
\[ L(\theta, x; \lambda) = \underset{\text{likelihood /} \\ \text{model fit}}{\underbrace{\ell(\theta, x)}} - \lambda \underset{\text{sensitivity} \\ \text{penality}}{\underbrace{h(\theta, x)}}\]
Federated Learning
Similar idea to federated analytics but communicating e.g. gradient of loss function using local data.
This means that data stays with user, only partial model updates are transmitted.
Strong links to distributed computing and this is how Apple (and others) collect performance data from phones.
Have to take care with non-responses so as not to bias the model.
Privacy of the model
Putting a model into production exposes a whole range of adversarial circumstances. Two of the most common would be:
- Attacks aiming to recover model structure
- to reproduce the model
- to exploit weaknesses
- Attacks aiming to recover individual training data
- Examples would be financial decision making and LLMs.
Rigaki, M. & Garcia, S. (2020). “A Survey of Privacy Attacks in Machine Learning”. ArXiv preprint.
Learning More
References
Gelman, Andrew, and Eric Loken. 2013. “The Garden of Forking Paths: Why Multiple Comparisons Can Be a Problem, Even When There Is No ‘Fishing Expedition’ or ‘p-Hacking’ and the Research Hypothesis Was Posited Ahead of Time.” Department of Statistics, Columbia University 348 (1-17): 3.