AGI and Privacy
Personal data, surveillance, inference at scale, and the new privacy questions raised by capable AI systems that learn from everything.
Executive summary
Privacy in the AGI era is not only about what data is collected. It is also about what can be inferred from data. Capable models can derive sensitive attributes — health, sexuality, political views, mental state — from innocuous inputs. Existing privacy frameworks were designed for an earlier problem.
The inference shift
Data-protection law (GDPR, CCPA) regulates collection and processing of personal data. AI systems can produce sensitive personal data from inputs that contain none. This is the inference shift, and it strains every existing privacy framework.
New surfaces
- Voice assistants with persistent memory.
- Personalised AI companions.
- Workplace surveillance and productivity scoring.
- Public-space biometric inference.
- Cross-platform profile linking.
What is being done
The EU AI Act bans certain practices (untargeted facial-image scraping, real-time biometric identification with narrow exceptions). The U.S. FTC has acted against firms for AI-related privacy harms. Differential privacy, federated learning, and on-device processing are technical responses.
What helps
Strong defaults, data minimisation, on-device processing where possible, transparent inference logs, and personal data controls users can actually exercise.
Key takeaways
- 01Inference is the new privacy frontier.
- 02Existing law regulates collection more than inference.
- 03Technical and legal responses are both developing.
Frequently asked questions
Is my data used to train models?
Often, yes. Most major providers offer opt-outs; the quality and verifiability of those opt-outs varies.
Are voice assistants always listening?
Wake-word detection is on; full transcription typically begins after wake. Specifics vary by vendor; check the documentation.