Synthetic media is any image, audio, video or text that has been generated or significantly altered using artificial intelligence. It includes AI-generated pictures, synthetic voices, AI avatars and deepfakes, which are realistic fakes of real people. Synthetic media enables new forms of creativity and communication, but it also makes deception, fraud and misinformation easier.
Key takeaways
- Synthetic media covers both fully generated and AI-altered content.
- Deepfakes are a subset: realistic synthetic media depicting real people doing or saying things they did not.
- Legitimate uses include film, education, accessibility, localisation and marketing.
- Harms include fraud, non-consensual imagery, misinformation and reputational damage.
- Defences combine detection, watermarking, content provenance, platform labels and law.
Synthetic media explained in simple terms
Photo editing has existed for decades, but it took skill and time. AI changed that. Anyone can now create a realistic photo of an event that never happened, a video of a person saying words they never said, or a voice message in someone else’s voice. Synthetic media is the name for all of this AI-made or AI-changed content, whether used honestly or deceptively.
Types of synthetic media
| Type | Examples | Related technology |
|---|---|---|
| Images | AI art, product scenes, fake photos | AI image generation |
| Video | AI clips, avatars, face swaps | AI video generation |
| Audio | Synthetic voices, cloned voices, AI music | AI voice generation, AI music generation |
| Text | AI-written articles and posts | AI-generated content |
| Mixed | Dubbed videos with lip-sync | Multimodal models |
Deepfakes
A deepfake is synthetic media that convincingly shows a real person saying or doing something they did not. The term emerged around 2017. Early deepfakes relied heavily on GANs and face-swapping autoencoders; newer ones also use diffusion and advanced voice cloning. Harms include:
- Fraud: cloned executive voices or video calls used to request payments.
- Non-consensual intimate imagery, which overwhelmingly targets women.
- Political misinformation around elections.
- Reputational harm and harassment.
- The “liar’s dividend”: real evidence dismissed as fake because fakes exist.
Legitimate uses
- Film and TV: de-ageing, visual effects, dubbing.
- Education and museums: bringing historical scenes to life, clearly labelled.
- Accessibility: voice banking for people losing their ability to speak.
- Business: training videos, localisation and product visuals.
- Privacy: synthetic data or faces that protect real identities in research.
How synthetic media is detected and labelled
| Method | How it works | Limits |
|---|---|---|
| Detection tools | Models look for statistical artefacts of generation | Accuracy drops on new generators and compressed media |
| Invisible watermarking | Hidden signals added at generation, such as Google DeepMind’s SynthID | Only works if the generator adds it; can be weakened by editing |
| Content provenance | Signed metadata records how media was made and edited, using the C2PA standard | Metadata can be stripped; needs wide adoption |
| Platform labels | Platforms label AI content, based on disclosure or detection | Depends on honest disclosure and detection |
| Human checks | Source verification, reverse image search, context | Time-consuming |
A worked example: verifying a viral clip
A video appears showing a CEO announcing layoffs. Before sharing it, a journalist:
- Checks the company’s official channels for the announcement.
- Looks for content credentials or platform labels.
- Runs a reverse image search on key frames to find the original source.
- Examines lip-sync, lighting and audio for inconsistencies.
- Contacts the company directly.
No single check is decisive; together they build confidence.
Law and regulation
The EU AI Act requires people who deploy AI to create deepfakes to disclose that the content is artificially generated or manipulated, and requires AI providers to mark synthetic outputs in machine-readable ways. Many countries and US states have laws targeting election deepfakes and non-consensual intimate imagery. Platforms also enforce their own labelling policies.
Limitations of current defences
- Detection is an arms race and often lags behind new generators.
- Watermarks and metadata can be removed.
- Open models may not include any marking.
- Labels can reduce, but not eliminate, harm once content spreads.