PromptRiskDBThreat intelligence atlas

Illegitimate surveillance and censorship

AI Risk

"The collection of large amounts of information about people for the purpose of mass surveillance has raised ethical and social concerns, including risk of censorship and of undermining public discourse (Cyphers and Gebhart, 2019; Stahl, 2016; Véliz, 2019). Sifting through these large datasets previously required millions of human analysts (Hunt and Xu, 2013), but is increasingly being automated using AI (Andersen...

Overview

A source-backed snapshot of this AI risk.

"The collection of large amounts of information about people for the purpose of mass surveillance has raised ethical and social concerns, including risk of censorship and of undermining public discourse (Cyphers and Gebhart, 2019; Stahl, 2016; Véliz, 2019). Sifting through these large datasets previously required millions of human analysts (Hunt and Xu, 2013), but is increasingly being automated using AI (Andersen, 2020; Shahbaz and Funk, 2019)."

Techniques0Attack methods connected to this risk.
Mitigations0Defenses that may help with related attacks.
Records2Source records unified into this concept.

Risk profile

How the MIT AI Risk Repository categorizes this risk.

Domain4. Malicious Actors & Misuse
Subdomain4.1 > Disinformation, surveillance, and influence at scale
Entity1 - Human
Intent1 - Intentional
Timing2 - Post-deployment
CategoryMalicious Uses; Risk area 4: Malicious Uses
SubcategoryIllegitimate surveillance and censorship

Merged risk records

Source records unified into this canonical risk concept.

2 recordsView all →

MITRISK-Weidinger2021-17.04.04 - Illegitimate surveillance and censorship

"The collection of large amounts of information about people for the purpose of mass surveillance has raised ethical and social concerns, including risk of censorship and of undermining public discourse (Cyphers and Gebhart, 2019; Stahl, 2016; Véliz, 2019). Sifting through these large datasets previously required millions of human analysts (Hunt and Xu, 2013), but is increasingly being automated using AI (Andersen, 2020; Shahbaz and Funk, 2019)."

Domain4. Malicious Actors & MisuseSubdomain4.1 > Disinformation, surveillance, and influence at scaleSourceEthical and social risks of harm from language modelsYear2021

MITRISK-Weidinger2022-16.04.04 - Illegitimate surveillance and censorship

Anticipated risk: "Mass surveillance previously required millions of human analysts [83], but is increasingly being automated using machine learning tools [7, 168]. The collection and analysis of large amounts of information about people creates concerns about privacy rights and democratic values [41, 173,187]. Conceivably, LMs could be applied to reduce the cost and increase the efficacy of mass surveillance, thereby amplifying the capabilities of actors who conduct mass surveillance, including for illegitimate censorship or to cause other harm."

Domain4. Malicious Actors & MisuseSubdomain4.1 > Disinformation, surveillance, and influence at scaleSourceTaxonomy of Risks posed by Language ModelsYear2022

Mitigations

Defenses that may help with related attacks.

No propagated mitigations. No defense is available through the connected attack methods.

Source evidence

Research source for this risk, when available.