APromptRiskDBThreat intelligence atlas
AI Risk

Social stereotypes and unfair discrmination

"Perpetuating harmful stereotypes and discrimination is a well-documented harm in machine learning models that represent natural language (Caliskan et al., 2017). LMs that encode discriminatory language or social stereotypes can cause different types of harm... Unfair discrimination manifests in differential treatment or access to resources among individuals or groups based on sensitive traits such as sex, religio...

AI Risk1. Discrimination & Toxicity1.1 > Unfair discrimination and misrepresentation3 - Other

Record summary

A quick snapshot of what this page covers.

Techniques0Attack methods connected to this risk.
Mitigations0Defenses that may help with related attacks.
Domain1. Discrimination & ToxicityThe broad risk area this belongs to.

Risk profile

How this risk is described and categorized.

"Perpetuating harmful stereotypes and discrimination is a well-documented harm in machine learning models that represent natural language (Caliskan et al., 2017). LMs that encode discriminatory language or social stereotypes can cause different types of harm... Unfair discrimination manifests in differential treatment or access to resources among individuals or groups based on sensitive traits such as sex, religion, gender, sexual orientation, ability and age."

Domain1. Discrimination & Toxicity
Subdomain1.1 > Unfair discrimination and misrepresentation
Entity2 - AI
Intent2 - Unintentional
Timing3 - Other
CategoryDiscrimination, Exclusion and Toxicity
SubcategorySocial stereotypes and unfair discrmination

Suggested mitigations

Defenses that may help with related attacks.

No propagated mitigations. No defense is available through the connected attack methods.

Source

Research source for this risk, when available.