Disparate Performance
AI RiskThe LLM’s performances can differ significantly across different groups of users. For example, the question-answering capability showed significant performance differences across different racial and social status groups. The fact-checking abilities can differ for different tasks and languages
Overview
A source-backed snapshot of this AI risk.
Risk profile
How the MIT AI Risk Repository categorizes this risk.
Merged risk records
Source records unified into this canonical risk concept.
MITRISK-Liu2024-30.03.04 - Disparate Performance
The LLM’s performances can differ significantly across different groups of users. For example, the question-answering capability showed significant performance differences across different racial and social status groups. The fact-checking abilities can differ for different tasks and languages
MITRISK-Solaiman2023-13.01.03 - Disparate Performance
"In the context of evaluating the impact of generative AI systems, disparate performance refers to AI systems that perform differently for different subpopulations, leading to unequal outcomes for those groups."
Mitigations
Defenses that may help with related attacks.
Source evidence
Research source for this risk, when available.
Included resource
Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignment
Original source
MIT AI Risk Repository
Open the public repository used for AI risk records and taxonomy fields.