PromptRiskDBThreat intelligence atlas
AI Risk

Reinforcement learning AI (Training performance related)

"As above, there are broadly two dimensions of technical failure modes: quality of data or input signal, and training performance. Due to a lack of transparency, it may be difficult to ascertain the type of technical failure that gives rise to a particular risk, and it is often a combination of several factors. Risks pertain- ing to AI failures are exacerbated by poor quality training data and imperfect training s...

AI RiskX.1 > Excluded1 - Pre-deployment

Record summary

A quick snapshot of what this page covers.

Techniques1Attack methods connected to this risk.
Mitigations5Defenses that may help with related attacks.
Domainn/aThe broad risk area this belongs to.

Risk profile

How this risk is described and categorized.

"As above, there are broadly two dimensions of technical failure modes: quality of data or input signal, and training performance. Due to a lack of transparency, it may be difficult to ascertain the type of technical failure that gives rise to a particular risk, and it is often a combination of several factors. Risks pertain- ing to AI failures are exacerbated by poor quality training data and imperfect training signals. Various measures can be implemented to improve the quality of the training data, and fine-tuning techniques can be used to disincentivize harmful model behavior."

Domainn/a
SubdomainX.1 > Excluded
Entity3 - Other
Intent3 - Other
Timing1 - Pre-deployment
CategoryDimension - Technical Attributes (AI inadequacy - technical failure)
SubcategoryReinforcement learning AI (Training performance related)

Suggested mitigations

Defenses that may help with related attacks.

Sanitize Training Data

Business and Data UnderstandingData Preparation+1 more
LifecycleBusiness and Data Understanding + 2 moreCategoryTechnical - ML

Validate AI Model

ML Model EvaluationMonitoring and Maintenance
LifecycleML Model Evaluation + 1 moreCategoryTechnical - ML

Code Signing

Deployment
LifecycleDeploymentCategoryTechnical - Cyber

Source

Research source for this risk, when available.