PromptRiskDBThreat intelligence atlas
AI Risk

Training-related (Robust overfitting in adversarial training)

"Adversarial training can be affected by robust overfitting, where the model’s robustness on test data decreases during further training, particularly after the learning rate decay. This issue has been consistently observed across various datasets and algorithms in adversarial training settings [163, 230]. Robust over- fitting can affect the model’s ability to generalize effectively and reduce its resilience to ad...

AI Risk7. AI System Safety, Failures, & Limitations7.3 > Lack of capability or robustness1 - Pre-deployment

Record summary

A quick snapshot of what this page covers.

Techniques0Attack methods connected to this risk.
Mitigations0Defenses that may help with related attacks.
Domain7. AI System Safety, Failures, & LimitationsThe broad risk area this belongs to.

Risk profile

How this risk is described and categorized.

"Adversarial training can be affected by robust overfitting, where the model’s robustness on test data decreases during further training, particularly after the learning rate decay. This issue has been consistently observed across various datasets and algorithms in adversarial training settings [163, 230]. Robust over- fitting can affect the model’s ability to generalize effectively and reduce its resilience to adversarial attacks."

Domain7. AI System Safety, Failures, & Limitations
Subdomain7.3 > Lack of capability or robustness
Entity3 - Other
Intent2 - Unintentional
Timing1 - Pre-deployment
CategoryModel Development
SubcategoryTraining-related (Robust overfitting in adversarial training)

Suggested mitigations

Defenses that may help with related attacks.

No propagated mitigations. No defense is available through the connected attack methods.

Source

Research source for this risk, when available.