APromptRiskDBThreat intelligence atlas
AI Risk

Misuse

"The misuse of generative AI refers to any deliberate use that could result in harmful, unethical or inappropriate outcomes (Brundage et al., 2020). A prominent field that faces the threat of misuse is education. Cotton et al. (2023) have raised concerns over academic integrity in the era of ChatGPT. ChatGPT can be used as a high-tech plagiarism tool that identifies patterns from large corpora to generate content...

AI Risk4. Malicious Actors & Misuse4.3 > Fraud, scams, and targeted manipulation2 - Post-deployment

Record summary

A quick snapshot of what this page covers.

Techniques0Attack methods connected to this risk.
Mitigations0Defenses that may help with related attacks.
Domain4. Malicious Actors & MisuseThe broad risk area this belongs to.

Risk profile

How this risk is described and categorized.

"The misuse of generative AI refers to any deliberate use that could result in harmful, unethical or inappropriate outcomes (Brundage et al., 2020). A prominent field that faces the threat of misuse is education. Cotton et al. (2023) have raised concerns over academic integrity in the era of ChatGPT. ChatGPT can be used as a high-tech plagiarism tool that identifies patterns from large corpora to generate content (Gefen & Arinze, 2023). Given that generative AI such as ChatGPT can generate high-quality answers within seconds, unmotivated students may not devote time and effort to work on their assignments and essays. Hence, in the era of generative AI, the originality of the work done by students could be difficult to assess. Text written by ChatGPT is regarded as plagiarism and is not acceptable (Thorp, 2023). Another form of misuse is cheating in examinations. If students have access to digital devices during examinations, they can resort to using ChatGPT to assist them in answering the questions. To address potential misuse in education, AI-generated content detectors such as Turnitin could be used and strict proctoring measures will need to be deployed (Susnjak, 2022). However, the challenges go beyond content detection and examination proctoring as the line between what is considered appropriate versus inappropriate use of ChatGPT could be fuzzy."

Domain4. Malicious Actors & Misuse
Subdomain4.3 > Fraud, scams, and targeted manipulation
Entity1 - Human
Intent1 - Intentional
Timing2 - Post-deployment
CategoryEthical Concerns
SubcategoryMisuse

Suggested mitigations

Defenses that may help with related attacks.

No propagated mitigations. No defense is available through the connected attack methods.

Source

Research source for this risk, when available.