PoisonGPT Open Source Model Supply Chain Demonstration

# PoisonGPT Open Source Model Supply Chain Demonstration

https://gcdradar.vercel.app/incidents/poisongpt-open-source-model-supply-chain-demonstration · AIR-2023-0004 · closed 2023-11-13

- Lane: Research
- Severity: 1 (Research / proof of concept)
- Fix status: Fix status unknown
- First reported: 2023-10-30 (happened 2023-07)
- AI role: Attack on AI
- Products: HuggingFace, LLM

## Summary

Researchers demonstrated that an open-source language model could be poisoned and uploaded to a major model hub. The proof-of-concept highlighted supply chain risks within public model repositories.

## What to do

No action recorded; read the sources below and the vendor's advisory.

## Latest updates

- 2023-10-30: PoisonGPT: How we hid a lobotomized LLM on Hugging Face to spread fake news (https://blog.mithrilsecurity.io/poisongpt-how-we-hid-a-lobotomized-llm-on-hugging-face-to-spread-fake-news)
- 2023-10-30: PoisonGPT (https://atlas.mitre.org/studies/AML.CS0019)

## Sources

- 2023-10-30 MITRE ATLAS: PoisonGPT https://atlas.mitre.org/studies/AML.CS0019
- 2023-10-30 blog.mithrilsecurity.io: PoisonGPT: How we hid a lobotomized LLM on Hugging Face to spread fake news https://blog.mithrilsecurity.io/poisongpt-how-we-hid-a-lobotomized-llm-on-hugging-face-to-spread-fake-news

Verify against the vendor's advisory before changing production. Data CC BY-SA 4.0, https://gcdradar.vercel.app.