closedclosed 28 Mar 2025Research1 Research / proof of conceptAI
ChatGPT Package Hallucination Supply Chain Risk
ChatGPT
Researchers highlighted that LLMs can hallucinate non-existent package names, which attackers could register to distribute malicious code. Developers following the generated advice risk downloading compromised packages.
- first report
- 14 Mar 2025
- last update
- 14 Mar 2025
Timeline · 5
- 14 Mar 2025ChatGPT Package Hallucination
- 14 Mar 2025Vulcan18's "Can you trust ChatGPT's package recommendations?"
- 14 Mar 2025Lasso Security Research: Diving into AI Package Hallucinations
- 14 Mar 2025AIID Incident 731: Hallucinated Software Packages with Potential Malware Downloaded Thousands of Times by Developers
- 14 Mar 2025Slopsquatting: When AI Agents Hallucinate Malicious Packages
Sources · 5
News and write-ups
- 2025-03-14Vulcan18's "Can you trust ChatGPT's package recommendations?"vulcan.io · vulcan.io/blog/ai-hallucinations-package-risk
- 2025-03-14Lasso Security Research: Diving into AI Package Hallucinationslasso.security · lasso.security/blog/ai-package-hallucinations
- 2025-03-14Slopsquatting: When AI Agents Hallucinate Malicious PackagesTrend Micro · trendmicro.com/vinfo/us/security/news/cybercrime-and-digital-threats/slopsquatti
Research and catalogues
- 2025-03-14ChatGPT Package HallucinationMITRE ATLAS · atlas.mitre.org/studies/AML.CS0022
- 2025-03-14AIID Incident 731: Hallucinated Software Packages with Potential Malware Downloaded Thousands of Times by DevelopersAI Incident Database · incidentdatabase.ai/cite/731
Titles and links belong to their publishers. The summary above is written by gcdRadar in its own words; data is CC BY-SA 4.0.
Classification
- lane
- Research
- AI role
- AI-enabled attack
- categories
- Supply chainOther
- severity
- 1 · Research / proof of concept
- targets
- OpenAI, Framework / tooling
- owasp llm top 10
- no entry for these categories
- mitre atlas
- no technique for these categories
- dates
- happened Jun 2024 (approx.) · first reported 14 Mar 2025
- origin
- MITRE ATLASDerived from MITRE ATLAS (Apache 2.0, © MITRE).
- record
- confidence 0.90 · created 2026-10-18 06:55Z · updated 2026-10-18 06:55Z
Lane, AI role, categories and severity come from the classifier; the OWASP and ATLAS rows are derived from the categories. How these are assigned
Copy a brief
This record as a short Markdown brief, for a ticket, a chat or a coding agent.
the brief as text
# ChatGPT Package Hallucination Supply Chain Risk https://gcdradar.vercel.app/incidents/chatgpt-package-hallucination-supply-chain-risk · AIR-2025-0052 · closed 2025-03-28 - Lane: Research - Severity: 1 (Research / proof of concept) - Fix status: Fix status unknown - First reported: 2025-03-14 (happened 2024-06) - AI role: AI-enabled attack - Products: ChatGPT ## Summary Researchers highlighted that LLMs can hallucinate non-existent package names, which attackers could register to distribute malicious code. Developers following the generated advice risk downloading compromised packages. ## What to do No action recorded; read the sources below and the vendor's advisory. ## Latest updates - 2025-03-14: Slopsquatting: When AI Agents Hallucinate Malicious Packages (https://trendmicro.com/vinfo/us/security/news/cybercrime-and-digital-threats/slopsquatting-when-ai-agents-hallucinate-malicious-packages) - 2025-03-14: AIID Incident 731: Hallucinated Software Packages with Potential Malware Downloaded Thousands of Times by Developers (https://incidentdatabase.ai/cite/731) - 2025-03-14: Lasso Security Research: Diving into AI Package Hallucinations (https://lasso.security/blog/ai-package-hallucinations) - 2025-03-14: Vulcan18's "Can you trust ChatGPT's package recommendations?" (https://vulcan.io/blog/ai-hallucinations-package-risk) - 2025-03-14: ChatGPT Package Hallucination (https://atlas.mitre.org/studies/AML.CS0022) ## Sources - 2025-03-14 MITRE ATLAS: ChatGPT Package Hallucination https://atlas.mitre.org/studies/AML.CS0022 - 2025-03-14 vulcan.io: Vulcan18's "Can you trust ChatGPT's package recommendations?" https://vulcan.io/blog/ai-hallucinations-package-risk - 2025-03-14 lasso.security: Lasso Security Research: Diving into AI Package Hallucinations https://lasso.security/blog/ai-package-hallucinations - 2025-03-14 AI Incident Database: AIID Incident 731: Hallucinated Software Packages with Potential Malware Downloaded Thousands of Times by Developers https://incidentdatabase.ai/cite/731 - 2025-03-14 Trend Micro: Slopsquatting: When AI Agents Hallucinate Malicious Packages https://trendmicro.com/vinfo/us/security/news/cybercrime-and-digital-threats/slopsquatting-when-ai-agents-hallucinate-malicious-packages Verify against the vendor's advisory before changing production. Data CC BY-SA 4.0, https://gcdradar.vercel.app.
Related
- closedChatGPT OpenAI Model Bypassed to Generate Fraudulent Receipts and Prescriptions
Research · since 31 Mar 2025
AI - closedAI Overviews AI Chatbots Reportedly Cited Fraudulent Cruise Hotline Enabling Scams
Breaches & attacks · Exploited · since 15 Aug 2025
AI - closedChatGPT Suspect Used ChatGPT to Assess Drug Lethality Before Fatal Poisonings
Breaches & attacks · Real loss · since 28 Jan 2026
AI - closedChatGPT Hacking ChatGPT's Memories with Prompt Injection
Research · since Feb 2024 (approx.)
AI - closedAWS Bedrock Threat Actors Hijack Cloud-Hosted LLMs via Stolen Credentials
Patch now · Exploited · since 6 May 2024
AI - liveOpenAI Codex OpenAI Codex Jailbreak Toolkit Hides Prompt Adware to Hijack AI Agents
Research · since 4 Nov 2026
AI