gcdRadar
archive/Research/AIR-2023-0001
closedclosed 15 Mar 2023Research1 Research / proof of conceptAI

Code Execution in MathGPT via Prompt Injection

MathGPT, GPT-3

Researchers demonstrated that prompt injection against the MathGPT application could steer GPT-3 into generating unauthorized executable Python code. The demonstration highlighted risks in applications that directly execute model-generated code.

first report
1 Mar 2023
last update
1 Mar 2023

Timeline · 5

  1. 1 Mar 2023Achieving Code Execution in MathGPT via Prompt Injection
  2. 1 Mar 2023Measuring Mathematical Problem Solving With the MATH Dataset
  3. 1 Mar 2023Training Verifiers to Solve Math Word Problems
  4. 1 Mar 2023Reverse Prompt Engineering for Fun and (no) Profit
  5. 1 Mar 2023Exploring prompt-based attacks

Sources · 5

News and write-ups

  1. 2023-03-01
    Reverse Prompt Engineering for Fun and (no) Profit
    lspace.swyx.io · lspace.swyx.io/p/reverse-prompt-eng
  2. 2023-03-01
    Exploring prompt-based attacks
    research.nccgroup.com · research.nccgroup.com/2022/12/05/exploring-prompt-injection-attacks

Research and catalogues

  1. 2023-03-01
    Achieving Code Execution in MathGPT via Prompt Injection
    MITRE ATLAS · atlas.mitre.org/studies/AML.CS0016
  2. 2023-03-01
    Measuring Mathematical Problem Solving With the MATH Dataset
    arXiv · arxiv.org/abs/2103.03874
  3. 2023-03-01
    Training Verifiers to Solve Math Word Problems
    arXiv · arxiv.org/abs/2110.14168

Titles and links belong to their publishers. The summary above is written by gcdRadar in its own words; data is CC BY-SA 4.0.

Classification

lane
Research
AI role
Attack on AI
categories
Prompt injectionAgent hijack
severity
1 · Research / proof of concept
targets
OpenAI, Enterprise deployment
owasp llm top 10
LLM01:2025 Prompt InjectionLLM06:2025 Excessive Agency
mitre atlas
AML.T0051 LLM Prompt InjectionAML.T0053 AI Agent Tool Invocation
tags
prompt-injectioncode-execution
dates
happened 28 Jan 2023 · first reported 1 Mar 2023
origin
MITRE ATLAS
Derived from MITRE ATLAS (Apache 2.0, © MITRE).
record
confidence 0.95 · created 2026-10-18 06:55Z · updated 2026-10-18 06:55Z

Lane, AI role, categories and severity come from the classifier; the OWASP and ATLAS rows are derived from the categories. How these are assigned

Copy a brief

This record as a short Markdown brief, for a ticket, a chat or a coding agent.

the brief as text
# Code Execution in MathGPT via Prompt Injection

https://gcdradar.vercel.app/incidents/code-execution-in-mathgpt-via-prompt-injection · AIR-2023-0001 · closed 2023-03-15

- Lane: Research
- Severity: 1 (Research / proof of concept)
- Fix status: Fix status unknown
- First reported: 2023-03-01 (happened 2023-01-28)
- AI role: Attack on AI
- Products: MathGPT, GPT-3

## Summary

Researchers demonstrated that prompt injection against the MathGPT application could steer GPT-3 into generating unauthorized executable Python code. The demonstration highlighted risks in applications that directly execute model-generated code.

## What to do

No action recorded; read the sources below and the vendor's advisory.

## Latest updates

- 2023-03-01: Exploring prompt-based attacks (https://research.nccgroup.com/2022/12/05/exploring-prompt-injection-attacks)
- 2023-03-01: Reverse Prompt Engineering for Fun and (no) Profit (https://lspace.swyx.io/p/reverse-prompt-eng)
- 2023-03-01: Training Verifiers to Solve Math Word Problems (https://arxiv.org/abs/2110.14168)
- 2023-03-01: Measuring Mathematical Problem Solving With the MATH Dataset (https://arxiv.org/abs/2103.03874)
- 2023-03-01: Achieving Code Execution in MathGPT via Prompt Injection (https://atlas.mitre.org/studies/AML.CS0016)

## Sources

- 2023-03-01 MITRE ATLAS: Achieving Code Execution in MathGPT via Prompt Injection https://atlas.mitre.org/studies/AML.CS0016
- 2023-03-01 arXiv: Measuring Mathematical Problem Solving With the MATH Dataset https://arxiv.org/abs/2103.03874
- 2023-03-01 arXiv: Training Verifiers to Solve Math Word Problems https://arxiv.org/abs/2110.14168
- 2023-03-01 lspace.swyx.io: Reverse Prompt Engineering for Fun and (no) Profit https://lspace.swyx.io/p/reverse-prompt-eng
- 2023-03-01 research.nccgroup.com: Exploring prompt-based attacks https://research.nccgroup.com/2022/12/05/exploring-prompt-injection-attacks

Verify against the vendor's advisory before changing production. Data CC BY-SA 4.0, https://gcdradar.vercel.app.

Related

  1. closed
    ChatGPT LLM Application Frameworks Allow Remote Code Execution via Prompting

    Research · since 26 Mar 2023 · updated 31 Mar 2023

    AI
  2. closed
    GPT-3 Remoteli.io GPT-3 Twitter bot hijacked via prompt injection

    Patch now · Exploited · since 15 Sep 2022

    AI
  3. closed
    Bing Chat Bing Chat Data Exfiltration via Indirect Prompt Injection

    Research · since 2023 (approx.)

    AI
  4. closed
    Bing Chat Bing Chat Initial Instructions Exposed via Prompt Injection by Testers

    Research · since 8 Feb 2023

    AI
  5. closed
    ChatGPT ChatGPT Conversation Exfiltration via Indirect Prompt Injection

    Research · since May 2023 (approx.)

    AI
  6. closed
    ChatGPT Samsung Source Code and Meeting Records Disclosed via ChatGPT Ingestion

    Breaches & attacks · Real loss · since 11 Mar 2023

    AI