LLM Application Frameworks Allow Remote Code Execution via Prompting

# LLM Application Frameworks Allow Remote Code Execution via Prompting

https://gcdradar.vercel.app/incidents/llm-application-frameworks-allow-remote-code-execution-via-p · AIR-2023-0005 · closed 2023-04-09

- Lane: Research
- Severity: 1 (Research / proof of concept)
- Fix status: Fix status unknown
- First reported: 2023-03-26
- AI role: Attack on AI
- Products: ChatGPT, langchain, boxcars.ai

## Summary

Frameworks like langchain and boxcars.ai allow direct query execution through LLMs, enabling remote code execution or SQL injection via prompt inputs. Researchers demonstrated that unauthorized actions could be triggered through conversational input.

## What to do

No action recorded; read the sources below and the vendor's advisory.

## Latest updates

- 2023-03-31: AVID-2023-V028: Frameworks like langchain (Python) and boxcars.ai (Ruby) offer apps and scripts to directly execute queries through LLMs as a built-in feature. (https://github.com/avidml/avid-db/blob/main/vulnerabilities/2023/AVID-2023-V028.json)
- 2023-03-26: Reddit thread on InjectGPT (https://reddit.com/r/netsec/comments/121gpay/injectgpt_remote_code_execution_by_asking_nicely)
- 2023-03-26: InjectGPT: the most polite exploit ever (https://blog.luitjes.it/posts/injectgpt-most-polite-exploit-ever)
- 2023-03-26: AVID-2023-R0003: Frameworks like langchain (Python) and boxcars.ai (Ruby) offer apps and scripts to directly execute queries through LLMs as a built-in feature. (https://github.com/avidml/avid-db/blob/main/reports/2023/AVID-2023-R0003.json)

## Sources

- 2023-03-26 AVID: AVID-2023-R0003: Frameworks like langchain (Python) and boxcars.ai (Ruby) offer apps and scripts to directly execute queries through LLMs as a built-in feature. https://github.com/avidml/avid-db/blob/main/reports/2023/AVID-2023-R0003.json
- 2023-03-26 blog.luitjes.it: InjectGPT: the most polite exploit ever https://blog.luitjes.it/posts/injectgpt-most-polite-exploit-ever
- 2023-03-26 reddit.com: Reddit thread on InjectGPT https://reddit.com/r/netsec/comments/121gpay/injectgpt_remote_code_execution_by_asking_nicely
- 2023-03-31 AVID: AVID-2023-V028: Frameworks like langchain (Python) and boxcars.ai (Ruby) offer apps and scripts to directly execute queries through LLMs as a built-in feature. https://github.com/avidml/avid-db/blob/main/vulnerabilities/2023/AVID-2023-V028.json

Verify against the vendor's advisory before changing production. Data CC BY-SA 4.0, https://gcdradar.vercel.app.