
A tool named PoCGen is revolutionizing how the security community generates Proof-of-Concept (PoC) exploits for vulnerabilities in the npm ecosystem. Developed by researchers Deniz Simsek, Aryaz Eghbali, and Michael Pradel at the University of Stuttgart, PoCGen leverages Large Language Models (LLMs) in combination with static and dynamic analysis to automate what was once a painstaking manual task.
“This is the first fully autonomous approach to use large language models (LLMs) in tandem with static and dynamic analysis techniques for PoC exploit generation,” the authors state.
The JavaScript ecosystem—particularly npm—is notorious for the proliferation of security vulnerabilities. In 2024 alone, over 40,000 CVEs were disclosed, with a significant share affecting npm packages. Yet, most vulnerability disclosures lack working PoC exploits, creating a gap between discovery and remediation.
“Many vulnerability reports lack a PoC exploit, and even many CVE reports do not have any,” the study explains.
PoCGen fills this gap by automatically generating exploits from informal CVE descriptions. It does so through a four-phase process:
- Vulnerability Understanding – Identifies the vulnerability type and likely entry point.
- Exploit Generation – Crafts a candidate exploit using an LLM, aided by taint paths and code snippets.
- Validation – Executes and checks the exploit against defined conditions.
- Refinement – Iteratively improves the exploit until successful.
PoCGen generated a working exploit for CVE-2024-57063, a prototype pollution flaw in the php-date-formatter package. After a failed first attempt, the system refined its approach and produced this payload:
This ultimately polluted Object.prototype
, confirming a successful exploitation.
PoCGen was tested against two datasets:
- SecBench.js (560 vulnerabilities): 77% exploit success rate (432 exploits).
- CWEBench.js (794 newer vulnerabilities): 39% success rate (312 exploits).
It outperformed Explode.js by 45 percentage points and generated exploits for “158 vulnerabilities that Explode.js does not.”
“On average each PoC exploit generation attempt takes 11 minutes… at a cost of only $0.02 per exploit,” the researchers noted.
PoCGen empowers:
- Developers – to understand vulnerabilities and test patches rapidly.
- Security researchers – to automate exploit generation across new reports.
- Incident responders – to reproduce and verify vulnerabilities faster.
“By automating the generation of PoC exploits, POCGEN enables developers and security teams to more rapidly understand and address vulnerabilities,” the researchers concluded.
PoCGen is publicly available for research use, including its source code and datasets, enabling future tools to integrate and evolve exploit generation techniques.