Abstract:
External vulnerability knowledge is increasingly important for large language models (LLMs) to understand vulnerability targets, craft exploit payloads, and interpret verification evidence. Recent approaches incorporate external vulnerability knowledge via prompt, task planning and multi-agent collaboration, to improve automated penetration testing. However, automated vulnerability reproduction remains challenging because public exploit materials are often heterogeneous and loosely structured, spanning scripts, HTTP requests, command-line snippets, and walkthrough in natural language. Prior work has leveraged retrieval-augmented generation, PoC semantic repair, and exploit code generation to better utilize external vulnerability knowledge, but still constrained by environment dependencies, exploit semantic extraction and model hallucinations, weakening the trustworthiness of automated reproduction.
To address these issues, we propose ValidPoC, an LLM-driven framework for automated vulnerability reproduction that can convert heterogeneous exploit walkthrough or scripts into structured and reliable PoC-Nuclei templates. Specifically, ValidPoC defines a PoC semantic model that provides a unified representation of exploit logic, including request construction, payload generation, execution dependencies, and verification criteria. We develop a Semgrep-assisted semantic parsing mechanism to extract PoC Semantic IR from public exploit materials, including exploit scripts and natural-language descriptions, and construct structured Nuclei YAML templates for automated vulnerability reproduction. To improve reproduction reliability, we establish a closed-loop execution–evaluation–repair framework and a three-level validation model (L1–L3) for cross-validation across multiple execution evidence.We evaluate ValidPoC on 67 real-world vulnerability reproduction tasks. ValidPoC achieves a Pass@3 reproduction rate of 76.12% and consistently surpasses existing automated penetration testing frameworks, including PentestAgent and PentestGPT. In addition, it maintains competitive efficiency with respect to both execution time and token usage. These results demonstrate the effectiveness of ValidPoC in understanding heterogeneous exploit materials, generating PoC templates, and vulnerability reproduction.