publications

2026

1. Agentic Vulnerability Reasoning on COTS Binaries

   Hwiwon Lee^*, Jongseong Kim^*, and Lingming Zhang

   * Equal contribution

   arXiv preprint arXiv:2605.05000, 2026

   40 Windows vulnerabilities (31 assigned CVEs) and $203,000 in Microsoft bounty awards

   Abs arXiv

   SLYP is an end-to-end agentic pipeline for vulnerability discovery and validation on commercial off-the-shelf binaries without source access. It combines binary exploration with dynamic debugging to produce debugger-verified proofs of concept. SLYP has uncovered 40 critical vulnerabilities in Windows systems, all disclosed to Microsoft, including 31 assigned CVEs.

2. SEC-bench Pro: Can Language Models Solve Long-Horizon Software Security Tasks?

   Hwiwon Lee, Jiawei Liu, Dongjun Kim, and 4 more authors

   arXiv preprint arXiv:2605.26548, 2026

   Adopted by OpenAI, DeepSeek, and NIST for frontier-model evaluation + Google VRP $20,000 bounty

   Abs arXiv

   SEC-bench Pro evaluates whether frontier models can hunt real vulnerabilities in large, complex software systems and reproduce working proof-of-concept inputs. The benchmark contains 344 validated vulnerabilities across the V8 and SpiderMonkey browser engines and the Linux kernel, spanning memory-safety, sandbox, JIT, race-condition, and kernel-subsystem bugs.

3. Evaluating Coding Agents on Kernel Exploit Generation

   Junyoung Jang^*, Gwanhyun Lee^*, Hwiwon Lee^*, and 4 more authors

   * Equal contribution

   In The 2026 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2026

   Oral Presentation

   Abs arXiv

   KEX-bench evaluates coding agents’ kernel exploit-generation capabilities through fine-grained verification of exploit primitives. It includes 45 tasks covering 40 Linux and Windows vulnerabilities, with outcome checks that distinguish concrete security effects from crashes. Even with a reference proof of concept, the strongest tested configuration solves 68.9% of tasks.

2025

1. SEC-bench: Automated Benchmarking of LLM Agents on Real-World Software Security Tasks

   Hwiwon Lee, Ziqi Zhang, Hanxiao Lu, and 1 more author

   In Proceedings of the Thirty-Ninth Annual Conference on Neural Information Processing Systems (NeurIPS’25), 2025

   Abs HTML PDF

   SEC-bench is a fully automated benchmarking framework for evaluating language-model agents on authentic security-engineering tasks, including proof-of-concept generation and vulnerability patching.

2024

1. BENZENE: A Practical Root Cause Analysis System with an Under-Constrained State Mutation

   Younggi Park, Hwiwon Lee, Jinho Jung, and 2 more authors

   In 45th IEEE Symposium on Security and Privacy (Oakland’24), 2024

   IEEE Symposium on Security and Privacy 2024 Distinguished Paper Award

   Abs HTML PDF

   BENZENE is a practical end-to-end root-cause analysis system that uses under-constrained state mutation to generate crashing and non-crashing behaviors for effective and efficient diagnosis.