SEC-bench

SEC-bench is a benchmark that measures how well LLM agents find and fix security vulnerabilities in real software.

2 articlesLast mentioned

SEC-bench is a benchmark for evaluating how well agents built on large language models perform software security tasks. Released in 2025, it is built from vulnerabilities reported in real open-source projects and assesses two tasks: writing proof-of-concept code that reproduces a vulnerability and patching it.

Agents analyze and modify code inside an execution environment and their results are verified automatically, which distinguishes SEC-bench from earlier security evaluations composed of short-answer questions.

This entry is based on AIPOST articles and widely known facts. If something is wrong, please send us a correction request.

Articles covering this entry

  • Mistral Large 4, OpenAI Model Spec, Touchscreen Mac: AI News Under the RadarAI News ·

    SEC-bench Pro Finding vulnerabilities 88.2%

    Read article
  • GPT-6.1 Terra, Dot Texting and Gemini 4 Carbon: What Works Now and What WaitsAI News ·

    …orld-2.1), knowledge work, multimodal understanding, long context, science and math, and cybersecurity tests including CWE-bench, Exploit-Gym and SEC-bench Pro.

    Read article

Found a mistake or missing an entry?

Wiki entries are based on AIPOST articles and widely known facts. If an entry is wrong, send us a correction request and we will check and fix it. Names from new articles are added to the wiki after the article is published.