SWE-bench is a benchmark that measures the software engineering capabilities of AI models. It was released in 2023 by researchers at Princeton University and is built from real GitHub issues and their corresponding pull requests in widely used open-source Python repositories. A model is given the issue description and the repository code, generates a patch to fix the problem, and the task counts as resolved if the patched code passes predefined unit tests.
Because scoring is based on the fraction of issues resolved, the benchmark does not evaluate code style or maintainability. Variants include SWE-bench Verified, a human-validated subset of 500 tasks released by OpenAI in 2024, as well as SWE-bench Lite and SWE-bench Multimodal, and it is one of the standard references for comparing coding agents and language models.