SWE-rebench is a benchmark that evaluates how well LLM-based coding agents resolve real software bugs and feature requests. It was released in 2025 by researchers at the cloud company Nebius. Tasks are built automatically from issues in public GitHub repositories and the pull requests that resolved them, and an agent's code changes are graded with hidden tests.
It follows the same format as the widely used SWE-bench, but instead of fixing a task set once it continuously adds recent issues to reduce contamination, where solutions appear in model training data. Tasks are drawn mainly from Python repositories, and a public leaderboard compares model results.