Harness², a research framework from Google Cloud AI Research and Northwestern University, tries to make AI agents better at their jobs without touching the underlying model. Instead of retraining weights, it lets an agent edit the instructions, rules, and helper files that surround the model while it works. In a trial on an arbitration review task, Gemini 3.5 Flash went from passing 20 of 55 automated checks to 22 after one round of self-editing, at a cost of about $11. That is a small gain, but it lines up with the average the researchers report, and some of their own case studies show far larger jumps.
Why the harness, not the model
Most work on smarter agents focuses on the model itself: retrain it, or fine-tune its weights on new data. Harness² starts from a different definition. An agent is the model plus its harness, and the harness is everything around the model that shapes its behavior: instructions, files, tools, constraints, and configuration.
Under this approach, the model’s weights stay locked for the entire process. The only thing the agent may change is its own harness. Because nothing is retrained, there is no training bill and no risk of catastrophic forgetting, the tendency of a model to lose earlier skills when it learns new ones.
The idea resembles how a new hire improves. On the first day they make mistakes, jot down notes, and build a checklist. On the second day they do much better. Harness² gives an agent the same feedback loop. Crucially, the agent grades its work against the task’s criteria without ever seeing the answer key.
The eight files an agent can rewrite
The framework wraps the model in eight modular files, and these are what the agent is allowed to modify:
- System prompt: the base instructions the agent starts from
- Guardrails: rules the agent must always follow
- Memory: notes carried over from earlier work
- Tool guidance: instructions on how to call tools correctly
- Subagents: helper agents that take on part of a job
- Skills: playbooks with know-how for a specific domain
- Scripts: code the agent can run directly
- Runtime plugins: add-ons that step in while a task is running
How one round of self-improvement works
The researchers walk through the cycle using a finance job in which an agent has to reconcile invoices, service incidents, and subscription credits into a single workbook, coping with blank cells and rows counted twice along the way. Each round has five steps:
- Run: The agent completes the task once with its original, unedited harness.
- Propose: It drafts two kinds of fixes. One is a general habit that should help on any task, and the other carries expert knowledge for this specific domain.
- Probe: Each fix is tried on its own against the same task before anything is kept.
- Compose: Only the fixes that proved their worth go into a final harness, and the agent runs the real deliverable again.
- Reflect: The agent condenses what it learned into a lesson stored in a permanent library file, so later runs inherit it.
The probe step matters most. According to the researchers, editing harness files blindly, without testing each change in a sandbox first, often lowers accuracy and leaves the agent worse off. Finding things to fix appears to be only half the job; checking that each fix actually helps is the other half.
▲ The five-step harness improvement loop
A legal review test with Gemini 3.5 Flash
To see how this plays out on a lightweight model, a test gave Gemini 3.5 Flash a legal task from Google’s official benchmark suite. The case involves arbitration under the International Chamber of Commerce (ICC). The agent has to review Draft Procedural Order No. 3, compare it with earlier procedural orders and strategy memos, and produce an annotated redline, meaning a marked-up copy that shows every proposed change, along with a letter setting out the main objections.
On its first attempt, the agent passed 20 of the 55 automated checks, a score of 36.4. It then examined what it had missed and changed two harness files. It wrote a new domain playbook, SKILL.md, which tells it to compare the draft against earlier orders line by line and to check hearing durations and evidence rules in a structured way. It also added a conditional instruction to systemprompt.md that loads this playbook whenever the job is an arbitration review.
| Measure | First attempt | After the harness edit |
|---|---|---|
| Checks passed | 20 of 55 | 22 of 55 |
| Score | 36.4 | 40.0 |
| Run time | 446.4 seconds (about 7.5 minutes) | 736.9 seconds (about 12 minutes) |
Assembling the final harness took 45.0 seconds, and the whole cycle of adapting and rerunning cost roughly $11 in API fees. Behind the 3.6-point gain, three criteria flipped from fail to pass: cross-referencing the first procedural order, restoring a hearing of 4.5 to 5 days, and deleting a standard cost clause. One criterion moved the other way. A check related to Zurich as the hearing venue passed on the first attempt and failed on the second.
▲ Arbitration order review test
How the result compares with the paper
The Gemini 3.5 Flash result looks more meaningful next to the numbers in the research paper.
| Setting | Before | After one round |
|---|---|---|
| Same arbitration task, best case with a frontier model | 44.4 | 94.5 |
| Average across the paper’s legal tasks | 79.9 | 81.9 |
| Same arbitration task, Gemini 3.5 Flash | 36.4 | 40.0 |
The best case, with a frontier model, shows how much a single round can deliver, but the average gain across the legal tasks was 2.0 points. The 3.6-point gain with Gemini 3.5 Flash falls close to that average trend. A fair reading may be that a typical round adds a few points, with occasional large leaps on tasks where the harness was clearly missing something.
Five tasks where the edits paid off
The paper also describes cases in which changes to the harness produced big improvements. In every one, the model stayed the same and only the agent’s working rules changed.
| Task | What the agent changed | Score |
|---|---|---|
| Reconciling a film producer’s budget memo | A general guardrail to reopen and verify every figure it outputs | 43.1 to 77.8 |
| Processing a sales workbook | Skill instructions for computing derived commission rates as weighted averages | 23.6 to 100.0 |
| Building an offline single-page web app | A guardrail that bans fetching code from external CDNs and keeps scripts, styles, and data inline | 64.0 to 100.0 |
| Synthesizing research | A runtime plugin that forces an early draft before limits run out | 72.0 to 100.0 |
| A coding benchmark | Strict tool guidance that rules out pseudo-code tool calls | 60.0 to 100.0 |
A CDN, or content delivery network, is an outside service that delivers files such as scripts and fonts to web pages. The research task kept timing out as it ran into its budget and its context limit, the amount of text a model can handle at once, so the agent built a plugin it called deliverable_watchdog. As the token limit approaches, the plugin stops further research queries so that a draft always gets written. Tokens are the units of text a model processes. In the coding case, the model had been emitting fake, pseudo-code versions of tool calls, and the new guidance taught it to use only valid native tool calls.
Caveats before you try it
Harness² is published on GitHub as the harness2 repository, where it has collected about 6,000 stars. It is experimental academic code from Google Cloud AI Research and Northwestern University, not an officially supported Google enterprise product.
Setup assumes a developer’s toolkit:
- Linux
- Python 3.12 and the uv package manager
- Node.js 20 or later
- Docker, which provides the sandbox for test runs
- The Google Cloud CLI, signed in with access to Vertex AI
That list means business users without technical help cannot easily deploy it yet. Self-editing also carries risk: as the Zurich venue check showed, a change that fixes some problems can break something that used to work. And each cycle adds time and money. The two runs in the legal test took nearly 20 minutes combined and cost about $11 for a single task.
What teams running agents can take from this
The core lesson of Harness² is that the model can stay fixed while the agent keeps improving through the files around it. The approach looks best suited to teams whose agents do predictable, repetitive work every week, because lessons from one run can carry into the next.
Even without installing the framework, teams that run their own agents can apply its principles now:
- Before swapping models, invest in the harness: system prompts, tool definitions, and guardrails.
- Test every prompt or skill change in a sandbox, one at a time, and merge only the ones that help.
- Sort improvements into two groups: general habits, such as rechecking output files, and domain knowledge, such as the norms of arbitration.
- Give long-running research agents a watchdog that forces a rough draft before the token or time budget runs out.
- Make web deliverables self-contained, with styles, scripts, and data inline, so they work offline without a CDN.
- Write tool guidance that bans pseudo-code tool calls and requires the native call format.