NVIDIA used GTC Berlin 2026 to argue that the biggest gains in AI agents now come from the software wrapped around a model rather than from the model itself. The company's new Agent Toolkit bundles harnesses, a sandbox runtime, a knowledge layer and telemetry tools, and NVIDIA backs the pitch with a striking number: an NVIDIA developer blog post dated August 21, 2026 reports that Claude Opus 5, with its weights left untouched, moved from 30% to 100% on the ARC-AGI-3 public set once it ran inside the NVIDIA AVO harness. Adel El Hallak, NVIDIA's vice president of software product management, laid out why the company is building this layer, how its Agent Knowledge Foundation differs from the retrieval-augmented generation (RAG) that most enterprises already run, and where the Kumo prediction models fit.
Five parts of an agent, and only one is the model
The framing comes from NVIDIA CEO Jensen Huang's keynote, which splits an agent into five parts:
| Part | Role | NVIDIA's piece |
|---|---|---|
| Model | The brain; one large language model or a system of several | Nemotron 3 Ultra, or a third-party model such as Claude Opus 5 |
| Harness | The body or exoskeleton that lets a model act instead of only emitting text | AVO harness, partner harnesses |
| Sandbox | Containment and control over what the agent may do | OpenShell |
| Knowledge | Company data turned into something the agent can use | Agent Knowledge Foundation |
| Tools | Execution capabilities, including CUDA-X libraries called from natural language | CUDA-X libraries, MCP integrations |
The point of the split is that a model on its own only generates tokens. Everything that turns those tokens into a completed task lives in the other four boxes, and that is where NVIDIA says it is placing its engineering effort.
A toolbox, not a competitor to LangChain
A natural question is whether the Agent Toolkit competes with frameworks such as LangChain, CrewAI or Agno. NVIDIA's answer is no. The toolkit is described as an umbrella that delivers models, libraries, integrations and hardware acceleration into frameworks that partners already build. NVIDIA works directly with Harrison Chase and the LangChain team, and has folded LangChain harnesses such as Deep Agents into its own reference implementations. It also collaborates with Nous Research on the Hermes agent and with the team behind OpenClaw.
Instead of a finished application, the toolkit ships as modular parts plus open reference agents called blueprints. A blueprint packages a chosen model or set of models, an execution harness, task-specific tools and skills, Model Context Protocol (MCP) integrations, and runtime containment inside OpenShell. Blueprints range from an agent that drives 3D design work in Blender to enterprise workflows built with LangChain, and teams are meant to clone, modify and embed them in their own software rather than start from scratch. Examples include NVIDIA NeMoClaw for Agent Optimization and NeMoClaw for LangChain Deep Agents Code.
Agent Knowledge Foundation: what it adds to RAG
RAG appeared almost immediately after ChatGPT launched in November 2022, when hallucinations made grounding a necessity. The first wave relied on vector embeddings over unstructured documents, and NVIDIA still invests heavily there: better embedding and reranking models, the argument goes, mean an agent needs fewer search turns to land on the right context. The company's NeMo Retriever parse, embed and rerank components feed Milvus vector stores for exactly this purpose.
The Agent Knowledge Foundation goes further by treating structured data as a first-class input. It adds knowledge graphs, ontologies and schemas that represent entities and the verbs connecting them, so that an agent navigates business logic rather than only text similarity.

▲ Knowledge graph harmonizing separate data schemas
The motivating problem is schema sprawl across thousands of relational tables. In a cellular billing example, one table might call a subscription level a "plan" while another calls it a "tier." A person reads both as the same thing; an autonomous agent often does not. NVIDIA Knowledge Foundations provides libraries that build a knowledge graph from such datasets and harmonize the naming into a single ontology, so the agent queries one consistent vocabulary and retrieves the correct relational facts.
Ontologies versus GraphRAG
The two concepts are complementary rather than competing. An ontology supplies the defined terms, domain vocabulary and relationship labels that get mapped onto the knowledge graph. GraphRAG is the retrieval technique that walks that graph to pull back connected entity nodes instead of flat text chunks. Both aim at the same outcome: grounding a model's reasoning in verified relational structure rather than in an unverified nearest-neighbor lookup.
Kumo: foundation models for tables
The second structured-data piece is NVIDIA Kumo, a family of structured models covering relational, tabular, forecasting and anomaly-detection tasks. The contrast is with the traditional tabular machine learning stack, where a data science team builds a separate pipeline for each prediction task using XGBoost or LightGBM, complete with hand-built feature engineering, ETL jobs, and benchmarking across classification and regression models.
Kumo models bring LLM-style generalization to numbers in tables: a single architecture handles churn forecasting, fraud analysis and anomaly detection without a bespoke pipeline per task. NVIDIA says enterprise partners are adopting the models broadly because they remove that per-task overhead. The models also address a known weakness of language models, which struggle with raw numerical work and with finding a value buried in a workbook full of tabs. Inside the Agent Knowledge Foundation, Kumo sits alongside NeMo Retriever's ontology and tabular components as the structured-data path, while parse and embed handle unstructured content.
Harness engineering: the same weights, a different result
Harness engineering means improving the program around the model rather than the model. Common changes are modest on paper: tuning the system prompt, adjusting middleware logic between calls, and refining tool definitions and schemas. NVIDIA's claim is that these carry far less engineering cost than post-training yet can produce outsized gains.

▲ Supervision, memory and context control in a harness
The ARC-AGI-3 study is the showcase. Researchers kept Claude Opus 5 fixed and worked only on the NVIDIA AVO harness, adding advanced context management, supervisory agents that keep sub-agents on track, and persistent memory handling. The public-set score went from 30% to 100%. A related discipline, context engineering, governs when to share context between tasks and when to spin up fresh context to avoid bias or contamination from a previous job. NVIDIA presents harness and context engineering together as the core of production agent architecture.
This does not mean post-training is dismissed. A keynote slide titled "Turn Proprietary Data Into Sovereign Intelligence" showed an open model, Nemotron 3 Ultra, gaining 30 accuracy points over GPT-3.5 after post-training on proprietary data, alongside enterprise case studies. The practical ordering NVIDIA suggests is to exhaust harness changes first, because they are cheaper, and then post-train where domain data justifies it. Developers, in NVIDIA's view, routinely underestimate how much the harness alone can move the needle.
Why demos break, and what NVIDIA does about it
Agents are probabilistic: the same input can yield a different output on the next run. At production scale they also hit throttling, rate limits and latency spikes, and those do not merely slow an agent down; they degrade its accuracy and reasoning. Any developer whose agent has never broken in a demo, the argument goes, has probably not deployed a real one. The response is a set of tools and policies:
- Telemetry from day one. NeMo Relay is an open-source library that records every step an agent takes, including traces, spans and decision trajectories, compatible with OpenTelemetry. It supports evaluation frameworks such as Harbor and the human-readable Agent Trace Interchange Format (A-TIF). In one sample trace of a Hermes session, chat completions cost 4,400 to 4,545 tokens and took 9.7 seconds, and a file read took 694 milliseconds. Traces serve governance and double as training data for the next iteration.
- A mandatory sandbox. Inside NVIDIA, running agents in OpenShell is described as non-negotiable. OpenShell wraps a gateway, sandbox, harness and policy proxy around model inference, tool execution, knowledge retrieval and telemetry reporting.
- Evaluation libraries. Two new NeMo libraries, Eval Author and Compass, consume telemetry to diagnose failure modes and synthesize evaluation datasets. One reference blueprint runs an agent on Hermes, which integrates NeMo Relay natively, and feeds the traces into both tools.
The email test illustrates the loop. An agent asked to draft an email sent it instead, because the send tool was enabled without a draft-only constraint. Telling users to write explicit negative instructions in prompts is unreliable, so the fix is to generate tests from the traces and bake the guardrail into the harness, so the agent cannot trigger an external action it was not authorized to take.
Skills and routers to control cost
Two further techniques address efficiency. Skills are saved, written procedures for one recurring job, so the model does not re-derive the same multi-step logic each time. Some harnesses detect repeated patterns and turn them into skills automatically; others let developers declare them. Skills can be grouped into plugins for longer workflows.
Routers send each task to the right-sized model. NeMo Switchyard handles routing, cascading and context orchestration across heterogeneous models, backed by NeMo Relay for traces and NVIDIA Dynamo for inference. Strategies range from an LLM-as-a-judge approach to confidence-based cascades that start with a small, fast model and escalate to heavyweight models such as Astra, Mythos or Fable only when a confidence score falls below a threshold. The warning is against treating every problem like a nail because a frontier-model hammer is available.
Where this is heading
El Hallak's own development stack mixes OpenClaw, Hermes, Claude Code, Codex and LangChain Deep Agents Code, with the coding assistants often authoring evaluation suites while the target agent runs in an open framework. Local execution is described as having a renaissance, with developers running Hermes and other agents at home on DGX Spark workstations. Blender is the showcase example: an agent executes 3D modeling commands through tool calls, so someone who cannot navigate Blender's dense interface can produce assets by describing them.
The takeaways for teams building agents are concrete. Before paying for fine-tuning, revisit the system prompt, middleware and tool schemas, and consider adding supervisory agents and memory management; the ARC-AGI-3 result suggests the headroom can be large. Run agents inside a sandbox from the start and instrument them with telemetry at the proof-of-concept stage, not after launch. Treat structured data as its own problem, using ontologies and tabular models rather than forcing everything through a vector store. Finally, push repetitive work into skills and route routine tasks to smaller models, reserving frontier models for the cases that need them.


