tau2-bench

A benchmark from Sierra that measures how well AI agents handle customer-service tasks involving users and tools.

1 articleLast mentioned

tau2-bench (τ²-bench) is a benchmark that evaluates how well an AI agent completes tasks by conversing with a user and calling tools. Released in 2025 by the US AI startup Sierra as the successor to its 2024 τ-bench, it consists of customer-service tasks in domains such as airline, retail, and telecom, in which the agent must handle user requests while following domain policy rules.

The user is simulated by a language model. Unlike Ï„-bench, the user can also operate tools, so the benchmark covers two-sided situations in which the agent must guide the user through a procedure. Reliability is measured by running the same task multiple times and counting how often all runs succeed. The benchmark is widely used to compare agents' tool use and policy adherence.

This entry is based on AIPOST articles and widely known facts. If something is wrong, please send us a correction request.

Articles covering this entry

  • Prompt Audits and Cache Reuse Cut Claude Costs Up to 72.9%, Anthropic SaysAI in Practice ·

    tau2-bench (retail) -72.9% +2.0 points

    Read article

Found a mistake or missing an entry?

Wiki entries are based on AIPOST articles and widely known facts. If an entry is wrong, send us a correction request and we will check and fix it. Names from new articles are added to the wiki after the article is published.