Vals AI is a US company that evaluates large language models (LLMs) and the AI products built on them as an independent third party. Founded in 2023, it is headquartered in San Francisco, California. The company builds benchmarks for specific professional domains such as law, finance, tax, and coding, and publishes the results for each model on public leaderboards.
Its evaluations are run separately from the scores that model developers report themselves, with multiple models tested under the same conditions by an outside party. Part of its evaluation data is kept private so that test questions do not end up in model training data. The company is also known for the Vals Legal AI Report, which compared legal AI products against the work of lawyers.