GPQA

GPQA is a benchmark that measures the scientific reasoning of AI models with graduate-level multiple-choice questions in biology, physics and chemistry.

1 articleLast mentioned

GPQA (Graduate-Level Google-Proof Q&A Benchmark) is a question-answering benchmark released in 2023 by researchers at New York University and other institutions. It consists of 448 multiple-choice questions written by domain experts in biology, physics and chemistry. The questions are designed so that experts in the field can answer them while non-experts struggle even with unrestricted access to web search.

GPQA Diamond is a subset of 198 questions validated as the highest in quality, and it is widely used to compare the scientific reasoning of language models.

This entry is based on AIPOST articles and widely known facts. If something is wrong, please send us a correction request.

Articles covering this entry

  • AI Agent Tuning: When to Rewrite Prompts and When to Retrain WeightsAI Performance ·

    Method AIME'24 GPQA-Diamond GPU hours

    Read article

Found a mistake or missing an entry?

Wiki entries are based on AIPOST articles and widely known facts. If an entry is wrong, send us a correction request and we will check and fix it. Names from new articles are added to the wiki after the article is published.