Vals Designer
Publish your benchmark on vals.ai
Upload your results file, write the page in Markdown and see exactly how it will look on vals.ai. A reviewer from Vals AI checks every version before it goes live.
GPQA Diamond
Updated 9/1/2026
Graduate-level Google-Proof Q&A benchmark evaluating models on questions that require deep reasoning.
95.45% ±1.0595.20% ±1.07 | $0.08 $0.11 | 112.33s 65.76s | |||
|---|---|---|---|---|---|
1 |  Gemini 3.1 Pro Preview (02/26) | $0.08 | 65.76s | ||
2 |  GPT-5.6 Sol | $0.11 | 58.30s | ||
3 |  Grok 4.6 | $0.05 | 3m29s | ||
4 | Gemini 3.8 Flash | $0.06 | 20.18s | ||
5 | Gemini 3.7 Flash | $0.02 | 8.77s | ||
6 |  Qwen 3.8 Max | $0.08 | 4m27s | ||
7 | Gemini 3.6 Flash | $0.03 | 18.06s | ||
8 |  Claude Opus 5 | $0.10 | 39.59s | ||
9 | Claude Fable 5.1 | $0.20 | 45.86s | ||
10 | Claude Fable 5 | $0.45 | 99.90s | ||
11 | GPT 5.5 | $0.08 | 112.33s | ||
12 |  Kimi K3 | $0.09 | 2m02s | ||
13 | Grok 4.5 | $0.04 | 3m38s | ||
14 | Gemini 3.5 Flash | $0.06 | 27.66s | ||
15 |  MiniMax-M3 | $0.03 | 4m39s | ||
16 | Claude Opus 4.8 | $0.34 | 2m14s |
Dataset and context
The Graduate-Level Google-Proof Q&A benchmark (paper) tests models on challenging, graduate-level questions across STEM fields. The questions were written to be "Google-proof": answering them takes real understanding and reasoning, not recall or search.
We use the diamond subset of 198 questions: the ones that expert validators answered correctly and no more than one in three non-experts did. They are hard, but their answers are unambiguous.
16 models · scroll for more
Methodology
As in the original paper, each model is evaluated two ways:
- Zero-shot chain of thought. The model is asked to reason through the problem step by step before giving its answer.
- Few-shot chain of thought. The model also sees five example questions, each with a worked answer.
We follow the paper's prompts and answer parsing as closely as possible. One difference: for zero-shot chain of thought we ask each question in a single query, where the paper used two.
Vals Designer
Publish your benchmark on vals.ai
Upload your results file, write the page in Markdown and see exactly how it will look on vals.ai. A reviewer from Vals AI checks every version before it goes live.
GPQA Diamond
Updated 9/1/2026
Graduate-level Google-Proof Q&A benchmark evaluating models on questions that require deep reasoning.
95.45% ±1.0595.20% ±1.07 | $0.08 $0.11 | 112.33s 65.76s | |||
|---|---|---|---|---|---|
1 | Gemini 3.1 Pro Preview (02/26) | $0.08 | 65.76s | ||
2 | GPT-5.6 Sol | $0.11 | 58.30s | ||
3 | Grok 4.6 | $0.05 | 3m29s | ||
4 | Gemini 3.8 Flash | $0.06 | 20.18s | ||
5 | Gemini 3.7 Flash | $0.02 | 8.77s | ||
6 | Qwen 3.8 Max | $0.08 | 4m27s | ||
7 | Gemini 3.6 Flash | $0.03 | 18.06s | ||
8 | Claude Opus 5 | $0.10 | 39.59s | ||
9 | Claude Fable 5.1 | $0.20 | 45.86s | ||
10 | Claude Fable 5 | $0.45 | 99.90s | ||
11 | GPT 5.5 | $0.08 | 112.33s | ||
12 | Kimi K3 | $0.09 | 2m02s | ||
13 | Grok 4.5 | $0.04 | 3m38s | ||
14 | Gemini 3.5 Flash | $0.06 | 27.66s | ||
15 | MiniMax-M3 | $0.03 | 4m39s | ||
16 | Claude Opus 4.8 | $0.34 | 2m14s |
Dataset and context
The Graduate-Level Google-Proof Q&A benchmark (paper) tests models on challenging, graduate-level questions across STEM fields. The questions were written to be "Google-proof": answering them takes real understanding and reasoning, not recall or search.
We use the diamond subset of 198 questions: the ones that expert validators answered correctly and no more than one in three non-experts did. They are hard, but their answers are unambiguous.
16 models · scroll for more
Methodology
As in the original paper, each model is evaluated two ways:
- Zero-shot chain of thought. The model is asked to reason through the problem step by step before giving its answer.
- Few-shot chain of thought. The model also sees five example questions, each with a worked answer.
We follow the paper's prompts and answer parsing as closely as possible. One difference: for zero-shot chain of thought we ask each question in a single query, where the paper used two.