Head to head

Any two models from the board, on the same tasks and graders.

Side A

Profile

Hosteddeepseek65 tok/sopenrouter

Side B

Profile

Hostedinclusionai264 tok/sopenrouter

Overall

Wins 7 tasks, 49 test cases

Overall

Wins 0 tasks, 9 test cases

DeepSeek V4 Flash 0731 leads by 14 points and wins 7 of 9 tasks.

Learn to build evals like this in the AI Engineering Academy

Task by task

DeepSeek V4 Flash 0731 wins 7 of 9

Average score on each task, best build on each side. The higher score is highlighted. Hover a score for the test cases behind it.

TaskADeepSeek V4 Flash 0731BLing-3.0 flashLead

Overall

+14

Tool Calling

Right tool, right args, right sequence?

+18

Structured Output

Valid JSON matching the schema, correct fields?

Even

RAG / Retrieval QA

A correct answer grounded in the given context, with nothing made up?

+29

Context Recall

Finds the one fact buried in a long context?

Coding

Working code that passes hidden unit tests?

+41

Reasoning & Math

Multi-step math and logic with the exact right answer?

Instruction Following

Obeys format, length, and content constraints to the letter?

+11

Classification

Correct label from a fixed set?

Even

Summarization

Compact summary that keeps the facts and invents none?

+18

The race

No lead changes

Score piled up over the 238 test cases both models ran, as a share of the suite's maximum. The dashed line is a perfect run.

DeepSeek V4 Flash 0731Ling-3.0 flash

Every test case

238 test cases

One square per test case, in suite order. A square takes the color of the model that scored higher on it.

Tool Calling112

Structured Output11

RAG / Retrieval QA91

Context Recall21

Coding110

Reasoning & Math21

Instruction Following41

Classification22

Summarization70

49 won by DeepSeek V4 Flash 07319 won by Ling-3.0 flash180 tied

Where it breaks

238 shared test cases

On the impossible tier, DeepSeek V4 Flash 0731 passes 38% of test cases and Ling-3.0 flash passes 20%.

TierADeepSeek V4 Flash 0731BLing-3.0 flashLead

Baseline

36 test cases

78%

75%

+3 pts

Hard

69 test cases

61%

46%

+15 pts

Impossible

133 test cases

38%

20%

+18 pts

Tiers come from the suite. Baseline is fair, hard is adversarial, and impossible is built so that nothing solves it. Each number is the share of test cases the model fully passed.

Thinking spend

DeepSeek V4 Flash 0731 spends 1.2× as many thinking tokens per test case.

DeepSeek V4 Flash 0731

3.5Ktokens

Per test case, 76% of it thinking

Ling-3.0 flash

4.1Ktokens

Per test case, 55% of it thinking

TaskADeepSeek V4 Flash 0731BLing-3.0 flash

Tool Calling

4.1K52% thinking

6.7K40% thinking

Structured Output

2.8K78% thinking

3K46% thinking

RAG / Retrieval QA

2.9K85% thinking

3.3K54% thinking

Context Recall

75787% thinking

87459% thinking

Coding

6.1K86% thinking

8K79% thinking

Reasoning & Math

5.9K73% thinking

6.2K52% thinking

Instruction Following

3.4K71% thinking

3.2K54% thinking

Classification

2.7K81% thinking

2.8K50% thinking

Summarization

1.8K81% thinking

2K49% thinking

Average output tokens per test case, and how much of it was thinking.

What you'd run

The build behind each score. The better value in each row is highlighted.

SpecADeepSeek V4 Flash 0731BLing-3.0 flash

Speed

65 tok/s

264 tok/s

Memory

—

Context

262.1K

Cost per run

$0.2742

Quantization

—

Harness

openrouter

Faster, smaller, cheaper and more context count as better. Quantization and harness only describe the builds.