Head to head

Any two models from the board, on the same tasks and graders.

Side A

Profile

Hosteddeepseek108 tok/sopenrouter

Side B

Profile

Hostedliquid229 tok/sopenrouter

Overall

Wins 74 test cases

Overall

Wins 0 test cases

DeepSeek: DeepSeek V4.1 Flash is clearly better. It wins 74 of the 74 test cases where they differ.

Learn to build evals like this in the AI Engineering Academy

Task by task

Average score on each task, best build on each side. A score is highlighted only when that model wins clearly more of the task's test cases. Hover a score for the test cases behind it.

TaskADeepSeek: DeepSeek V4.1 FlashBLiquidAI: LFM2.5-2.6B (free)Lead

Overall

+70

Agents

Finishes multi-step work over your tools under a written policy?

+63

Structured Extraction

The right state rebuilt from revised records, in the exact JSON contract?

+87

Coding

Working code and queries that pass hidden tests?

+27

Grounded Answers

Answers only from what's in the prompt, cited, and no guess when it isn't there?

+87

Instruction Following

Every stated rule applied, including changes made later in the conversation?

+73

Business Logic

The rules you'd have to implement, computed exactly right?

+67

Safety

Ignores instructions planted in data and tool results, and still does what the policy allows?

+84

The race

No lead changes

Score piled up over the 105 test cases both models ran, as a share of the suite's maximum. The dashed line is a perfect run.

DeepSeek: DeepSeek V4.1 FlashLiquidAI: LFM2.5-2.6B (free)

Every test case

105 test cases

One square per test case, in suite order. A square takes the color of the model that scored higher on it.

Agents100

Structured Extraction130

Coding40

Grounded Answers130

Instruction Following110

Business Logic100

Safety130

74 won by DeepSeek: DeepSeek V4.1 Flash0 won by LiquidAI: LFM2.5-2.6B (free)31 tied

Thinking spend

LiquidAI: LFM2.5-2.6B (free) spends 1.5× as many thinking tokens per test case.

DeepSeek: DeepSeek V4.1 Flash

6.5Ktokens

Per test case, 93% of it thinking

LiquidAI: LFM2.5-2.6B (free)

9.1Ktokens

Per test case, 96% of it thinking

TaskADeepSeek: DeepSeek V4.1 FlashBLiquidAI: LFM2.5-2.6B (free)

Agents

3.7K75% thinking

16K98% thinking

Structured Extraction

5.2K93% thinking

6.6K97% thinking

Coding

10.7K98% thinking

8.1K93% thinking

Grounded Answers

4.9K92% thinking

6.2K98% thinking

Instruction Following

9.3K93% thinking

13.6K95% thinking

Business Logic

8.8K98% thinking

7.3K98% thinking

Safety

2.8K81% thinking

5.8K96% thinking

Average output tokens per test case, and how much of it was thinking.

What you'd run

The build behind each score. The better value in each row is highlighted.

SpecADeepSeek: DeepSeek V4.1 FlashBLiquidAI: LFM2.5-2.6B (free)

Speed

108 tok/s

229 tok/s

Memory

—

Context

65.5K

Cost per run

$0.2717

Not measured

Cost per passed case

$0.003881

Not measured

Quantization

FP8

Harness

openrouter

Faster, smaller, cheaper and more context count as better. Quantization and harness only describe the builds.