[mle]/DaBench

Ling 3.1 Flash vs Underdog Saluki 27B

Side A
Model page
Side B
Model page
Overall
45
vs
Overall
33

Ling 3.1 Flash is clearly better. It wins 32 of the 47 test cases where they differ.

Learn to build evals like this in the AI Engineering Academy

Task by task

A score turns orange only when that model wins clearly more of the task's test cases. Under each score, the average time per test case on the machine it ran on. Both models spend about the same on thinking per test case.

Thinking tokensAnswer tokens
TaskAB
Overall
451m 16s
7.9K, 73% thinking6.9K, 84% thinking
335m 15s
Agents
4155.9s
4.5K, 66% thinking11.8K, 97% thinking
409m 2s
Structured Extraction
711m 11s
8.1K, 69% thinking4.4K, 92% thinking
433m 17s
Coding
241m 42s
11.3K, 89% thinking9.2K, 49% thinking
166m 54s
Grounded Answers
5258.9s
6.7K, 74% thinking4.5K, 92% thinking
433m 22s
Instruction Following
761m 55s
11.4K, 77% thinking8K, 94% thinking
466m 5s
Business Logic
381m 22s
9.7K, 56% thinking5.2K, 81% thinking
333m 55s
Security & permissions
5456.1s
5.2K, 74% thinking4.9K, 91% thinking
433m 47s
Diagnosis & investigation
042.1s
3.2K, 72% thinking8.7K, 94% thinking
06m 39s

What you'd run

The build behind each score. Orange marks the faster, smaller, cheaper or longer-context side.

SpecAB
Speed
112 tok/s
23 tok/s
Machine
Hosted
Apple M5 Pro, 48 GB
Memory at 32K
—
10.2 GB
Context
256K
256K
Cost per run
Not measured
Not measured
Cost per case
Not measured
Not measured
Time per case
1m 16s (better)
5m 15s
Quantization
—
IQ2_XS
Harness
openrouter
llama.cpp

The race

1 lead change

Each model's running score as the test cases go by. The dashed line is a perfect run.

Ling 3.1 FlashUnderdog Saluki 27B
025507510050100150Perfect run4533

Every test case

153 test cases

Each square is one test case, colored by the model that scored higher.

32 won by Ling 3.1 Flash15 won by Underdog Saluki 27B106 tied