MiMo-V2.6-Pro vs Underdog Saluki 27B
Task by task
A score turns orange only when that model wins clearly more of the task's test cases. Under each score, the average time per test case on the machine it ran on. Both models spend about the same on thinking per test case.
TaskAMiMo-V2.6-ProBUnderdog Saluki 27B
Overall
482m 13s
335m 15s
Agents
474m 48s
409m 2s
Structured Extraction
601m 26s
433m 17s
Coding
242m 58s
166m 54s
Grounded Answers
621m 9s
433m 22s
Instruction Following
591m 23s
466m 5s
Business Logic
571m 52s
333m 55s
Security & permissions
711m 41s
433m 47s
Diagnosis & investigation
02m 54s
06m 39s
What you'd run
The build behind each score. Orange marks the faster, smaller, cheaper or longer-context side.
SpecAMiMo-V2.6-ProBUnderdog Saluki 27B
Speed
55 tok/s
23 tok/s
Machine
Hosted
Apple M5 Pro, 48 GB
Memory at 32K
—
10.2 GB
Context
1.1M (better)
256K
Cost per run
$1.80
Not measured
Cost per case
$0.012
Not measured
Time per case
2m 13s (better)
5m 15s
Quantization
BF16
IQ2_XS
Harness
openrouter
llama.cpp
The race
Each model's running score as the test cases go by. The dashed line is a perfect run.
Every test case
Each square is one test case, colored by the model that scored higher.
Agents
Structured Extraction
Coding
Grounded Answers
Instruction Following
Business Logic
Security & permissions
Diagnosis & investigation