MiMo-V2.6-Pro vs Qwen3.8 27B
Task by task
A score turns orange only when that model wins clearly more of the task's test cases. Under each score, the average time per test case on the machine it ran on. Both models spend about the same on thinking per test case.
TaskAMiMo-V2.6-ProBQwen3.8 27B
Overall
482m 13s
4833.3s
Agents
474m 48s
5736.6s
Structured Extraction
601m 26s
7320.6s
Coding
242m 58s
3753.0s
Grounded Answers
621m 9s
6131.6s
Instruction Following
591m 23s
6733.6s
Business Logic
571m 52s
3632.9s
Security & permissions
711m 41s
5122.6s
Diagnosis & investigation
02m 54s
040.0s
What you'd run
The build behind each score. Orange marks the faster, smaller, cheaper or longer-context side.
SpecAMiMo-V2.6-ProBQwen3.8 27B
Speed
55 tok/s
196 tok/s (better)
Machine
Hosted
Hosted
Memory at 32K
—
—
Context
1.1M (better)
256K
Cost per run
$1.80
$1.28 (better)
Cost per case
$0.012
$0.0083 (better)
Time per case
2m 13s
33.3s (better)
Quantization
BF16
FP8
Harness
openrouter
openrouter
The race
Each model's running score as the test cases go by. The dashed line is a perfect run.
Every test case
Each square is one test case, colored by the model that scored higher.
Agents
Structured Extraction
Coding
Grounded Answers
Instruction Following
Business Logic
Security & permissions
Diagnosis & investigation