DeepSeek V4.1 Flash vs Mistral Large 4
Task by task
A score turns orange only when that model wins clearly more of the task's test cases. Under each score, the average time per test case on the machine it ran on. DeepSeek V4.1 Flash spends 1.2× as many thinking tokens per test case.
TaskADeepSeek V4.1 FlashBMistral Large 4
Overall
643m 35s
281m 56s
Agents
464m 42s
1126.1s
Structured Extraction
862m 8s
153.78s
Coding
514m 55s
332m 48s
Grounded Answers
762m 17s
382m 45s
Instruction Following
894m 24s
693m 47s
Business Logic
814m 1s
242m 40s
Security & permissions
693m 3s
331m 9s
Diagnosis & investigation
172m 8s
01m 19s
What you'd run
The build behind each score. Orange marks the faster, smaller, cheaper or longer-context side.
SpecADeepSeek V4.1 FlashBMistral Large 4
Speed
30 tok/s
56 tok/s (better)
Machine
Hosted
Hosted
Memory at 32K
—
—
Context
1M (better)
512K
Cost per run
$0.39 (better)
$2.67
Cost per case
$0.0026 (better)
$0.017
Time per case
3m 35s
1m 56s (better)
Quantization
FP8
—
Harness
openrouter
openrouter
The race
Each model's running score as the test cases go by. The dashed line is a perfect run.
Every test case
Each square is one test case, colored by the model that scored higher.
Agents
Structured Extraction
Coding
Grounded Answers
Instruction Following
Business Logic
Security & permissions
Diagnosis & investigation