277.4tok/s
Mean per test case, 278 tok/s over the run
8.5Ktokens
Answer plus reasoning
6.3Ktokens
75% of the output
2.7M
1.4M in, 1.3M out
1h 29m
35.1s per caseRequest time, summed
Not measured
No price recorded
Every test case
One square per test case. Select a square to read the answer.
26Agents3 of 21 passed
52Structured Extraction11 of 21 passed
4Coding0 of 21 passed
29Grounded Answers6 of 21 passed
36Instruction Following4 of 21 passed
14Business Logic3 of 21 passed
40Security & permissions7 of 21 passed
0Diagnosis & investigation0 of 6 passed
Speed and output per test case
One bar per test case. Hover to inspect a case, select it to read the answer.
Peak 389 tok/s
ReasoningAnswer
Test by test
Every answer with its reasoning and the checks it had to pass.
Reproduce this run
The prompts stay private so models can't train on them. The hashes pin the exact suite and config this run used.
OpenRouterHosted
curl https://openrouter.ai/api/v1/chat/completions \
-H "Authorization: Bearer $OPENROUTER_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"apodex/apodex-1.1-mini:free","provider":{"order":["novita/bf16"],"allow_fallbacks":false},"service_tier":"default","messages":[{"role":"user","content":"Hello"}]}'The request the benchmark sent, pinned to the provider that served it.
Configuration
Where the suite ran, and the token budget each task got.
- Hosted
- OpenRouter
- Novita
- BF16
- 256K
- 0
- v1.0.0
- 2026-10-05 20:59 UTC
{
"provider": {
"order": [
"novita/bf16"
],
"allow_fallbacks": false
},
"service_tier": "default"
}| Task | Reasoning | Max output per call |
|---|---|---|
| Agents | 4.1Khigh | 12.3K |
| Structured Extraction | 4.1Khigh | 12.3K |
| Coding | 4.1Khigh | 12.3K |
| Grounded Answers | 4.1Khigh | 12.3K |
| Instruction Following | 4.1Khigh | 12.3K |
| Business Logic | 4.1Khigh | 12.3K |
| Security & permissions | 4.1Khigh | 12.3K |
| Diagnosis & investigation | 4.1Khigh | 12.3K |