run
This run
- suite
- starter@v1
- ran at
- 2026-07-30 14:10 UTC
record id
config
Deployment configuration
- harness
- llama.cpp
- quantization
- Q8_0
- context
- 193.8K
- temperature
- 0
- multi-token prediction
- —
- engine
- v1.0.0
- provider
- llama.cpp
category call budgetreasoningtotal
Tool Calling4.1K/ high8.2K/ call
Structured Output2.0K/ medium4.1K/ call
RAG / Retrieval QA2.0K/ medium4.1K/ call
Context Recall2.0K/ medium4.1K/ call
Coding4.1K/ high8.2K/ call
Reasoning & Math4.1K/ high8.2K/ call
Instruction Following2.0K/ medium4.1K/ call
Classification2.0K/ medium4.1K/ call
Summarization2.0K/ medium4.1K/ call
run this buildllama.cpp
llama-server -hf bartowski/Nanbeige_Nanbeige4.2-3B-GGUF:Q8_0 -c 193792Pulls the GGUF from Hugging Face and serves an OpenAI-compatible endpoint on :8080.
measured
How it was measured
- latency
- 8919435 ms
- cost
- $1.5122est.
- memory samples
- 8965
tok/s and latency measure the scored generation window; per-test figures above are the readings for each case. Memory (VRAM, unified RAM, or host RAM) is the peak footprint polled throughout the run — measured via process rss across 8965 samples, or blank when no probe could attribute it — never a hand-typed number.
recipe
prompts → hashes
Reproduce this run
The private prompts never leave the suite. What’s public is the command, the pinned suite, and the hashes that stand in for the tests — enough to re-run the exact config against your own copy of the suite.
command
npm run engine -- run --provider llama.cpp --model bartowski/Nanbeige_Nanbeige4.2-3B-GGUF:Q8_0 --suite suites/starter.suite.json --name 'Nanbeige4.2 3B' --category 'tool-calling,reasoning'- suite hash
- config hash
- suite pinned to
- starter@v1