[mle]/DaBench
Leaderboard

LFM3 1.7B

LiquidAI/LFM3-1.7B-GGUF

LocalQ8_02.8 GB at 32K104 tok/s on M5 Pro256K context
Run recordCompareOpen weights
Overall score
6

5 of 153 test cases fully passed

Scores by task

Scores give partial credit. Passed counts fully correct answers only. Select a task to read its test cases.

  1. 13Agents2 of 21 passedp90 latency 4m 44s
  2. 0Structured Extraction0 of 21 passedp90 latency 1m 45s
  3. 5Coding1 of 21 passedp90 latency 1m 46s
  4. 0Grounded Answers0 of 21 passedp90 latency 3m 21s
  5. 14Instruction Following0 of 21 passedp90 latency 7m 7s
  6. 0Business Logic0 of 21 passedp90 latency 1m 44s
  7. 16Security & permissions2 of 21 passedp90 latency 4m 18s
  8. 0Diagnosis & investigation0 of 6 passedp90 latency 2m 15s

Run it yourself

The exact build behind this score, and the command that serves it.

Deployment
Local
Engine
llama.cpp
Quantization
Q8_0
Server flags
-ngl all --temp 0.1 --top-k 50 --reasoning on --repeat-penalty 1.1
Context window
256K
Memory
2.8 GB at 32K
Machine
Apple M5 Pro, 48 GB
Throughput
104 tok/s
Cost of a suite run
Not measured
Cost per case
Not measured
Time per case
2m 11s
Measured
2026-10-29
Run this buildllama.cpp
llama-server -hf LiquidAI/LFM3-1.7B-GGUF:Q8_0 --port 9931 -c 262144 -ngl all --temp 0.1 --top-k 50 --reasoning on --repeat-penalty 1.1

Pulls the GGUF from Hugging Face and serves an OpenAI-compatible endpoint on port 9931.

Memory it needs

2.0 GB of weights, 300 MB of runtime, and 16 MB per 1K tokens of context.

By context length
ContextMemorySmallest MacSmallest GPU
8K2.4 GB8 GB8 GB
32K2.8 GB8 GB8 GB
128K4.3 GB8 GB8 GB
256K6.3 GB16 GB8 GB

llama.cpp's projection for this file. One slot, a 16-bit KV cache, no vision projector. A Mac fits when the build stays within the share of RAM macOS lets the GPU use, about two thirds up to 32 GB and three quarters from 36 GB. Macs and GPUs both keep 1 GB free for llama.cpp.

One email when a new model lands on the board or in the showcase. Unsubscribe anytime.