[mle]/DaBench
Leaderboard

LFM2.5 24B A2B

LiquidAI/LFM2.5-24B-A2B-GGUF

LocalQ8_025.0 GB at 32K82 tok/s on M5 Pro128K contextBest of 2 builds
Run recordCompare
Overall score
7

6 of 153 test cases fully passed

Scores by task

Scores give partial credit. Passed counts fully correct answers only. Select a task to read its test cases.

  1. 12Agents1 of 21 passedp90 latency 10m 13s
  2. 10Structured Extraction2 of 21 passedp90 latency 1m 2s
  3. 1Coding0 of 21 passedp90 latency 2m 34s
  4. 5Grounded Answers1 of 21 passedp90 latency 1m 2s
  5. 16Instruction Following0 of 21 passedp90 latency 3m 21s
  6. 5Business Logic1 of 21 passedp90 latency 56.9s
  7. 10Security & permissions1 of 21 passedp90 latency 4m 41s
  8. 0Diagnosis & investigation0 of 6 passedp90 latency 8m 17s

Run it yourself

Best build: liquidai · Q8_0

The exact build behind this score, and the command that serves it.

Deployment
Local
Engine
llama.cpp
Quantization
Q8_0
Server flags
-ngl -1 --temp 0.6 --top-k 50 --reasoning on --repeat-penalty 1.05
Context window
128K
Memory
25.0 GB at 32K
Machine
Apple M5 Pro, 48 GB
Throughput
82 tok/s
Cost of a suite run
Not measured
Cost per case
Not measured
Time per case
1m 53s
Measured
2026-10-14
Run this buildllama.cpp
llama-server -hf LiquidAI/LFM2.5-24B-A2B-GGUF:Q8_0 --port 9931 -c 131072 -ngl -1 --temp 0.6 --top-k 50 --reasoning on --repeat-penalty 1.05

Pulls the GGUF from Hugging Face and serves an OpenAI-compatible endpoint on port 9931.

Memory it needs

24.0 GB of weights, 298 MB of runtime, and 23 MB per 1K tokens of context.

By context length
ContextMemorySmallest MacSmallest GPU
8K24.5 GB36 GB32 GB
32K25.0 GB36 GB32 GB
128K27.2 GB48 GB32 GB

llama.cpp's projection for this file. One slot, a 16-bit KV cache, no vision projector. A Mac fits when the build stays within the share of RAM macOS lets the GPU use, about two thirds up to 32 GB and three quarters from 36 GB. A GPU keeps 1 GB free.

Builds compared

2 builds
BuildOverallAgentsStructured ExtractionCodingGrounded AnswersInstruction FollowingBusiness LogicSecurity & permissionsDiagnosis & investigation
liquidai · Q8_0BaselineQ8_025.0 GB at 32K82 tok/s7121015165100
liquidai · Q4_K_MQ4_K_M14.7 GB at 32K117 tok/s6−18−40−10110+515−10−511+10

Changes are measured against liquidai · Q8_0, the highest-precision build. Open a build for its run record.

One email when a new model lands on the board or in the showcase. Unsubscribe anytime.

LeaderboardCompare LFM2.5 24B A2B with another model