Will it run? A GPU shop for local models

Build a one-screen GPU shop for people who want to run AI models at home, laid out as a board: models are cards, machines are columns. Drag a model onto a machine and see right away whether it fits, how fast it runs and what the machine costs, then watch its answer arrive at that speed.

Data

Card	Memory (GiB)	Bandwidth (GB/s)	Price
RTX 3060 12GB	12	360	$280
RTX 3090 24GB (used)	24	936	$700
RTX 4090 24GB	24	1008	$1,999
RTX 5090 32GB	32	1792	$2,599
RTX A6000 48GB (used)	48	768	$3,300

Model	Total parameters	Active parameters	Context memory per token	Longest context
Gemma 4 12B	11.95 billion	11.95 billion	16 KiB	262,144
gpt-oss-20b	20.9 billion	3.6 billion	24 KiB	131,072
Gemma 4 26B A4B	25.2 billion	3.8 billion	20 KiB	262,144
Qwen3.8 27B	27.8 billion	27.8 billion	64 KiB	262,144
Gemma 4 31B	30.7 billion	30.7 billion	80 KiB	262,144
Qwen3.6 35B-A3B	35 billion	3 billion	20 KiB	262,144
gpt-oss-120b	116.8 billion	5.1 billion	36 KiB	131,072
GLM-5.3-Flash	320 billion	18 billion	11 KiB	1,048,576
GLM-5.3	753 billion	40 billion	87.75 KiB	1,048,576
Kimi K2.6	1 trillion	32 billion	68.625 KiB	262,144

Quantizations: Q8_0 is 8.5 bits per weight, Q4_0 is 4.5.

Rules

• Weights take total parameters × bits per weight ÷ 8 bytes. Context memory takes tokens × the model's per-token figure, halved with the 8-bit cache. 1 GiB is 2^30 bytes, 1 KiB is 1,024 bytes and 1 GB/s is 10^9 bytes per second.
• A machine is one card or two of the same card, and it can hold several models at once. Two cards split every model's weights and context memory in half. Each card also needs 1 GiB of overhead for each model on the machine. The machine fits when every card's total is at most its memory. When it doesn't fit, none of its models run.
• A model's speed ceiling is the card bandwidth ÷ (active parameters × bits per weight ÷ 8), in tokens per second with one decimal. Two cards hold more but are no faster: each token passes through both cards in turn, so the speed uses one card's bandwidth. Only one model generates at a time, so models on the same machine don't slow each other down. Mark every model with fewer active than total parameters as "MoE".
• Spilling to system RAM, when a machine has it switched on: every model's context memory and overhead stay on the cards, the weights fill what is left on the cards starting with the top model in the column, and the rest of the weights go to RAM. The machine fits when the cards hold the context and overhead and the RAM holds the rest. RAM reads at 80 GB/s, and the share of a model's active parameters read from RAM equals the share of its weights in RAM, so its speed ceiling becomes 1 ÷ (active bytes on the cards ÷ card bandwidth + active bytes in RAM ÷ RAM bandwidth).

Build

• The board: a Models column with a card for each model (name, MoE or dense, total parameters, context memory per token, longest context, and its weights at Q4_0 and Q8_0), then up to three machine columns, with an "Add machine" slot while there are fewer than three.
• Machine header: the graphics card, a 1 or 2 count, the price (card price × count), a "Spill to" switch with a RAM size of 32, 64, 128, 256 or 512 GiB, a "Cheapest fit" button and a remove button. Under them, one memory bar per card, split into each model's weights, context and overhead in three colors, plus a RAM bar while spilling, with the part past the memory marked. Under the bars, the fit line in two decimals: "Fits, X GiB to spare", "Fits, X GiB in RAM", "Over by X GiB" or "Over by X GiB of RAM".
• Dragging a model from the Models column puts a copy on a machine, starting at Q4_0, 32,768 tokens and the 16-bit cache. Dragging a card moves it to another machine or to another place in its column, Alt-drag copies it, and dropping it on the Models column removes it.
• Each card on a machine has its own Q4_0/Q8_0 switch, 16-bit/8-bit cache switch, and context slider from 1,024 to the model's longest context in steps of 1,024, labeled with the most context that fits (the largest step its machine can hold with everything else unchanged). It shows its speed ceiling and how much of it sits in RAM, or "Won't run" and the over line.
• While a card is dragged over a machine, show the verdict it would get there before the drop ("Fits · X tok/s", "X tok/s · Y GiB in RAM" or the over line) and preview that machine's bars and fit line with the card in place.
• A panel under the board follows one picked card at a time. Clicking a card that runs, or dropping a card on a machine, picks it and streams a 300-word answer into the panel at its speed ceiling, one word per token, with the word count and elapsed seconds as it goes and "300 words in X s · Y tok/s average" at the end. Write the answer yourself: 300 words on why memory bandwidth sets a local model's speed. The panel also shows the picked card's speed and its own verdict, so a card fully on the cards never reads as spilled.
• The stream follows its card: change the card's quantization, cache or context, move it, or change its machine, and the stream carries on from the current word at the new speed. While the card doesn't fit, the stream pauses and its clock stops, and it carries on when the card fits again. Play restarts it.
• When the picked card won't run, the panel lists every single change that would make its machine fit on its own, with its numbers: the most context that fits, the 8-bit cache, Q4_0, a second card, or spilling to the chosen RAM size. Leave out a change that still doesn't fit. Clicking a change applies it.
• A Math switch (M) replaces the answer with every number for the picked card worked out with its values plugged in: weights, context, overhead, each card's total, the fit line, the bytes read per token, the speed ceiling and the most context.
• "Cheapest fit" tries every card at counts 1 and 2 with the machine's models and settings and switches to the cheapest one where they all fit. A tie goes to the faster build, judged by its slowest model. If nothing fits, it says how much memory the models need in total: weights + context + 1 GiB per model.
• Keyboard: Enter picks a card, 1 to 3 sends a card to that machine, Delete removes it, and Alt+↑/↓ moves it up or down its column.
• The page opens on two machines, both with spilling off and 64 GiB of RAM: one RTX 4090 holding Qwen3.8 27B, picked but not playing yet, and one empty RTX 3060.
• Keep everything in memory. localStorage and cookies throw in this sandbox.

What separates great from passable

• The board, the bars and the fit lines work before anything else. If something has to give, it gives in this order: the keyboard, Cheapest fit, Math, the fit changes, spilling, the drag preview, the stream, and never the board, the bars or the fit lines.
• The bars, fit line, speed and most context update with every slider step, and the fit line flips at the exact step where the machine stops fitting.
• The stream keeps exact pace at any speed: words follow the clock, several per frame when needed, so a card at 800 tokens per second finishes in under half a second, not in five, and every finish time matches 300 ÷ the speed shown.
• A stream changes pace mid-sentence without restarting: turn spilling on and off, or move a spilled model to the top of its column, and the words visibly slow down or speed up.
• It reads well on video at 1920×1080 and in a 1280×800 window: big numbers, clear colors, nothing tiny, and the whole board on one screen.