GPU TPS calculator

This is a database of 818 graphics cards, filtered down to the ones with enough memory to hold a language model at all. Every card has its own page, and on that page every model we track has been assessed against it — whether it fits, at what compression, and how many tokens per second it is likely to produce.

The calculation runs when you open a card rather than being looked up from a stored table, which is what allows the context length and the minimum quality to be things you set rather than things we assumed on your behalf. Change either and every figure recomputes.

Running a language model on your own hardware became an ordinary thing to do in a short space of time. Open-weight models caught up far enough to be genuinely useful, and the tools to run them stopped requiring a research background. What did not arrive alongside that was a straight answer to the first question anyone asks: will it run on the card I already have, and will it be fast enough to be worth using.

Spec sheets do not answer it. They are written for graphics workloads and lead with the figures that decide frame rates, which are almost entirely the wrong ones. The two numbers that actually decide the outcome are memory capacity, which sets whether a model fits, and memory bandwidth, which sets how fast it generates once it does.

Tokens per second is the number that expresses the result in something you can feel. Reading speed is somewhere around ten tokens per second, so anything above that arrives faster than you can read it and anything well below feels like waiting. That is the figure this site calculates, for every combination of card and model it holds.

Memory across the catalogue runs from 4 GB to 288 GB. That range is the difference between a card limited to the smallest usable models and one that holds models no desktop machine can touch — capacity is a hard gate rather than a gradual trade-off, so a model either fits or it does not run.

Memory bandwidth runs from 10 GB/s at the low end to 8,190 GB/s at the top, held by the Radeon Instinct MI355X. Because generating each token means reading the whole model out of memory once, that spread translates almost directly into a spread in generation speed: a card with ten times the bandwidth produces tokens roughly ten times faster on the same model.

Cards come from AMD, ATI, Intel and NVIDIA. The manufacturer matters more than it should: inference software has had far more attention paid to the CUDA path than to any other, so an AMD or Intel card of equivalent bandwidth generally reaches a smaller share of its theoretical ceiling. Our estimates apply a penalty for that rather than pretending the hardware is the only variable.

Power draw spans 6 W to 2,400 W. The upper end is datacentre hardware that a desktop supply cannot feed, and it is the constraint people tend to discover last — after the card fits the budget and the case.

Release dates run from 2009 to 2025, so the catalogue covers hardware people still own as well as hardware they might buy. An older card is not excluded for being old — if it has the memory, it can still run a model, and the page will say how fast.

Vendors can list cards for sale against any entry in this catalogue, so a card page can show both what the hardware will run and where it can be bought. Listings are attached to the specific board variant rather than to a model name, which matters here more than it usually would — the same name often covers several memory capacities, and capacity is the thing that decides which models run.