Memory-bound throughput, calculated live
Can your GPU run that AI Model? The GPU TPS calculator
Pick a graphics card or a language model and we calculate — live, not from a lookup table — whether it fits, at what compression, and roughly how many tokens per second to expect.
Live estimate
EntryFormat
Q8_0
Context
13k
Confidence
high
Worked examples
Check out these powerful combos
A few pairings we'd point a friend to, at different budgets.
01
Compare GPUs
VRAM, memory bandwidth, tensor cores and FP16 throughput side by side.
02
Estimate tokens per second
Decode speed is bound by memory bandwidth, so we model it rather than quoting peak TFLOPS.
03
Check what fits
Which models run on a card, at which quantisation, and with how much context.
The problem
Why "will my GPU run this?" is a hard question
A spec sheet tells you a card's memory and its clock speed. It does not tell you whether a specific language model fits in that memory once compressed, how much of the card is left over for the conversation itself, or how many tokens per second actually come out the other end — and those three questions do not have the same answer for any two GPU-and-model pairs.
The honest version of this calculation pulls in the model's parameter count and architecture, the compression format it is stored at, the card's memory bandwidth and how mature its inference software is, and the length of the conversation you intend to have with it. Get any one of those wrong and the number that comes out looks just as confident and is simply incorrect.
We do not pretend this produces a single precise figure, because it cannot. What we do instead is run the full calculation for every card and every model we hold data for, publish the result as a range rather than a false-precision point value, and show our working — every formula behind every number on this site is written out in the open, not hidden behind the answer.
What this site actually gives you
The most complete dataset we could put together: thousands of graphics cards and thousands of language models, cross-checked against each other rather than looked at one at a time. Start from a card you own or are considering, and see every model that fits on it. Start from a model you want to run, and see every card that can hold it.
That works in both directions because the underlying calculation is symmetric — the same memory-and-bandwidth arithmetic, run from whichever side you already know. Neither direction is the "real" one; they are the same answer asked two different ways.
Read more about how and why on our about page.
Newly released
Check out these powerful new GPUs
Browse all GPUsNVIDIA
Oct 2025
Both directions
How the calculator works
The same calculation, usable from whichever side of the question you're starting from.
Starting from a GPU you own
- 01 Find your graphics card . Search the GPU catalogue for the card you own or are considering, by name or by memory size.
- 02 Open its page . Every card has its own page listing every language model we can assess against it, with an estimated tokens-per-second figure for each.
- 03 Set your context length and quality floor . Adjust the conversation length you expect to work at and the lowest compression you will accept. Both change the answer live.
- 04 Read the range, not a single number . Each estimate is published with a confidence range rather than one false-precision figure, because the same card and model vary between inference engines in ways no spec sheet predicts.
Starting from a model you want to run
- 01 Find the model . Search the AI model catalogue by name or by size — even when the name itself does not state the parameter count.
- 02 Open its page . A model with open weights lists every graphics card we can assess it against, smallest that fits and fastest first.
- 03 Set your context length and quality floor . The same two controls as the GPU-first path — a longer conversation needs more memory, which can push a card from "fits" to "does not".
- 04 Compare cards by what you'll actually notice . Memory decides whether it runs at all; bandwidth decides how fast it feels once it does. The table sorts by tokens per second by default because that is usually the one that matters.
Answers
Common questions
How do I calculate if my GPU can run a local LLM?
Find your card in the GPU catalogue and open its page — it lists every model we can assess against it, whether each one fits in memory, and an estimated tokens-per-second figure. You can also start from a model and see which cards can run it, which is the same calculation from the opposite direction.
Can my GPU run local LLM models at all?
If it has at least a few gigabytes of memory of its own, almost certainly something. Integrated graphics that share system memory are the main exception, since they have no dedicated pool to hold a model in. Beyond that it is a question of which model, at which compression — not yes or no.
What determines tokens per second on a GPU?
Generating a token means reading the model's weights out of memory once, so speed is bounded by memory bandwidth divided by how much has to be read per token — which depends on the model's size, its architecture, and the compression format it is stored at. Clock speed and core counts, the numbers a spec sheet leads with, barely factor in.
How accurate are the estimates on this site?
Expect the real figure to land within roughly 20–40% of the estimate. The same GPU and model vary that much between inference engines, engine versions and serving configuration — no calculation from spec sheets alone can predict that, which is why every figure here is published as a range.
Is a bigger, newer GPU always faster for AI?
For a model that already fits comfortably on both cards, memory bandwidth decides the winner, and a newer card usually has more of it. But the more common bottleneck is capacity, not speed — the more useful question for a specific card is usually "what's the largest model that fits", which is what each card's page answers directly.
Do I need to understand quantisation to use this site?
No — every card and model page picks the best-quality compression that actually fits, automatically. Quantisation is the technique that shrinks a model's memory footprint at a small cost to output quality, and you can set a quality floor yourself if you want to, but nothing requires it.
Where do the GPU and AI model datasets come from?
Graphics card specifications and language model records covering thousands of open-weight models and their parameter counts, licensing and release dates. Both are refreshed as new hardware and models are released — see the GPU and AI model catalogues for full coverage.
Selling hardware
Interested in becoming a vendor?
List your GPUs for sale right next to the answer to what they can actually run.
Coming soon
Get notified about new GPUs and models
A newsletter is coming soon.
Explore further
Browse the full GPU catalogue or browse the full AI model catalogue. Buying hardware? check the vendor directory. More questions are answered on our FAQ page, or contact us directly.