How to Estimate Local LLM Cost per Token

The Pied Piper Trap: When Your Server Costs More Than It Makes

In Silicon Valley, Richard Hendricks builds a brilliant compression algorithm that eventually brings down his own company's infrastructure — the deeper he pushes, the more the hardware bill balloons while the product fights to justify it. It is fiction, but every developer who has self-hosted an LLM has lived a smaller version: the GPU sits in the corner glowing, the electricity meter spins, and somewhere in the back of your mind is the question "am I saving money or just paying a different bill?"

The honest answer requires a number most people never compute: the cost per token of your local setup. Cloud APIs advertise prices per million tokens; local hardware comes with a price tag but no per-token figure. Until you translate one into the other, "local is cheaper" and "local is expensive" are both vibes, not facts. This guide builds the math from first principles so you can answer the question for your own machine.

The Local Cost per Token Formula

The cost of a local token has three components, all of which you can estimate before buying anything:

Each piece matters differently depending on your setup. Hardware amortization dominates for a new GPU. Electricity dominates for a machine that runs 24/7. Operational overhead — your time, maintenance, cooling, storage — is the piece everyone forgets and the one that quietly makes self-hosting expensive for a small team.

The crucial insight is that the denominator grows with utilization. A GPU that generates a million tokens a month spreads its fixed cost across a million tokens; the same GPU generating ten thousand spreads the same fixed cost across ten thousand. This is why "local is cheap" and "local is expensive" are both true — the answer depends entirely on how much you actually use the hardware.

Amortizing the Hardware

Hardware is a fixed cost, so the way to price it is amortization: spread the purchase price over the useful life of the machine. The standard assumption for consumer GPU hardware is 36 months — long enough that the cost-per-month is meaningful, short enough that you are not pretending a GPU lasts forever.

A real example makes this concrete. An RTX 5090 lists at $1,999 MSRP[nvidia-rtx5090], though street prices run higher. Amortized over 36 months at the MSRP, that is about $56 per month before electricity. If that card generates, say, 40 million tokens per month (a heavy but realistic single-user workload on a fast model[quantized-benchmarks]), the hardware alone contributes roughly $1.40 per million tokens.

The formula for hardware per-token cost is simple:

If your usage is a tenth of that — 4 million tokens a month — the same card costs $14 per million tokens, suddenly more than many cloud APIs. The fixed cost does not care how much you use it; that is the whole trap.

Two refinements make the amortization honest. First, subtract a resale value: a 36-month-old consumer GPU still sells for a meaningful fraction of its original price, so the true cost is the purchase price minus what you recover at the end, spread over the months you actually own it. Second, remember that scaling means buying hardware again: moving from a 7B to a 70B model may mean a second card or a workstation, which resets the amortization clock and roughly doubles the fixed cost overnight. Amortization is not a one-time decision; it is a commitment that renews every time your requirements grow.

The deeper point is that hardware cost is a step function while cloud cost is a curve. Your local setup has discrete cost jumps every time you add a GPU or a bigger machine, and between those jumps the per-token cost keeps falling as you generate more. The cloud has no steps — the price per token is flat whether you send one request or a million. That difference in shape is why local favors heavy, growing usage and cloud favors light, variable usage, and why the two never converge into one clean answer.

A useful discipline is to re-run the estimate on a fixed schedule — monthly, or every time your usage visibly changes. Hardware prices, electricity rates, cloud pricing, and your own workload all drift over time, and the answer to "is local worth it?" drifts with them. A setup that was clearly worth it in March can quietly become a money pit by September, or vice versa, with no single event marking the flip. Treating the estimate as a living number, not a one-time calculation, is the difference between making a decision and pretending a decision stays made.

The Electricity Bill Nobody Mentions

Electricity is the variable cost that scales with actual runtime, and it is the piece most cost estimates skip entirely. The formula is straightforward:

An RTX 5090 has a 575W thermal design power[nvidia-rtx5090]. Add the rest of the system — CPU, motherboard, fans — and a loaded inference box commonly draws 700-900W at the wall. At a 15 cents per kWh rate, running that box at full load for 12 hours a day lands near $40-55 a month. That is often more than the hardware amortization, and it is pure recurring cost that never goes away.

This is why the electricity component punishes always-on setups. A GPU that idles between requests still draws a baseline; a server that must stay responsive 24/7 pays the full tab even if you use it an hour a day. Cloud APIs, by contrast, charge only for the tokens you actually send — the idle time is free because it is someone else's electricity.

Putting It Together: Your Real Cost per Token

Combining the pieces for the same RTX 5090 at heavy usage gives a realistic picture. Hardware amortization contributes about $1.40 per million tokens, electricity another $1.00-1.40 depending on utilization, and operational overhead (your time, setup, maintenance) adds whatever you value your hours at. The honest all-in number for a well-utilized single-GPU setup lands in the range of $2.50-4 per million tokens — before you count your own labor.

Now compare that to cloud pricing in 2026. DeepSeek V4 Flash costs $0.14 per million input tokens and $0.28 per million output[deepseek-pricing]. Claude Sonnet 5 runs $2 per million input and $10 per million output[anthropic-pricing], and GPT-5.6 tiers span a wide range depending on the model[openai-pricing].

The pattern is the key takeaway: a well-utilized local box lands near the mid-tier cloud pricing, competitive with Sonnet 5 and far below the flagship tiers, but dramatically more expensive than the cheapest cloud models like DeepSeek Flash. If your workload can tolerate a smaller model, the cloud at $0.14/M is simply unbeatable — local hardware cannot touch that price per token no matter how hard you amortize it.

Cloud Pricing as the Benchmark You Cannot Ignore

Local cost only means something against a reference, and that reference is what the same tokens cost as an API call. The 2026 cloud market spans two orders of magnitude, which is exactly why a single "is local cheaper?" answer does not exist. The range, with verified August 2026 pricing, looks like this:

The way to use this table is to pick the cloud model closest to what you would run locally. If you would self-host a compact 7B model, the honest comparison is against DeepSeek Flash at $0.14/M input — and local loses, hard. If you would self-host a top-tier 70B, the comparison is against Claude Sonnet 5 at $2/M, where a well-utilized local box can genuinely compete[anthropic-pricing][openai-pricing][deepseek-pricing].

This is why "local is cheaper" debates are always underspecified. The real question is cheaper than which model, at what utilization? Until you pin down both, you are comparing a number you have not computed against a range you have not scoped.

When Local Actually Wins the Cost Game

The numbers above reveal that local is rarely the cheapest option on pure cost-per-token. Where it wins is in the situations where the price per token is not the whole equation. Three cases justify self-hosting despite the math:

Privacy and data control. If your prompts contain sensitive data you do not want to send to an API provider, the cost of local is the price of keeping the data in the building. That is a compliance and risk decision, not a per-token optimization.

Predictable heavy usage. If you genuinely generate tens of millions of tokens a month on a mid-tier model, the fixed hardware cost gets spread thin enough to undercut the equivalent cloud model — the break-even analysis in the Local LLM Break-Even Calculator finds your exact crossover point.

Latency and availability control. A local model has no network hop, no rate limits, and no provider outages. For applications where a few hundred milliseconds matter or where reliability is contractual, that has real value even if the token price is higher.

In every other case — light usage, bursty demand, budget-conscious projects, or workloads that only need a small model — the cloud wins on cost. The discipline is to compute your own numbers instead of assuming.

The Step-by-Step Cost Estimate

Here is the process you can run today, without buying anything. First, decide your realistic monthly token volume — count your actual usage with the Token Counter Calculator rather than guessing. Second, price the hardware you are considering (card, PSU, RAM, storage) and divide by your amortization period in months. Third, estimate electricity from the card's power draw and your expected runtime. Fourth, add a line for operational overhead — your hours have a value even if you do not bill them. Finally, divide the monthly total by your monthly token volume to get cost per token, and compare it against the cloud price for the same class of model using the LLM API Cost Calculator.

The single most common mistake is skipping step one and assuming a usage level. Run the numbers for both a light month and a heavy month — the spread will tell you whether local is a rounding error or a liability. Most people discover that local is either obviously worth it or obviously not, and the grey zone is narrower than the marketing suggests.

Practical Tips for Cost Estimation

• Amortize over 36 months, not "forever." GPUs become obsolete faster than they physically fail. A 36-month window is the honest middle ground.
• Count electricity at the wall, not the TDP. Add 20-40% to the card's rated power to cover the system around it.
• Include your time. Setup, updates, and debugging are real costs. If your hours are worth $50, a week of tinkering is $400 of operational overhead.
• Recompute at your real volume. The break-even point lives at the intersection of usage and cost. Guess the volume wrong and every downstream number is wrong.
• Remember the cloud has no idle cost. For bursty or unpredictable demand, paying per token beats paying per month for hardware you do not saturate.
• Do not forget resale. A 36-month amortization can subtract a partial resale value at the end, which many people ignore and which makes local a bit cheaper in hindsight.

Limitations and Edge Cases

The per-token estimate is only as good as its assumptions, and several assumptions are easy to get wrong. Electricity rates vary wildly by region and time-of-day; a card running at idle draws far less than its peak TDP, so always-on setups may cost less — or more — than the naive formula. Token volume is tokenizer-dependent: the same workload produces different token counts across models, which silently changes the per-token comparison. And comparing local against cloud output-only pricing misses the reality that real API bills mix cheap input and expensive output. Finally, the largest unquantified term is usually your own labor — it does not appear in the formula unless you add it, but it is often the difference between local being a bargain and local being a hobby.

Frequently Asked Questions

❓ Is running a local LLM cheaper than cloud APIs?

✅ Only under heavy, predictable usage. A well-utilized box lands near mid-tier cloud pricing, but the cheapest cloud models (like DeepSeek at $0.14/M input) beat local hardware on pure price per token. Local wins on privacy, control, and latency, not usually on raw cost.

❓ How much does it cost to run an RTX 5090 locally per month?

✅ Roughly $56 in amortization plus $40-55 in electricity at 12h/day, before your labor. That translates to roughly $2.50-4 per million tokens at heavy utilization — competitive with mid-tier cloud, far above the cheapest cloud models.

❓ What is the break-even point for local vs cloud?

✅ It depends on your model choice, hardware, and volume. Against Claude Sonnet 5 the crossover is around 135M tokens a month; against cheap DeepSeek pricing it may be unreachable on a single consumer GPU. Use the Local LLM Break-Even Calculator for your specific numbers.

❓ Why is local LLM cost per token never quoted by vendors?

✅ Because it depends entirely on your utilization, electricity rate, and hardware. There is no fixed per-token price for hardware, so nobody can quote one honestly. You have to compute it from your own numbers.

❓ Does local cost less if I only use it occasionally?

✅ Almost never. Occasional use leaves the fixed hardware and electricity costs spread over very few tokens, which pushes cost per token far above cloud rates. Light users should strongly prefer pay-per-token cloud APIs.

❓ How do I count my own labor in the cost?

✅ Estimate the hours spent on setup, updates, and maintenance, value them at whatever your time is worth, and divide by your monthly token volume. It is often the largest hidden term and the one that turns 'free' self-hosting into a real expense.