What you actually pay for
The tokens you can see are not the tokens you are billed for. A 6-word question costs 24 tokens, your system prompt is re-sent on every single turn, and a long conversation bills for text nobody typed.
Fourth in a series, after what the model reads, where the vocabulary comes from and how the next word is chosen. Counts from the real tokenizer. 12 August 2026.
Your question is wrapped in scaffolding
Send a model a system prompt and a question and you might reasonably count the tokens in those two strings. That is not what goes over the wire. This is:
<|im_start|>system<|im_sep|>You·are·a·helpful·assistant.<|im_end|><|im_start|>user<|im_sep|>What·is·the·capital·of·France?<|im_end|><|im_start|>assistant<|im_sep|>
Those <|im_start|> and <|im_end|> marks
are structure, not text: each is one token, and they exist so the model can tell
where one speaker stops and another begins. Note the request ends with
<|im_start|>assistant<|im_sep|> — an unfinished header
that hands the floor over. That trailing fragment is why a model answers at all
rather than continuing your sentence.
The content was 13 tokens. The request is 24.
The overhead is exactly 4 per message
| Messages | Your content | Actually billed | Overhead |
|---|---|---|---|
| 1 | 5 | 12 | +7 |
| 2 | 10 | 21 | +11 |
| 4 | 20 | 39 | +19 |
| 8 | 40 | 75 | +35 |
| 16 | 80 | 147 | +67 |
So the rule is
billed = content + 4 × messages
+ 3. The build checks that formula against the
tokenizer's own chat encoder on 200 randomly generated conversations, because a
rule that is nearly right about billing is worse than no rule.
On its own this is a rounding error. It stops being one when it is multiplied by every turn of a conversation.
The bill nobody predicts
Language model APIs are stateless. The model does not remember your conversation — the client re-sends the entire history on every request. Turn twenty carries turns one through nineteen with it.
Take a 64-token system prompt, 30-token questions and 150-token answers, over 20 turns:
| Turn | Sent this turn | Billed so far | Re-sent | |
|---|---|---|---|---|
| 1 | 105 | 105 | 65% | |
| 2 | 293 | 398 | 87% | |
| 3 | 481 | 879 | 92% | |
| 5 | 857 | 2,405 | 96% | |
| 10 | 1,797 | 9,510 | 98% | |
| 15 | 2,737 | 21,315 | 99% | |
| 20 | 3,677 | 37,820 | 99% |
By the final turn, 99% of what you send is a re-run of what you already sent. Across the conversation you are billed for 37,820 input tokens, of which 600 is text the user actually typed — a factor of 63.0×. That 64-token system prompt alone accounts for 1,280 tokens, because you buy it again every turn.
This is why system prompt length matters far more than it looks. Every token you add is not paid once — it is paid once per turn, for the life of every conversation your product ever has. Trimming fifty tokens from a system prompt used in a twenty-turn conversation saves a thousand tokens per conversation.
Work out your own
Paste a real system prompt. It is tokenized in your browser with the same tokenizer the model uses; nothing is sent anywhere.
What this does and does not mean
Two honest qualifications, because a scary number is easy to overstate.
Caching changes the price, not the arithmetic. Most providers now discount tokens they have seen before at the start of a request — a stable system prompt may bill at a fraction of the normal rate after the first call. The tokens above are still processed and still counted; what they cost depends on your provider's caching rules. The way to benefit is to keep the unchanging part of your prompt at the front, which is only obvious once you know the request is a flat sequence being re-sent.
The exact wrapper is not universal. The 4-tokens-per-message figure is the ChatML layout used by the GPT-4 family. Other providers wrap messages differently and some publish no format at all. That there is a wrapper, and that history is re-sent every turn, is true across all of them.
None of this is hidden, exactly. It is just never shown, and the unit you are billed in is not the unit you think in.
Counts come from the same o200k_base tokenizer used throughout this site. The billing rule is verified against the tokenizer's own chat encoder on every build — see chatcost.py. Prices are deliberately absent: they change, and the token counts do not.