sweedworks

What you actually pay for

The tokens you can see are not the tokens you are billed for. A 6-word question costs 24 tokens, your system prompt is re-sent on every single turn, and a long conversation bills for text nobody typed.

Fourth in a series, after what the model reads, where the vocabulary comes from and how the next word is chosen. Counts from the real tokenizer. 12 August 2026.

Your question is wrapped in scaffolding

Send a model a system prompt and a question and you might reasonably count the tokens in those two strings. That is not what goes over the wire. This is:

<|im_start|>system<|im_sep|>You·are·a·helpful·assistant.<|im_end|><|im_start|>user<|im_sep|>What·is·the·capital·of·France?<|im_end|><|im_start|>assistant<|im_sep|>

The actual serialised request. Grey chips are special tokens — single tokens that spell out a whole tag.

Those <|im_start|> and <|im_end|> marks are structure, not text: each is one token, and they exist so the model can tell where one speaker stops and another begins. Note the request ends with <|im_start|>assistant<|im_sep|> — an unfinished header that hands the floor over. That trailing fragment is why a model answers at all rather than continuing your sentence.

The content was 13 tokens. The request is 24.

The overhead is exactly 4 per message

MessagesYour contentActually billedOverhead
1512+7
21021+11
42039+19
84075+35
1680147+67
Same message repeated. Overhead is 4 tokens per message plus 3 for the request itself.

So the rule is billed = content + 4 × messages + 3. The build checks that formula against the tokenizer's own chat encoder on 200 randomly generated conversations, because a rule that is nearly right about billing is worse than no rule.

On its own this is a rounding error. It stops being one when it is multiplied by every turn of a conversation.

The bill nobody predicts

Language model APIs are stateless. The model does not remember your conversation — the client re-sends the entire history on every request. Turn twenty carries turns one through nineteen with it.

Take a 64-token system prompt, 30-token questions and 150-token answers, over 20 turns:

TurnSent this turnBilled so farRe-sent
110510565%
229339887%
348187992%
58572,40596%
101,7979,51098%
152,73721,31599%
203,67737,82099%
Input tokens only. The bar is what each turn sends.

By the final turn, 99% of what you send is a re-run of what you already sent. Across the conversation you are billed for 37,820 input tokens, of which 600 is text the user actually typed — a factor of 63.0×. That 64-token system prompt alone accounts for 1,280 tokens, because you buy it again every turn.

This is why system prompt length matters far more than it looks. Every token you add is not paid once — it is paid once per turn, for the life of every conversation your product ever has. Trimming fifty tokens from a system prompt used in a twenty-turn conversation saves a thousand tokens per conversation.

Work out your own

Paste a real system prompt. It is tokenized in your browser with the same tokenizer the model uses; nothing is sent anywhere.

input tokens billed
tokens actually typed
ratio
re-sent on the last turn

What this does and does not mean

Two honest qualifications, because a scary number is easy to overstate.

Caching changes the price, not the arithmetic. Most providers now discount tokens they have seen before at the start of a request — a stable system prompt may bill at a fraction of the normal rate after the first call. The tokens above are still processed and still counted; what they cost depends on your provider's caching rules. The way to benefit is to keep the unchanging part of your prompt at the front, which is only obvious once you know the request is a flat sequence being re-sent.

The exact wrapper is not universal. The 4-tokens-per-message figure is the ChatML layout used by the GPT-4 family. Other providers wrap messages differently and some publish no format at all. That there is a wrapper, and that history is re-sent every turn, is true across all of them.

None of this is hidden, exactly. It is just never shown, and the unit you are billed in is not the unit you think in.


Counts come from the same o200k_base tokenizer used throughout this site. The billing rule is verified against the tokenizer's own chat encoder on every build — see chatcost.py. Prices are deliberately absent: they change, and the token counts do not.