5 min read
o200k_base vs cl100k_base explained
The two tiktoken encodings behind every modern OpenAI model, what doubling the vocabulary buys you, and why the wrong one overstates non-English text.
tokenizationtiktokenencodings
Writing
Tokens, tokenizers and what they cost.
The calculator gives you a number. These posts explain where the number comes from — how byte-pair encoding splits your text, what a context window looks like in pages, and which parts of a rate card actually move your bill.
5 min read
What cached input pricing actually saves
Cached input is advertised as 90% off, and the discount is real — but it applies to one column, on a byte-identical prefix, with an expiry clock running.
pricingprompt cachingcost optimization
4 min read
How many tokens is a page of text?
A single-spaced page of English prose is about 650–700 tokens. Where that number comes from, context windows in pages, and when the rule breaks down.
tokenscontext windowestimation
4 min read
How LLM tokenization actually works
Language models never see your characters or your words — they see token ids from a byte-pair encoder. What that encoder does, and how it surprises people.
tokenizationfundamentalsBPE
Want the number rather than the explanation? Use thetoken counter, count a document with thePDF token counter, or compare rates across every provider on theLLM pricing page.
New posts are available over RSS.