Tokens, Not Words: The Unit That Quietly Decides Your AI Bill

A token is the actual unit a language model reads and bills you for, not a word but a text chunk, roughly four characters or three-quarters of an English word, produced by an algorithm called byte-pair encoding. Whitespace, punctuation, and rare words all quietly add tokens that your word count never shows.

Quick answer: An LLM never reads "words" — it reads tokens, sub-word chunks that average roughly ¾ of an English word each. Every commercial API prices by token, not by word, so filler phrasing, extra spacing, and unusual vocabulary all quietly add cost your word count never shows you.

What Tokens Actually Are (Not Words, Not Characters)

A token is whatever a model's tokenizer maps to one number in its fixed vocabulary — a whole common word, a fragment of a rarer word, a punctuation mark, or a stretch of whitespace. GPT-4-class models draw from a vocabulary of roughly 100,000 possible chunks, not an unlimited set of English words.

Every model family ships its own tokenizer: a small program that converts raw text into a sequence of integers before anything reaches the neural network. The model has never "seen" the word cat — it has seen token ID 4719, which happens to decode back to cat. Common words usually earn their own single token. Rarer words get sliced into pieces, and that slicing is where your intuition and your invoice part ways.

Here's what that slicing looks like on real text, run through OpenAI's publicly available cl100k_base tokenizer (the same encoding behind GPT-4-class models, testable yourself at platform.openai.com/tokenizer):

Notice the last row: a single dollar figure with a comma and a decimal costs six tokens on its own. That's not a rounding error — it's the tokenizer treating punctuation and digit groups as separate chunks, a detail no word-count estimate would ever surface.

Where the Idea Came From

Byte-pair encoding isn't an AI invention:

• 1994 — programmer Philip Gage first published it as a plain data-compression trick, in the C Users Journal.
• 2016 — researchers Rico Sennrich, Barry Haddow, and Alexandra Birch adapted it for neural machine translation in "Neural Machine Translation of Rare Words with Subword Units," and it's been the default way to tokenize text for large language models ever since.

This slicing also explains a few odd habits large language models have that confuse first-time users. AI researcher Andrej Karpathy, in his widely watched walkthrough of how GPT tokenizers work, points out that models are often surprisingly bad at spelling out a word letter-by-letter or counting characters in it — because the model never saw individual letters, only one or two opaque chunks standing in for the whole word.

That's a useful mental model for a PM: the AI isn't reading your text the way a proofreader would. It's reading a compressed code for your text, and cost, quality, and quirky failures all trace back to how that code gets assembled.

If you want the fuller map of how a model turns those tokens into a response — attention, context windows, inference cost — our complete guide to LLM fundamentals covers the rest of the pipeline this article assumes.

Why "1 Token ≈ 0.75 Words" Is a Rule of Thumb, Not a Law

OpenAI's own tokenizer documentation states that, on average, one token equals roughly four characters or three-quarters of an English word — so 100 tokens covers about 75 words of typical prose. That average holds reasonably well for clean, common English. It breaks down fast for code, jargon, non-English text, or heavily formatted documents.

Use the ratio for rough sizing, not precision billing:

Two things push the ratio further than most PMs expect:

• Formatting overhead. Markdown tables, bullet indentation, and copy-pasted HTML all add tokens that carry zero instructional value.
• Language. Independent tokenizer analysis by AI researcher Yennie Jun found that many non-English languages need substantially more tokens than English for equivalent text — in some cases two to three times as many or more — because most vocabularies are trained on English-heavy corpora.

If your AI feature serves a multilingual user base, that gap alone can double your per-request cost before a single prompt-engineering decision gets made.

Five Sneaky Things That Inflate Your Token Count

Word count hides five common cost inflators: extra whitespace, decorative punctuation, rare or foreign vocabulary, technical jargon, and formatted numbers. None of these show up when you're estimating cost by eyeballing word count, and all five show up on the invoice.

We tested the whitespace row directly: "The quick brown fox jumps over the lazy dog." tokenizes to 10 tokens. Add double spacing between every word — same nine words, same meaning — and it jumps to 18 tokens. Nobody counting words would ever catch that.

A quick self-audit before you hit send:

1. Run the draft through a public tokenizer tool, not a word counter.
2. Delete throat-clearing openers ("I was wondering if," "just kind of").
3. Collapse repeated whitespace and indentation left over from copy-paste.
4. Swap jargon for a common synonym wherever the meaning survives.
5. Keep exact numbers only where precision genuinely matters to the task.

Deciding which of these to cut is itself a tradeoff between cost and clarity, not a pure cost-minimization exercise — our tradeoff analysis complete guide has a framework for weighing "cheaper" against "clearer" before you commit to a trimmed version.

The Bloated vs. Trimmed Prompt: A Real Token Audit

Below is an actual prompt rewritten from polite filler into a direct ask, both run through the same tokenizer. The bloated version costs 116 tokens for 109 words. The trimmed version costs 25 tokens for 19 words — a 78% cut in tokens without dropping a single instruction.

Bloated (109 words): "I was wondering if you could possibly help me out with something. So basically what I am trying to do is I need you to take a look at the user feedback that we have collected over the last quarter and just kind of generally summarize it for me in a way that would be useful, if that makes sense, for our product review meeting that is coming up next week, and please make sure to include any themes or patterns that you notice, as well as anything that seems particularly urgent or important that we should probably prioritize, thanks so much in advance for your help with this!"

Trimmed (19 words): "Summarize last quarter's user feedback for next week's product review. Highlight recurring themes and flag anything urgent to prioritize."

Read what actually got cut. The filler — "I was wondering if you could possibly," "just kind of," "if that makes sense," "thanks so much in advance" — carries zero instructions. The substance — summarize, last quarter, feedback, next week's review, themes, flag urgent items — survives intact in the trimmed version. Cutting tokens isn't about terseness for its own sake; it's about removing tokens that don't carry instruction weight.

There's a floor, though. Cut too far in the other direction — strip context the model actually needs to disambiguate the ask — and you create a different failure mode entirely. Our guide to the UX of failure in AI products maps what happens when a model is asked to work with too little context to do the job right.

From Tokens to Dollars: How This Shows Up on Your Invoice

Providers price API calls per token, and nearly every major provider prices output tokens several times higher than input tokens (commonly 3–5x). A bloated prompt taxes you twice: once going in, and again if it drags out a longer, equally bloated answer. Multiply that per-call waste by every request your feature sends, and the compounding adds up fast.

The arithmetic is unglamorous but worth doing before you scope a feature. Trim 90 tokens per call — roughly what our bloated-to-trimmed prompt saved above — across 10,000 calls a day, and you've removed 900,000 tokens of daily input volume. None of that shows up if you're sizing the feature by "how many words is a typical prompt," which is exactly why token-level thinking belongs in the estimate, not just the postmortem.

Five levers determine what actually lands on the bill:

Two of these deserve a closer look. How a retrieval-augmented feature decides what to fetch — and how many tokens that fetch costs before the model even starts reasoning — is the subject of our embeddings explained for PMs guide. And every additional planning loop in an agent workflow re-sends context on top of whatever it already sent, which is why our agentic workflows complete guide treats token budget as a first-class design constraint, not an afterthought.

Providers have started building product answers to the "repeated context" row directly. Anthropic, for example, offers prompt caching on Claude models specifically so a large, unchanging system prompt or reference document isn't re-billed at full price on every single call. That's a real mechanism worth asking your engineering team whether you're using.

Building Token Discipline Into Your Product Workflow

Treat token count the way you'd treat any other cost driver: estimate it during discovery, put an explicit budget on it, and test before you ship. A feature that looks cheap against three tidy demo prompts can look completely different once real users send real, messy, run-on input.

A practical checklist:

1. Estimate token load during discovery, not after launch. Forecasting typical prompt and context length belongs in the same discovery pass where you're validating the problem itself — our product discovery complete guide covers how to build that estimate alongside the rest of your problem framing.
2. Set a token budget per interaction, not just a dollar budget per month. A dollar number hides which specific call is expensive; a token budget per call doesn't.
3. Draft two versions of any prompt template — a bloated first pass and a trimmed one — and compare their token counts before picking either.
4. Cap output tokens for anything that doesn't need a long-form answer; verbose responses cost more per token than the prompt that triggered them.
5. Re-check token counts whenever tone or formatting guidelines change, not only when the underlying content changes — a "friendlier" phrasing pass can quietly add tokens.

This is the muscle Prodinja's Context Engineering exercise, part of its Studio suite, is designed to train. You specify what a model should see, what to leave out, and a token budget for the packet — and its critique is built to pressure-test that packet for bloat before it reaches a live model.

It's a rehearsal space for the habit this article has been arguing for: watch the token count move in response to your own phrasing choices before that phrasing ever turns into an invoice line, not after.

Key Takeaways

• Models bill in tokens, not words; roughly 1 token ≈ 0.75 English words is a useful sizing average, not a per-sentence guarantee.
• Byte-pair encoding chunks text into common sub-words; rare words, jargon, and formatted numbers get sliced into multiple tokens each.
• Extra whitespace and decorative punctuation can inflate token count without adding a single additional word.
• In a real side-by-side test, trimming filler from a prompt cut token count by 78% while preserving every original instruction.
• Output tokens typically cost several times more than input tokens, so a bloated prompt that drags out a bloated answer taxes you twice.
• Multi-step agent workflows and retrieval-augmented (RAG) features compound token cost per call, not just per feature.
• Estimating token load belongs in product discovery and prompt-template design — not in the meeting where someone asks why the invoice tripled.