A token is the small unit of text that an AI model actually reads and writes. It is not quite a word and not quite a letter: it is a chunk, often a whole short word, part of a longer word, a number or a punctuation mark. Tokens matter because AI usage is measured in them. The more tokens you send in and the more the model writes back, the higher the cost and the closer you get to the model's limits.
How text becomes tokens
Before a model like Claude processes your message, the text is split into tokens by a tokenizer. A common, short word may be a single token. A rare or long word is usually broken into several pieces. Spaces, punctuation, numbers, code symbols and line breaks all take up tokens too.
Two practical consequences follow. First, you cannot reliably convert words to tokens in your head, so treat any rule of thumb as a rough guide only. Second, different languages and content types split differently. Text in Bahasa Malaysia, Chinese, tables, code or long product codes may use tokens at a different rate from plain English prose. If cost or limits matter to your use case, measure with your own real documents rather than assuming.
Why tokens drive cost
Most AI services that you pay for by usage charge for the tokens that go in (your prompt, attached documents, earlier conversation) and the tokens that come out (the answer). Output tokens are typically priced differently from input tokens, and the exact rates change over time, so always check the provider's current pricing page rather than relying on an article, including this one.
What stays constant is the logic: cost grows with volume. One person asking occasional questions barely notices it. A workflow that sends a 40-page document to the model for every one of 500 customer enquiries a day will notice it quickly. This is why token awareness matters most when you move from experiments to automation.
Why tokens also set limits
Every model can handle only a certain number of tokens at once. That ceiling is its context window, and it covers everything: your instructions, the documents you attach, the conversation so far and the answer being written. When a long conversation runs past the limit, older material may be dropped or the request may fail.
This is the reason a model can seem to "forget" something said early in a very long chat. It is not a memory fault; the content may simply no longer fit, or it may be getting less attention among a large amount of other text.
Practical ways to keep token usage efficient
- Send only what is needed. If the task concerns section 4 of a contract, send section 4, not the whole file.
- Start fresh conversations for new tasks. Carrying a long unrelated history into a new request adds cost and noise.
- Be specific about the output. Ask for a three-bullet summary rather than "tell me everything". Shorter answers cost less and are easier to review.
- Keep instructions tight. Clear, concise prompts usually perform as well as long, repetitive ones. Our guide to writing a good prompt for Claude shows how.
- Reuse repeated material wisely. If the same large block of text is sent again and again, look at prompt caching, which is designed for that situation.
- Choose the model to fit the task. Simple, high-volume tasks may not need the largest model. See bigger vs smaller AI model for the trade-offs.
Do not cut so much that quality suffers
Saving tokens is not the goal in itself. A prompt stripped of context can produce vague or wrong answers, and the time your staff spend fixing them costs more than the tokens you saved. The sensible approach is to get the task working first, then trim what is clearly unnecessary and check that the output quality holds.
For business owners, a useful habit is to estimate usage before you scale: how many requests per day, roughly how much text goes in each, and how long the answers are. A small pilot with real data gives you a far better cost picture than any estimate. Our article on how to run an AI pilot covers how to set one up.
Where tokens show up in everyday tools
If your team uses a chat app on a fixed subscription, you may never see a token count, but the limits still exist in the background as message length caps or usage allowances. Tokens become visible when your developers or automation tools call the model through an API, where billing follows usage directly. Managers do not need to count tokens personally. They do need to know that long prompts, large attachments and high volumes are the three levers that move the bill.
Learn this hands-on
Understanding tokens, context and cost is part of using AI responsibly at work, and it is covered in practical terms in our workshops. See AI training in Malaysia for programs from Agmo Studio, a Select Partner of the Anthropic Claude Partner Network.