Why does every token cost money?

1How does one token turn into a bill?

Every token is work for a GPU, a computer chip that does thousands of sums at the same moment. Running GPUs costs the company money every hour, so it charges you for every token.

The chain, step by step:

1. 📝 You send text. For example: Plan a 3-day trip to Goa.
2. 🔤 The text is cut into tokens, about 10 here (lesson 3).
3. 🧮 Each token needs crores of small sums (section 2).
4. ⏱️ A GPU does those sums (section 3).
5. 💸 Every hour a GPU runs costs the company money.
6. 🧾 So it charges for every token: the ones you send and the ones the model writes back.

• 💲Prices are given per million tokens (10 lakh tokens). Claude Opus 5.5 charges $4 for every 10 lakh tokens you send.
• 🪙One token costs very little. $4 ÷ 10,00,000 = $0.000004 for one token you send to Claude Opus 5.5.
• 📦The bill grows with the number of tokens. 1 lakh questions of 1,000 tokens each are 10 crore tokens. Sending them to Claude Opus 5.5 costs $400.

2How much work is one token?

One token needs about one multiply-and-add for every dial in the model. A multiply-and-add is one multiplication plus one addition. The dials are the model's parameters (lesson 2).

What the model does with one token:

1. 📥 The token goes in as a list of numbers (lesson 4).
2. ✖️ The numbers are multiplied by the dials and added up, layer after layer. A real layer also does a little more (lesson 2).
3. 🎯 Out comes a chance for every possible next token, and one token is picked (lesson 6).

• 🔢In a small network, you can count the sums. Lesson 2's tiny network has 6 dials, so one pass did 6 multiplications: 1 × 3, 2 × 1, 1 × 2, 2 × 4, 5 × 2 and 10 × 1.
• 🦙A real model has crores of dials. Llama 3 8B, an open model from Meta, has 8 billion (800 crore) dials. So one token takes about 800 crore multiply-and-adds.
• 🧮Many of these sums can be done at the same time. In lesson 2's network, 1 × 3 does not need the result of 1 × 2. So a chip can work out both at the same moment.

3Why does a model need a GPU?

A model needs a GPU because a GPU does thousands of sums at the same moment. A normal computer chip, the CPU, does only a few at a time.

Why GPUs cost the company a lot:

• 💾The model's dials must sit in the GPU's own memory while it runs. Llama 3 8B's 800 crore dials take 16 GB of memory, 2 bytes each (lesson 2).
• 🖥️A big model needs several GPUs at once. Llama 3 70B's dials take about 140 GB. An NVIDIA H100, a common data-centre GPU, holds 80 GB, so this model needs at least two.
• 💸GPUs are often rented by the hour. In October 2026, one rental company listed an H100 at $3.99 an hour.
• 🔌GPUs use a lot of electricity. A server for large models uses 10,000 to 15,000 watts. A home geyser uses about 2,000 watts, so one server uses as much power as 5 to 7.5 geysers.

4Why does the model's answer cost more than your question?

On Claude's price list, an output token (a token the model writes) costs 5 times as much as an input token (a token you send). Anthropic does not say why.

Engineers explain it with rounds of work: the model reads your whole question in one round, but writes the answer one token per round. Use the buttons to go through the first reply of the Goa chat, one step at a time:

Why a round that makes one token is costly:

• 🚚In every round, the GPU moves all the dials from its memory to its cores. For Llama 3 8B, that is about 16 GB moved in every round.
• 🍳When reading, one round serves many tokens. When writing, one round serves one token. A cook brings the whole spice box from the store room for each trip. Ten plates ordered together need one trip. Ten plates ordered one after another need ten trips.
• ⏳So an output token keeps the GPU busy for longer. In the Goa reply on its own, round 1 handled 10 input tokens: a tenth of a round each. Every later round handled 1 output token: a whole round each.

Why the difference is 5 times, not more:

• 👥A server works on many people's chats in the same round. So one move of the dials serves many tokens, when writing and when reading. If 50 chats share a writing round, the 16 GB moved serves 50 output tokens, one for each chat. A reading round can read every chat's whole question at once, so it serves far more tokens than that. (50 is our example.)

An output token still uses more of the GPU than an input token. Claude's price list puts the difference at 5 times.

5How much does one question cost?

Prices are for 10 lakh tokens. So multiply the input tokens by the input price and divide by 10 lakh. Do the same for the output tokens with the output price. Then add the two. These are the real prices of three Claude models:

Move the sliders to set the tokens. They start at a 1,000-token question and a 500-token answer:

• 🏖️The first reply of the Goa chat is almost all answer. On Claude Opus 5.5: 10 × $4 ÷ 10,00,000 = $0.00004 for your message, and 300 × $20 ÷ 10,00,000 = $0.006 for the plan. Total $0.00604, and over 99% of it is the plan.

Why is a small model cheaper?

• 🪶Fewer dials means fewer multiply-and-adds for each token. Llama 3 70B has about 9 times as many dials as Llama 3 8B, so each token takes about 9 times the work.
• 🏷️Claude's prices are also lower for the smaller model. Claude Haiku 5.5 costs 1/40 of Claude Opus 5.5 for every token, for questions up to 1 lakh tokens.
• 🤷Anthropic does not publish how many dials each Claude model has. So the dial counts on this page come from Llama 3, whose sizes are public.

6What is a context window?

A context window is the most tokens one request can hold: your input and the model's answer together.

If a request needs more tokens than that, it does not fit.

• 📐Each model has its own size. Claude Opus 5.5, Sonnet 5.5 and Haiku 5.5 hold 1 million tokens (10 lakh). The older Claude Haiku 4.5 holds 200,000 (2 lakh).
• ➕Input and answer share the same space. Send 9,90,000 tokens into a 10 lakh window, and only 10,000 are left for the answer.
• 💸You pay for every token you put in it. Filling Claude Opus 5.5's 10 lakh window with input costs $4, on every request.
• 🔁One reason for the limit: attention (lesson 5) compares each token with every token before it. 1,000 tokens make about 5 lakh comparisons (1,000 × 1,000 ÷ 2). 10,000 tokens make about 5 crore. So 10 times more tokens means 100 times more comparisons.

7Does a model remember what you said earlier in a chat?

No. The model remembers nothing between messages. Every request starts with nothing from the messages before it. The chat app sends the whole conversation again with each new message, so the model can read it.

Use the buttons to go through one chat, one step at a time, and see what the app sends:

• 📈Each turn costs more than the one before, because the list only grows. Turn 1 sent 10 tokens. Turn 4 sent 861.
• 🗒️"Memory" features in chat apps save short notes about you and paste them into each new chat. The note User is vegetarian is added to every request. The model itself learned nothing.

Now think of a chat where you paste a 500-page PDF. The app would send every page again with every question. How does a chatbot answer from a 500-page PDF without that? Lesson 9 shows how.

← Lesson 7  ·  All lessons  ·  Next lesson →