How does a model know which earlier words matter?

1Why can't a model tell which "bank" you mean from the word alone?

Because the word bank has two meanings, and its token is the same in both. The model needs the other words in the sentence to choose.

• 🏞️Bank can mean the land next to a river. I sat on the river bank.
• 💰Bank can also mean a place that keeps your money. I put money in the bank.
• 🏷️Both sentences use the same token for bank. In both, bank is ␣bank, id 6922. As in lesson 3, ␣ means a space that is part of the token.
• 📋As in lesson 4, the model swaps each id for its list of numbers, its embedding. One token has one list. ␣bank gets the same list in both sentences. So the list alone can't say "river" or "money".
• 👀You don't get confused, because you read the words before bank. River tells you it is land by the water. Money tells you it is the place for money.
• 🤖A model does the same. Before it decides what bank means, it looks at the words before it. At bank, the model looks back at I sat on the river.

2How does a model decide which earlier words matter?

Attention is the step where a word looks at every word before it, and at itself, gives each one a share of 100%, and then mixes in their lists by those shares.

A big share means "this word matters a lot to me". A small share means "this word matters a little".

• ⬅️A word looks only at the words before it. It never looks at a word that comes after it. In I sat on the river bank, river looks at I, sat, on, the and itself. It can't look at bank, because bank comes later.
• 🪞A word also gives a share to itself. At bank, bank gives 15% to bank.
• 💯All the shares add up to 100%. At bank: river 60%, bank 15%, sat 10%, I 5%, on 5%, the 5%. 60 + 15 + 10 + 5 + 5 + 5 = 100. (Made-up shares.)

How does the model work out the shares? Think of a search box.

• 🔎The word the model is working on makes a search: what it is looking for. Bank's search, put into words: "a word that says which kind of bank". (Made-up words. A real search is a list of numbers.)
• 🏷️Every earlier word, and the word itself, has a tag: what it offers. River's tag, put into words: "about water". The tag of the: "a small linking word". (Made-up words. A real tag is a list of numbers.)
• 🤝The model compares the search with each tag and gets a match number. A better match gives a bigger number. At bank: river 12, bank 3, sat 2, I 1, on 1, the 1. (Made-up numbers.)
• ➗The model turns the match numbers into shares that add up to 100%. Here we divide each match number by the total. The total is 12 + 3 + 2 + 1 + 1 + 1 = 20, so river gets 12 ÷ 20 = 60%.
• 📐Real models use a different formula for this step. A bigger match number still always gets a bigger share. The real formula would give river even more than 60%.
• 🎛️The searches and tags come from dials the model learned in training, as in lesson 2. Nobody typed "river is about water" into the model. Training nudged the dials until river's tag matched bank's search.

3How does attention change the numbers for "bank"?

Attention mixes the earlier words' lists into bank's list. A word with a big share adds a lot, and a word with a small share adds a little.

1. 1️⃣Take each word's share of its list. In I sat on the river bank, river has 60%, and the first number in its list is 1.00. 60% of 1.00 is 0.60.
2. 2️⃣Add up those parts. The total is the mix. The first numbers of the six parts (I, sat, on, the, river, bank): 0.01 + 0.02 + 0.01 + 0.01 + 0.60 + 0.06 = 0.71.
3. 3️⃣Add the mix to bank's own starting list. Bank's first number starts at 0.40. 0.40 + 0.71 = 1.11.

• 🔀Bank starts with the same list in both sentences, and ends with a different one. Start: 0.40, 0.40 in both. After the river sentence: 1.11, 0.49. After the money sentence: 0.49, 1.11.
• 🧩So inside a model, a word's meaning is not fixed. It is its starting list plus what the words before it added. In the river sentence, bank's water number grew most. In the money sentence, its money number grew most.

4How does a model know what "it" refers to?

The model looks back at the earlier words for one that fits. In The animal didn't cross the street because it was too tired, "it" is the animal.

• 🔄Change one word, and "it" changes. In The animal didn't cross the street because it was too wide, "it" is the street. A street can be wide. An animal can be tired.
• ➡️A model like ChatGPT reads from left to right. At "it", it has not seen tired or wide yet. At "it", the model has only The animal didn't cross the street because it.
• 🧩So the model finds out what "it" means one word later, at the last word. At tired, the model looks back for something that can be tired. At wide, for something that can be wide.

• 🐾At tired, the thickest lines go to animal and to it. animal 45%, it 20%, street only 6%. So the model links "tired" to the animal.
• 🛣️At wide, the thickest line moves to street. street 44%, it 16%, animal only 5%.

5What is a Transformer?

A Transformer is a design for a model: a stack of layers, where every layer has attention. Most LLMs today are built on it, including the ones behind ChatGPT.

It first appeared in 2017.

• 👀The first part of each layer is attention. Each word mixes in the lists of the words before it. In layer 1, bank's list mixes in a lot of river's list, as in section 3.
• 🧠Each layer also has a second part. It changes each word's list using only that word's list and its own dials. In the river sentence, bank's list after attention, 1.11, 0.49 (section 3), goes in. A new list for bank comes out. No other word's list is used.
• ⬆️Each layer passes its new lists up to the next layer. The next layer mixes again, starting from those new lists. Layer 2 starts from bank's new list from layer 1, not from its starting list.
• 🧱The number of layers is different in each model. Phi-3-mini, a small model from Microsoft, has 32 layers. Big models have more than a hundred.
• 🎯At the top, the list for the last word is used to give a chance to every possible next word. After The cat sat on the, mat gets a high chance, as in lesson 1.

6Why did the Transformer change so much?

The Transformer made training much faster. Faster training let people build much bigger models.

• 🐢Older designs read a text one word at a time in training. Each word had to wait for the word before it. A text of 1,000 words took 1,000 steps, one after the other.
• 🚀A Transformer, in training, works on every word of a text at the same time. All 1,000 words are worked on together. Each word still looks only at the words before it.
• 🖥️Working on every word at once is a good fit for a GPU, a chip built to do many small sums at the same time. Like 1,000 cashiers each billing one customer at once, instead of one cashier with 1,000 customers waiting.
• 📈Faster training made it possible to build bigger models and train them on far more text. Transformers grew to tens of billions (thousands of crores) of parameters. This is one big reason LLMs got much better after 2017.
• ✍️Writing an answer did not change. The model still writes one word at a time. As in lesson 1: guess "mat", add it to the text, then run the whole text through again.

After the last layer, the model has a chance for every token it knows. How does it pick just one, and why do you get a different answer when you ask again? Lesson 6 starts there.

← Lesson 4  ·  All lessons  ·  Next lesson →