How does a model know "cat" and "dog" are alike?

1Does a token's number tell the model what a word means?

No. A token id is only a label. It tells the model which token it has, but nothing about what the token means.

• 🏷️A token id works like a roll number in class. Roll number 23 tells the teacher whose name to call. It does not say if you are tall, or good at maths.
• ␣As in lesson 3, the mark ␣ means a space that is part of the token. ␣cat is a space, then cat, as in I fed my cat.
• 🔢Words with similar meanings do not get similar ids. A kitten is a baby cat, but the id of ␣kitten is far from the id of ␣cat. The id of ␣car is much nearer.

So the id can't tell the model that cat and kitten are alike. The model needs a second thing for each token, one that carries meaning. To see how a model builds it, start with how you learned what a cat is.

2How did you learn what a cat is when you were a baby?

You learned from examples, not from rules. Someone pointed at an animal and told you whether it was a cat.

• 👶Nobody gave you a list of rules. No one said "a cat has whiskers and pointy ears, and it says meow" before you knew the word.
• 👉You saw examples, each with the right answer. "That's a cat." "No, that one is a dog."
• 🔍You found the clues yourself. A clue you learn to look for is called a feature. Whiskers.
• 🎚️A feature has a value for each thing you look at. A cat: whiskers yes. A spoon: whiskers no.
• ❌Some clues you found were wrong. Every cat you saw came in the morning, and the dog came at night. So "morning means cat" seemed true.
• 🗑️A wrong clue drops away when an example does not match it. One night you see a cat. Now the time of day no longer tells a cat from a dog.
• 💪A good clue stays, because it keeps matching. Cat after cat has whiskers, so you keep looking for whiskers.
• 🤖A model learns in a similar way. It gets examples with the right answers, finds its own features, and keeps the ones that keep matching. A model is shown thousands of photos, each marked "cat" or "not a cat". Nobody tells it to look for whiskers.

3What is an embedding?

An embedding is a list of numbers that a model keeps for every token.

The model works with this list, not with the token id.

• 📋Every token has its own list. ␣cat has one list, and ␣dog has another.
• 🎚️Think of each number as the value of one feature, like whiskers yes or no in section 2. If one number meant "is an animal", ␣cat would have a high value there, like 0.90, and ␣car a low one, like 0.05. These two numbers are made up.
• ➡️Another name for a list of numbers is a vector. "The vector for cat" and "the embedding of cat" mean the same list.

Real lists differ from this made-up table in two ways:

• 📜Real lists are much longer, and every list in a model has the same length. In Phi-3-mini, a small model from Microsoft, every token's list has 3,072 numbers.
• ❔Real numbers have no names. Nobody chooses a feature like "is an animal". Training sets every number. If you look at one number in the list for cat, say the 500th, it is only a value, like 0.31 (made up). Nobody can say which idea it stands for.

4Why do cat and dog end up with similar lists?

Because people use cat and dog in the same kinds of sentences. Training then pushes their two lists to become alike.

• 🎲At the start of training, every list is random. ␣cat could start as 0.12, 0.67, 0.40 and ␣dog as 0.81, 0.05, 0.33. Nothing alike. (Made-up numbers.)
• 🎛️The numbers in the lists are dials, the parameters from lesson 2. The 3 numbers shown for ␣cat in the table are 3 dials.
• 🔁Training turns these dials as in lesson 2: guess the next word, check the real one, nudge the dials. The model reads My cat is and guesses the next word. The real next word is sleeping.
• 🔧The list of a token gets nudged only when that token was part of the guess. A dial that did not affect the guess does not move, as in lesson 2. In My cat is, the list for ␣cat was used, so it gets nudged. The list for ␣car was not used, so it stays the same.
• 👯Words used in the same places get nudged in similar ways. I fed my cat and I fed my dog. My cat is sleeping and My dog is sleeping.
• 🚗A word used in other places gets nudged in other ways. I parked the car. The car needs petrol.
• ✅After a lot of training, the lists for cat and dog usually end up alike, and the list for car ends up different. The made-up table in section 3: ␣cat 0.90, 0.80, 0.05; ␣dog 0.85, 0.90, 0.05; ␣car 0.05, 0.10, 0.95.

5How can a computer measure how close two meanings are?

One simple way: compare the two lists number by number, and add up the differences. A small total means the meanings are close, and a big total means they are far apart.

1. 1️⃣Put the two lists side by side. ␣cat is 0.90, 0.80, 0.05. ␣kitten is 0.88, 0.85, 0.02.
2. 2️⃣For each position, take the bigger number minus the smaller one. The first numbers: 0.90 − 0.88 = 0.02.
3. 3️⃣Add up the differences. We call this total the gap. 0.02 + 0.05 + 0.03 = 0.10.
4. 4️⃣A small gap means close in meaning. A big gap means far apart. The gap from ␣cat to ␣kitten is 0.10. The gap to ␣car is 2.45.

• 🏷️The ids play no part in this. The id of ␣kitten (114502) is far from the id of ␣cat (9059), but its list has the smallest gap of the three.

6Can a model learn the wrong clue?

Yes. A model keeps any clue that keeps matching its training examples, even a clue that is wrong. In section 2, "morning means cat" stayed until a cat came at night.

A real case from 2018, with chest X-rays:

• 🩻Researchers trained a model to spot pneumonia, a lung illness, in chest X-rays from 3 hospitals. At one hospital, about 34 in 100 X-rays showed pneumonia. At the other two, about 1 in 100. The model learned to tell the hospitals apart. One clue was a small metal marker that shows in the corner of the X-ray.
• 📉The hospital was a wrong clue that kept matching. If the model spotted the hospital, it could often guess pneumonia right without looking at the lungs. On X-rays from a hospital it had not trained on, the model did worse in 3 of 5 tests.

Embeddings can pick up a wrong clue from text:

• 🧑‍⚕️Models often learn lists where doctor is closer to man, and nurse is closer to woman. If more texts say "the doctor said he was late" and "the man said he was late", then doctor and man are used in the same places. As in section 4, their lists get nudged in similar ways. The same happens for nurse and woman.
• 📚This is not a fact about the world. It is a pattern in the text the model read, unfair parts included. A woman can be a doctor, but the lists can still put doctor closer to man.

Each token gets only one list. ␣bank has one list, but a river bank and a money bank are different things. How does a model work out which bank you mean? Lesson 5 starts there.

← Lesson 3  ·  All lessons  ·  Next lesson →