Episode 8 — Values
Episode 8 — Values
So far, every word has done two things. It created a Query to ask what information it needs. It created a Key to describe what information it can provide. Now suppose the Query of one word closely matches the Key of another. The transformer has successfully identified a useful connection. But identifying the right word is only the beginning. The real goal is to collect information from it. That information is called the Value. A Value is the actual representation that one word shares with another. Think of it this way. A Query asks, "Who can help me? " A Key answers, "I might be able to help. " And the Value says, "Here is the information you need. " This process happens for every word in the sentence. Each word gathers values from many other words, but not equally. Some words contribute a lot. Others contribute very little. The transformer now has one final task. How does it decide exactly how much importance to give to each Value? That decision is made by comparing Queries and Keys mathematically. The result is called the Attention Score. That's what we'll explore in the next episode.