Episode 1 — Why Do Transformers Think in Vectors?
Episode 1 — Why Do Transformers Think in Vectors?
When you read the word apple, you immediately recognize it as a fruit. A computer doesn't. To a computer, the word apple is simply a sequence of characters. It has no meaning. It doesn't know whether "apple" is a fruit, a company, or part of a sentence. So the first challenge in building a transformer is this: How do we represent words in a form that a computer can understand? One obvious idea is to assign every word a number. For example, Apple = 1 Car = 2 Tree = 3 But this creates a new problem. The numbers 1, 2, and 3 don't carry any meaning. The computer cannot tell that apple and orange are more similar than apple and airplane. To represent meaning, we need something much richer than a single number. We need a collection of numbers that together describe different properties of a word. That collection of numbers is called a vector. Every word is transformed into a vector before it enters a transformer. From this point onward, the transformer never sees words again. It only processes vectors. In the next episode, we'll answer the next question. How does a sentence become the individual pieces that are eventually converted into vectors?