What a vector actually is

Every dot below is a word. Not a picture of a word and not a definition of one. This is close to how a language model actually stores it, flattened onto a page so it fits.

Hover a few of them. The words you would file together have landed together, and nobody sorted them by hand.

So here is the question this lesson answers. What, exactly, is one dot?

You can probably guess that the answer involves numbers. By the end of the next fifteen minutes that guess will be precise enough to build on, which matters because this one object is what every model in every later course is made of.

Describing things as ordered lists of numbers

Before going back to language models, describe three ordinary things using nothing but numbers.

A student's report card:

One pixel on the screen you are reading this on:

What someone likes to eat:

Three completely unrelated things. A person, a dot of light, a preference. Look at what the three descriptions have in common anyway.

Each one uses a fixed number of slots. Each slot means one specific thing, and it means that same thing every time. And once you agree what the slots are, the description is nothing but numbers in a set order.

That shared skeleton is the whole idea. It has a name.

What a vector is, and why order matters

A vector is an ordered list of numbers where every slot has a fixed meaning.

The word sounds like it belongs in a physics class. It does not. Nothing in that sentence is about arrows, force, or space, and you have already written three vectors without being told what one was.

The word ordered in that definition is doing real work, so it is worth breaking on purpose. Below, the labels stay where they are and the numbers move.

Same three numbers, and now a different student. Slot two means english for everyone or it means nothing for anyone.

That is why a vector is not a bag of numbers you can rearrange. It is a list with fixed positions, and later on, when two vectors get compared slot by slot, this is the property that makes the comparison mean something.

The same vector as a list, arrow, point and data row

Here is where linear algebra usually loses people. A vector gets introduced as a list, then two pages later it is an arrow, then it is a point on a graph, and nobody says out loud that these are the same object.

They are the same object. Below, the numbers, the arrow, the point and the row of data are all one vector, and every panel is wired to the same value.

Type a number and the arrow moves. Drag the arrow and the numbers change. Press the button and the arrow drops away, leaving the point at its tip.

Get comfortable switching between these views, because later lessons switch without warning. A dataset gets drawn as a cloud of points. A transformation gets drawn as arrows being moved. Those are the same picture, and knowing that is what stops the switch from feeling like a trick.

Vectors beyond three dimensions

Two numbers fit on a page. Three, with some squinting. The word you hovered at the top of this lesson had one hundred.

Climb the rungs below and watch what happens to the picture.

The arrow gives up at four. That is not a failure of your imagination, and it is not something you get better at with practice. There is no fourth direction on a flat page, so there is nothing to draw.

Notice what does not give up. The list survives every rung, and so does every piece of arithmetic you can do to it. Adding, scaling, comparing, measuring distance: none of it needs a picture, at any size.

This is the single most useful attitude to carry out of this lesson. The picture is a convenience for the two cases where it happens to fit. When it stops working, you have lost a convenience, not the maths.

Vector notation and indexing

Now that all of it is in your head, here is the shorthand for writing it down.

A vector with $n$ numbers is written $v \in \mathbb{R}^n$, which reads as "v is a list of $n$ real numbers." The report card above is $v \in \mathbb{R}^3$. A word from the map at the top is $v \in \mathbb{R}^{100}$.

Individual slots get a subscript, so $v_2$ is the second slot. Books usually draw a vector as a column rather than a row, purely because it makes the matrix rules in later lessons line up neatly.

None of that is new information. It is the same object you just dragged around, written in fewer characters.

In code

Everything above is one line of NumPy. Run this, then change a number and run it again.

One detail worth planting now, because it causes more real bugs than anything else in this course. The shape (3,) is not the same as the shape (3, 1). The first is a flat list of three numbers. The second is a table with three rows and one column. NumPy treats them differently, and a later lesson in this course, on tensor shapes, is about exactly what happens when you mix them up.

Word embeddings are vectors

Go back to the word map, now with one word selected and its closest neighbours lit up.

Each dot is exactly what you have been working with all lesson: an ordered list of numbers, each slot meaning one fixed thing, too many slots to draw.

Try apple. Its nearest neighbours include software and computer, because in the text this model learned from, apple is a company at least as often as it is a fruit. Nobody wrote that rule down. It fell out of the numbers.

Which leaves one loose end. All lesson the word close has been doing a lot of work, and it has not been defined once. Two vectors sit near each other, fine, but near according to what?

That is a measurement, and measuring it is exactly what the next two lessons build.