Brown University crest
CPSY 1291 Computational Methods for Mind, Brain & Behavior
Lecture 15 · Theme 3: Dynamics, sequences & world models

Attention

Tuesday, November 10 · Fall 2026

Reader-friendly version — lecture content with figure descriptions

Before 2017: one architecture per field

Six tiles, one per field of AI before 2017, each with its own kind of network and a small drawing of it. Vision: convolutional networks, drawn as shrinking stacks of feature maps (AlexNet 2012, ResNet 2016). Language: recurrent networks, a chain of cells reading the, cat, sat, on one word at a time. Speech: deep belief nets with hidden Markov models, drawn as stacked layers. Translation: sequence-to-sequence networks, an encoder chain feeding a decoder chain. Decisions and robots: reinforcement learning, an agent and the world in a loop (DQN 2015). Proteins: hand-built pipelines of align, score and fold steps

Idea after Lucas Beyer's transformer lecture

Today: one architecture wins them all

The same six tiles, now each with the same drawing: a transformer block, a small network above a highlighted attention step, repeated L times. Vision: DINOv2 and Segment Anything; images and video: Stable Diffusion 3 and Sora. Language: ChatGPT, Claude, Gemini. Speech: Whisper, speech recognition in about 100 languages. Translation: the first transformer, 2017, now done by chatbots too. Decisions and robots: Decision Transformer and RT-2, a robot's actions as tokens. Proteins: AlphaFold 2 and 3, structure from the amino-acid sequence

  • All are transformers, from one paper: "Attention is all you need" (2017), cited about 200,000 times

Step one: everything becomes tokens

Three rows, each read from left to right: input, tokens, one vector per token. Text: the cat sat on the mat is cut into word pieces, the first four shown: the, cat, sat, on. Image: a photograph of a tiger is cut into a 4 by 4 grid of patches, and four patches are shown as tokens. Sound: a waveform is cut into short slices, four of them shown as tokens. In every row, each token becomes a small vector drawn the same way

  • Each token becomes a vector. After that, the transformer does not care what a token stood for
  • Today our tokens are characters; the next lecture shows how pieces of words are chosen

Inside a transformer

Schematic of a transformer, read from the bottom up, with one column per token. At the bottom, six tokens, the characters t, h, e, a space drawn as an underscore, c and a. Each becomes a small blue vector, labelled embedding plus position. A grey line runs up from each vector, the token's residual stream. Inside a dashed frame marked times L blocks, a gold self-attention band spans all six columns: lines from every column fan in to the last one, so the last token reads itself and the five earlier tokens and none after it. Above it, each column has its own small box labelled MLP, the small network that works on each token alone. At the top, an output layer and softmax spans the columns and gives each column's predicted next token: h, e, space, c, a, t, the input shifted by one

  • Here each token is one character, as in the model you train. Read the figure from the bottom: one column per token
  • Each token becomes a vector: embedding + position embedding
  • LL blocks, each with its own weights, in two steps:
    • Causal self-attention: each token reads only itself and earlier tokens
    • A small network, the same for every token, works on each token alone
  • The output layer predicts the next token

Most of a transformer is not new

Part of a transformer Where it comes from
Embeddings: each token becomes a learned vector Language models, 2003 old
A small network at every position The multilayer perceptron old
Residual connections ResNets, 2016 old
Layer normalization Ba, Kiros & Hinton, 2016 old
Self-attention: each token reads the earlier ones The transformer, 2017 new
  • Old pieces: embeddings come from language models of the 2000s; the rest from the networks we have already seen, mostly CNNs

Most of a transformer is not new

Part of a transformer Where it comes from
Embeddings: each token becomes a learned vector Language models, 2003 old
A small network at every position The multilayer perceptron old
Residual connections ResNets, 2016 old
Layer normalization Ba, Kiros & Hinton, 2016 old
Self-attention: each token reads the earlier ones The transformer, 2017 new
  • Old pieces: embeddings come from language models of the 2000s; the rest from the networks we have already seen, mostly CNNs
  • The one new piece is self-attention

Self-attention at work

Two columns of the same sentence, The Law will never be perfect, but its application should be just. A thick purple line runs from its in the right column to Law in the left column, which is shaded strongly; a thinner pink line runs to application, shaded more lightly. No other word receives a line.

Published: both columns are the same sentence; its (right) reads the words on the left

  • The first transformer translated English into German. Both columns are the English sentence it reads first; its tokens are words, not characters
  • An attention head is one way of weighting the other words. A model runs several heads side by side: this one has 8 in each of its 6 reading blocks. Each color here is one head
  • Both heads put most of their weight on Law, the noun its refers to; one also reads application
  • Nobody told the model about grammar: it learned where to look from 4.5 million English–German sentence pairs

Why attention?

Readers predict what comes next. A model that predicts needs the right part of what came before

Read this

PARIS
IN THE
THE SPRING

Answer this

How many animals of each kind
did Moses take on the ark?

You read what you expect

  • The triangle says Paris in the the spring. Most readers miss the second the
  • It was Noah, not Moses, who took the animals. Most people answer "two" without noticing (Erickson & Mattson 1981)
  • Raeding wrods with jubmled lettres is still easy, though a little slower (Rayner et al. 2006)
  • Readers predict the next letters and words, and notice the surprises that break the prediction: He spread the warm bread with socks (Kutas & Hillyard 1980). A language model is trained to do the same

Predict what comes next

  • Machines can predict too. A language model assigns probabilities to text; the ones in this lecture, like the chatbots, predict the next token from the ones before
  • Our running example: the next character of TinyStories, the stories of Assignment 4 (97 possible characters)
  • At position tt the model has read tokens 1,…,t1,\dots,t, its context, and predicts token t+1t+1: the tokens and predicted next tokens of the transformer figure, bottom and top

Two rows of character boxes. The bottom row, in blue, is the input: t, h, e, underscore for a space, c, a, t, underscore, s, a. The top row, in green, is the target at each position: the next character, h, e, underscore, c, a, t, underscore, s, a, t. A gray arrow runs up from each input box to its target

  • Which of the tokens read so far should the prediction use?

One token at a time, or the whole window at once

One token at a time

  • People read one word after another
  • So do recurrent networks, a later lecture: one memory vector is carried forward, and everything read so far must fit in it. For them tt is a time step

The whole window at once

  • A transformer gets the whole window of text at once: its context
  • For a transformer, tt is a position in the window. Position tt can read every earlier position directly. A causal mask, later today, hides the later ones: they hold what it must predict
  • "Earlier" means earlier in the window: all of the context is there at the same time

One character of context: a bigram table

A five-by-five table for the made-up toy text that hat ate the tea, rows the current symbol and columns the next, for a, e, h, t and the space, drawn as an underscore, with a row-sum column: the number of times each symbol follows each other one in the toy text: for example a is followed by t 3 times, t by e, h or a space 2 times each, and many pairs never occur

A made-up toy text with five symbols: "that hat ate the tea"

  • Long before transformers, the simplest model used one character of context: count how often each symbol follows each other one in the training text
  • One character already says a lot: after q comes u; after a space, a new word starts
  • Such tables were the standard language models for decades, in speech recognition and translation

One character of context: a bigram table

A five-by-five table for the made-up toy text that hat ate the tea, rows the current symbol and columns the next, for a, e, h, t and the space, drawn as an underscore, with a row-sum column: the same table with 1 added to every count: no cell is 0 any more, and each row sum grows by 5

A made-up toy text with five symbols: "that hat ate the tea"

  • Long before transformers, the simplest model used one character of context: count how often each symbol follows each other one in the training text
  • One character already says a lot: after q comes u; after a space, a new word starts
  • Such tables were the standard language models for decades, in speech recognition and translation
  • A pair never seen in training gets probability 0, and an infinite loss on new text. Smoothing: add 1 to every count

One character of context: a bigram table

A five-by-five table for the made-up toy text that hat ate the tea, rows the current symbol and columns the next, for a, e, h, t and the space, drawn as an underscore, with a row-sum column: the smoothed counts divided by their row sum, so every row holds probabilities that add up to 1: for example P(t | a) = 1/2 and P(e | t) = 3/11

A made-up toy text with five symbols: "that hat ate the tea"

  • Long before transformers, the simplest model used one character of context: count how often each symbol follows each other one in the training text
  • One character already says a lot: after q comes u; after a space, a new word starts
  • Such tables were the standard language models for decades, in speech recognition and translation
  • A pair never seen in training gets probability 0, and an infinite loss on new text. Smoothing: add 1 to every count
  • Divide each row by its sum: each row now holds probabilities

Scoring a model: the cross-entropy

  • To compare models, score them on text they never saw: at each position, look up the probability pp the model gave to the character that came next, and take −ln⁡p-\ln p
  • Average over every position: the cross-entropy, in nats. Lower is better
Toy text tea Next pp from the table −ln⁡p-\ln p
after t e 3/11 1.30
after e a 1/4 1.39
average 1.34
  • The goal: predict the next token as well as possible. A model that uses more context than one character should beat the bigram table's score

Why not count longer contexts?

Characters of context Possible contexts
1 97
2 9,409
3 912,673
4 88,529,281
  • Each extra character multiplies the number of contexts by 97, and each context needs 97 counts, one per possible next character. Most long contexts never occur in the training text
  • We need a way to use long contexts without a separate count for every possible context

Symbols become vectors

  • Instead of a table of counts, give each token a vector that training adjusts
  • The plain code is one-hot: 97 numbers, all 0 but one. Every character is then equally different from every other: the dot product of any two is 0
  • An embedding table holds one learned vector per token, with dmodeld_{\text{model}} numbers: ei\mathbf{e}_i for the token at position ii. Tokens that behave alike can end up with similar vectors: vowels, or, when tokens are words, cat and dog
  • Multiplying a one-hot vector by the table picks out one row: the table is an ordinary weight matrix, and "looking up" a token is that product
  • A window of TT tokens becomes a matrix, one row per position: X∈RT×dmodel\mathbf{X}\in\mathbb{R}^{T\times d_{\text{model}}}

Position embeddings: where is each token?

  • A token's embedding says what the token is, not where it sits
  • Yet dog bites man and man bites dog mean different things, and the steps that come next treat their vectors the same whatever the order
  • Fix: a second table, the position embedding, with one learned vector per position, added to the token's vector:

xi=ei+pi\mathbf{x}_i=\mathbf{e}_i+\mathbf{p}_i

  • The original transformer used fixed sine waves of different frequencies instead of a learned table

Look back at everything: average

Position ii is the ii-th token of the window, with vector xi\mathbf{x}_i. The simplest way to use the whole context: give position ii a new vector yi\mathbf{y}_i, the average of positions 1 to ii

yi=1i∑j≤ixj\mathbf{y}_i = \frac{1}{i}\sum_{j\le i} \mathbf{x}_j

the cat sat
vector xj\mathbf{x}_j (made up) (1,0)(1,0) (0,1)(0,1) (0.5,0.5)(0.5,0.5)
weight (equal) 13\tfrac13 13\tfrac13 13\tfrac13
  • For sat, i=3i = 3: y3=13(1,0)+13(0,1)+13(0.5,0.5)=(0.5, 0.5)\mathbf{y}_3 = \tfrac13(1,0)+\tfrac13(0,1)+\tfrac13(0.5,0.5)=(0.5,\,0.5). But every word counts the same
  • What is lost: the counts as much as cat, the word that says who sat. In a story of 500 tokens, a name read once gets a weight of 1/500

From the new vector to a prediction

Schematic of the model so far, read from the bottom up, with one column per token: the characters t, h, e, a space drawn as an underscore, c and a. Each becomes a small blue vector x, embedding plus position. A gray line runs up each column. In a taupe band labelled mixing step y, lines of equal width run from every column to the last one: the average, equal weights; next, attention. Above it a bar labelled output layer W out plus softmax spans the columns. Above the last column, small green bars give the probabilities of the next character: t is the tallest, then n, r and the space

  • At every position ii, the output layer, a weight matrix Wout\mathbf{W}_{\text{out}}, turns yi\mathbf{y}_i into 97 scores, one per possible next character
  • A softmax turns the 97 scores into probabilities; the cross-entropy trains every weight, as in the backpropagation lecture
  • Only the mixing step looks at other positions. The average was one mixing step; attention, next, is a better one

Choose what matters: weights from similarity

  • Keep the weighted sum, but let the weights depend on the tokens: yi=∑j≤iaij xj\mathbf{y}_i = \sum_{j\le i} a_{ij}\,\mathbf{x}_j
  • Score each earlier position jj by how well it matches position ii, a dot product: sij=xi⋅xjs_{ij} = \mathbf{x}_i\cdot\mathbf{x}_j. Two vectors match when they point the same way: in the queen said that she, she and queen share directions (a person, female), so she scores queen high and said low
  • A softmax over jj turns the scores into weights that sum to 1:

aij=esij∑j′≤iesij′a_{ij}=\frac{e^{s_{ij}}}{\sum_{j'\le i} e^{s_{ij'}}}

  • This is attention: positions that match count more

Self-attention, step by step

How does one word decide which other words to read?

A memory you can search

  • First, attention on its own: one question, a few stored items, nothing trained
  • Three objects were put away. Each has a key, its description, and a value, where it is:
key value
red triangle desk
red circle shelf
blue triangle drawer
  • The query is the question: where is the red triangle?
  • Attention scores the query against every key, turns the scores into weights with a softmax, and returns the weighted mix of the values. Which location comes back most?

Try it: can attention find it?

Each position plays three roles

Each position makes three vectors from its own x\mathbf{x}, each with its own learned matrix:

  • Query qi\textcolor{#B5396B}{\mathbf{q}_i}: what position ii is looking for
  • Key kj\textcolor{#3E6D8E}{\mathbf{k}_j}: what position jj offers to be matched on
  • Value vj\textcolor{#3F7A6B}{\mathbf{v}_j}: what position jj passes on when it is read

As in "Choose what matters", the new vector at position ii is a weighted sum with softmax weights; now it sums the values, and the weights come from how well the query matches each key:

yi=∑j≤iaij vj,aij=e qi⋅kj/dk∑j′≤ie qi⋅kj′/dk\mathbf{y}_i=\sum_{j\le i} a_{ij}\,\textcolor{#3F7A6B}{\mathbf{v}_j},\qquad a_{ij}=\frac{e^{\,\textcolor{#B5396B}{\mathbf{q}_i}\cdot\textcolor{#3E6D8E}{\mathbf{k}_j}/\sqrt{d_k}}}{\sum_{j'\le i} e^{\,\textcolor{#B5396B}{\mathbf{q}_i}\cdot\textcolor{#3E6D8E}{\mathbf{k}_{j'}}/\sqrt{d_k}}}

dkd_k: the number of entries in each query and key (the next slides explain the division)

Queries, keys and values come from the same input

  • Each position's query, key and value come from its own vector, through three learned matrices. For all positions at once:

Q=XWQ,K=XWK,V=XWV\textcolor{#B5396B}{\mathbf{Q}}=\mathbf{X}\mathbf{W}_Q,\qquad \textcolor{#3E6D8E}{\mathbf{K}}=\mathbf{X}\mathbf{W}_K,\qquad \textcolor{#3F7A6B}{\mathbf{V}}=\mathbf{X}\mathbf{W}_V

  • Row ii of Q\textcolor{#B5396B}{\mathbf{Q}} is position ii's query; the same holds for keys and values
  • Why three matrices: what a token looks for, what it offers to be matched on, and what it passes on can then differ. A name can be matched because it is a name, and pass on its letters
  • All three come from the same X\mathbf{X}, so this is self-attention. When the queries come from one sequence and the keys and values from another, it is cross-attention: German words reading the English sentence, or words reading the patches of an image

Scaled dot-product attention

  • The same attention, yi=∑j≤iaij vj\mathbf{y}_i=\sum_{j\le i} a_{ij}\,\textcolor{#3F7A6B}{\mathbf{v}_j}, for all positions at once, as matrix products: one step that a GPU computes fast
  • Scores, every query against every key: S=QK⊤\mathbf{S}=\textcolor{#B5396B}{\mathbf{Q}}\textcolor{#3E6D8E}{\mathbf{K}}^\top, a T×TT\times T table with sij=qi⋅kjs_{ij}=\textcolor{#B5396B}{\mathbf{q}_i}\cdot\textcolor{#3E6D8E}{\mathbf{k}_j}
  • Divide by dk\sqrt{d_k}, where dkd_k is the number of entries in each query and key: dot products of long vectors are large, and a softmax of large scores puts all the weight on one position
  • Softmax along each row, then average the values with those weights:

Attention(Q,K,V)=softmax ⁣(QK⊤dk)⏟A V\mathrm{Attention}(\textcolor{#B5396B}{\mathbf{Q}},\textcolor{#3E6D8E}{\mathbf{K}},\textcolor{#3F7A6B}{\mathbf{V}})=\underbrace{\mathrm{softmax}\!\left(\frac{\textcolor{#B5396B}{\mathbf{Q}}\textcolor{#3E6D8E}{\mathbf{K}}^\top}{\sqrt{d_k}}\right)}_{\mathbf{A}}\,\textcolor{#3F7A6B}{\mathbf{V}}

Attention by hand: three words

Three tokens, here whole words: the cat sat. Update the last position, sat (i=3i = 3). Its query is q3=(1,0)\textcolor{#B5396B}{\mathbf{q}_3}=(1,0); keys and values are made up, two numbers each, so dk=2d_k = 2:

the cat sat
key kj\textcolor{#3E6D8E}{\mathbf{k}_j} (0,1)(0,1) (2,0)(2,0) (1,1)(1,1)
score q3⋅kj\textcolor{#B5396B}{\mathbf{q}_3}\cdot\textcolor{#3E6D8E}{\mathbf{k}_j} 00 22 11
score ÷2\div\sqrt{2} 00 1.411.41 0.710.71
weight a3ja_{3j}, softmax 0.140.14 0.580.58 0.280.28
value vj\textcolor{#3F7A6B}{\mathbf{v}_j} (1,0)(1,0) (0,1)(0,1) (0.5,0.5)(0.5,0.5)

y3=0.14 (1,0)+0.58 (0,1)+0.28 (0.5,0.5)=(0.28, 0.72)\mathbf{y}_3 = 0.14\,(1,0)+0.58\,(0,1)+0.28\,(0.5,0.5)=(0.28,\,0.72), mostly cat's value

Try it: attention, step by step

Try it: move the query

  • The same three words as the last slide: drag the query of sat and watch the weights and the output follow

One sentence, one matrix of weights

A six-by-six grid of attention weights for the sentence the cat sat on the mat, rows the word being updated, columns the word being read, where every word can read every word: sat reads cat most, on reads sat and mat. The space on the right, where the masked version appears on the next slide, is empty

Schematic: hand-set scores, softmax computed · every row sums to 1

  • Row = query word, column = key word, cell = weight aija_{ij}. No mask: every word reads every word; sat reads cat most

The causal mask: no peeking ahead

Two six-by-six grids of attention weights for the sentence the cat sat on the mat, rows the word being updated, columns the word being read. Left, every word can read every word. Right, the causal mask: every cell above the diagonal is hatched and holds zero, so each word reads only itself and the words before it

Schematic: hand-set scores, softmax computed · every row sums to 1

  • Right: the causal mask sets later words' scores to −∞-\infty before the softmax. A next-token model uses only this one

Back to the bigram table

Model Context it uses Held-out loss (nats) Effective choices, elosse^{\text{loss}}
Uniform guess none 4.57 97
Bigram table the current character 2.28 9.8
Average every earlier character, equal weights 2.03 7.6
One attention head every earlier character, chosen weights 1.23 3.4
  • Same held-out TinyStories text for every model. Effective choices: the number of equally likely characters that would give the same loss. Each step uses the context better, and the loss falls
  • The attention head is the one you build in Assignment 4, Part 1

Recap

  • A language model predicts the next symbol from the ones before. A bigram table uses one symbol of context; longer tables explode
  • Averaging the past uses all of it but weighs everything the same; attention weighs each position read so far by how well it matches what the current one looks for
  • softmax(QK⊤/dk) V\mathrm{softmax}(\textcolor{#B5396B}{\mathbf{Q}}\textcolor{#3E6D8E}{\mathbf{K}}^\top/\sqrt{d_k})\,\textcolor{#3F7A6B}{\mathbf{V}}: queries, keys and values are learned projections of the same input
  • The causal mask hides the future; position embeddings give order

Further reading for the research trail

Optional: a few research-trail entries over the semester. Any paper mentioned in any lecture qualifies, but papers from these lectures are preferred.

Minute paper

Submit on Canvas → Minute papers → Minute Paper 16 (access code read out in class).

Write three brief points in your own words:

  1. Averaging the past weighs every earlier token the same. In your own words, how does attention let a token choose which earlier tokens to read?
  2. Something you do not yet understand, or a question still open
  3. Another idea you found interesting, and why it matters for brains, behavior or AI

Credit for a thoughtful attempt, not for being correct.

- This lecture opens Theme 3, on sequences: text, speech, a stream of experience. Until now every input was taken in one piece (an image, a set of ratings). From now on the input is a sequence, and the question is how a model uses what came before - Until about 2017, each field of AI had its own kind of network, designed for its kind of data: convolutional networks for images, recurrent networks for text and speech, sequence-to-sequence models for translation - You met convolutional networks in Theme 2; recurrent networks come back in a later lecture

- Within a few years of 2017, the transformer replaced the specialized networks in almost every field. Vaswani et al. (2017) introduced it for translation; the citation count is Semantic Scholar's, November 2026 - Each system arranges and trains its blocks in its own way and adds parts of its own (AlphaFold adds a good deal). What they share is the transformer block and, in it, attention - Systems in the tiles: DINOv2 (Oquab et al. 2023) and Segment Anything (Kirillov et al. 2023) for images; Stable Diffusion 3 (Esser et al., ICML 2024) and Sora (OpenAI 2024) generate images and video from patches; Whisper (Radford et al., ICML 2023) transcribes speech; Decision Transformer (Chen et al. 2021) and RT-2 (Brohan et al. 2023) treat an agent's or a robot's actions as tokens; AlphaFold 3 (Abramson et al. 2024) predicts the structure of proteins and the molecules they bind

- Cutting the input into tokens is called tokenization. Text is cut into characters or pieces of words (common words are one token, rare words a few); an image into a grid of patches, often 16 × 16 pixels; a sound into short slices of its spectrogram, about 20 ms each; a protein into its amino acids - After this step the transformer sees only a sequence of vectors, one per token. A model can even mix them: text tokens and image tokens in one sequence - The waveform in the figure is made up for the drawing; the tiger is the photograph from the first lecture

- The lines of the attention band only go back in time: a token never reads a later one, because the model's job is to predict that later token. This is causal self-attention; the causal mask that enforces it comes later today - Read the figure from the bottom up, one column per token. The only place where one token learns anything about another is self-attention; everything else works on each column separately. That is why a transformer without attention would be a bag of unrelated tokens - The tokens here are characters, as in today's running example and the assignment. Real language models cut text into pieces of words; the next lecture shows how those are chosen. The machinery is the same - The gray line running up each column carries the token's vector from block to block; each step adds its result to it. It is called the residual stream, and the next lecture builds it - The small network has the same weights at every position within a block; each block has its own - Each step starts with a layer normalization, left out of the drawing for clarity. The next lecture assembles a complete block, normalizations included, and Part 2 of the assignment builds and trains one - A block is also called a layer, the word most papers use. GPT-2 small has 12 blocks of 12 heads each with vectors of 768 numbers; GPT-3 has 96 blocks. The design of the block is the same; only the number of blocks, the width and the training data grow

- First the old pieces. Every one of them was in use before 2017

- Bengio et al. (2003), "A neural probabilistic language model": each word gets a learned vector, its embedding, and a small network predicts the next word from the vectors of the few words before it. That is the first step of every transformer today, and the same next-word task as this lecture. Word2vec (Mikolov et al. 2013) later made word embeddings widely used - Attention itself is older than the transformer: Bahdanau et al. (2015) let a translation network look back at the words of the input sentence, and a few 2016–2017 models applied attention of a sequence on itself inside recurrent networks. The transformer's step was to build the whole network from self-attention, without recurrence - The next lecture assembles the full block: two normalizations, attention, a small network, and two residual additions

- The figure is from the first transformer paper: two of the eight heads in layer 5 (of 6) of the part of the model that reads the English sentence. The word on the right (its) is the one reading; the words on the left are the ones it reads. The thicker and darker the line and the bar, the more weight that word gets - Linking its to Law is called anaphora resolution: an anaphor is a word, such as a pronoun, that points back to something mentioned earlier, and resolving it means finding what it points to - This part of the model sees the whole sentence, so its may also read application, which comes after it. The models in the rest of this lecture predict the next word, so they may only read words that come before - One sentence shows what two heads do on that sentence, not what they do in general; the authors call them "apparently involved in anaphora resolution". The next lecture measures heads of GPT-2 over many sentences - The paper prints the words sideways; here the panel is rotated so that they read across, and cut after "should be just"

- The next three slides are quick tests on yourself: readers guess what comes next and see what they expect. Then the same job, predicting the next character, is given to a model, and the rest of this part asks how much of the text before it the model should use: one character, all of it, or the parts that matter. That last step is attention

- Read the triangle once, quickly, before going on

- Answer quickly, before going on to the next slide

- The doubled "the" sits at a line break, where the eye expects a new word; readers see the phrase they predict - The Moses illusion (Erickson & Mattson 1981): Moses fits the question well enough (a biblical figure, like Noah), so the mismatch goes unnoticed. People who know the story still answer "two" - Rayner, White, Johnson and Liversedge (2006) tracked readers' eyes: text with letters swapped inside words is still read, but more slowly, and swaps at the start of a word cost the most. Prediction helps, but readers still use the letters - Kutas and Hillyard (1980) recorded EEG while people read sentences one word at a time. A word that makes no sense where it appears, such as socks after "spread the warm bread with", evokes a negative brain wave about 400 ms after it appears, the N400; the more expected the word, the smaller the wave - A later lecture measures this directly: words that are hard to predict take longer to read

- TinyStories (Eldan & Li 2023) is a set of short stories written by GPT-3.5 and GPT-4, using only words a three- or four-year-old would know. The authors built it to ask how small a language model can be and still write fluent, consistent English; it has since become a standard training set for small models, including those of Assignment 4 - Characters keep the examples small: 97 symbols, every one visible on a slide. Real models predict pieces of words; the next lecture shows that the same machinery works on them

- Recurrent networks dominated language modelling before transformers. Their single memory vector is a bottleneck: a name read long ago must survive many updates to be used. A later lecture covers them and what they still offer - A transformer has no such bottleneck: any earlier position can be read directly, in one step. When this lecture says a position reads "the past", it means the positions to its left in the window, all of which the model has at once - Training uses this: every position of the window predicts its own next token at the same time

- Text drawn at random from a character bigram table of English has English-looking letter pairs but few real words; longer contexts give more real words - Shannon (1948) built tables like this from English text, for pairs of letters and longer sequences, and showed that text generated from them looks more like English as the context grows. Counting tables of words (n-grams) were the standard language models of speech recognition and machine translation until neural networks replaced them in the 2010s

- Adding 1 to every count is called add-one (Laplace) smoothing. It costs little: each row sum grows by the number of symbols, here 5

- In the toy table, an underscore stands for a space. All three observed transitions from a lead to t (the a at the very end has no successor), so before smoothing P(t | a) = 3/3 = 1, and after smoothing (3 + 1)/(3 + 5) = 1/2; two of the six transitions from t lead to e, so P(e | t) goes from 2/6 to 3/11 - The loss is easier to read as e^loss, the effective number of choices: the number of equally likely next characters that would give the same loss. A uniform guess over 97 characters gives 97; a perfect model gives 1. The assignment's report prints it - Before scoring held-out text, add 1 to every count (add-one smoothing): a pair never seen in training would otherwise get probability 0, and an infinite loss. In the toy table, smoothing turns P(t | a) = 3/3 into (3 + 1)/(3 + 5) = 4/8, with five symbols - Shannon (1948) generated text from letter-pair ("digram") and longer letter statistics of English, and showed that the text looks more like English as the context grows - In the lecture on baselines, a score meant something only next to a baseline computed on the same data. The same rule applies here: compare on the same held-out text

- A sure and correct guess (p = 1) costs 0; a guess among 97 equally likely characters costs ln 97 = 4.6 - Averaging −ln p is the same as taking minus the log of the product of all the p's, divided by the number of positions: the cross-entropy measures how probable the model finds the whole held-out text, per character - This is the same loss as for class probabilities in the backpropagation lecture: here the classes are the 97 possible next characters - e to the power of the loss is easier to read: the effective number of choices. A uniform guess over 97 characters gives 97; a perfect model gives 1. Here e^1.34 = 3.8: on these two positions the table is as uncertain as a choice among about four characters - The two numbers come from the smoothed table of the last slide: P(e | t) = 3/11 and P(a | e) = 2/8 = 1/4 - Held-out text matters: on its own training text, a model can score well by memorizing. The lecture on baselines made the same point: compare models on the same held-out text

- With four characters of context there are about 89 million possible contexts, so a table of about 8.6 billion counts; a training text of 250 million characters cannot fill it - The table also treats "the cat" and "the dog" as unrelated contexts. With vectors, a model can give related symbols similar vectors, so what it learns about one context can carry over to similar ones

- An example with word tokens: cat and dog appear in the same places (the ___ sat, feed the ___), so training gives them similar vectors, and what the model learns about "the cat sat" carries over to "the dog sat". King and queen are another such pair - Why not keep the one-hot codes? They make every pair of characters equally far apart, so nothing learned about one character carries over to another. Learned vectors can put characters that behave alike, such as the vowels, close together. That matters next: attention scores positions with dot products, and between one-hot vectors of different tokens every dot product is 0 - The embedding table is a weight matrix of size (vocabulary size × d_model), learned by gradient descent like any other weights. Looking up row k gives exactly what multiplying the one-hot vector of token k by the table would give, without doing the multiplication - Size: the model of Assignment 4, Part 1 gives each of its 97 characters 128 numbers, more than the one-hot code, because the point is similarity, not saving space. For words the table also saves space: GPT-2 maps about 50,000 tokens to 768 numbers each - T is the length of the window of text the model reads at once, the number of tokens; d_model is the width of every vector inside the model. Each position's vector is a row of X, and subscripts i and j name positions; this matches the papers and the (batch, sequence, width) layout of tensors in PyTorch - Here every token is a character. Real language models use pieces of words as tokens; the next lecture covers how they are chosen. The mechanism is the same: one row per token

- e_i is the row of the embedding table for the token at position i, and p_i the row of the position table for position i; both tables are learned. The step-by-step demo later in the lecture writes the same thing as X = E + P - The vectors x_i now carry both what the token is and where it sits, and every later step can use either - Without position embeddings, the averaging and attention steps would give the same set of outputs for any shuffle of the input tokens; only the causal mask, later, adds a sense of order

- Each position averages over itself and the positions before it, never the ones after: when predicting, the future is not available. A model that uses only the past like this is called causal - The vectors are made up, two numbers each, so the arithmetic fits on a slide; a trained embedding table learns its own. Three words keep the example readable; characters work the same way - Here y_3 happens to equal sat's own vector, a coincidence of the made-up numbers - As a matrix product: multiply the stacked vectors by a fixed T × T averaging matrix whose row t holds 1/t in its first t entries and 0 after. The assignment builds the average this way

- The output layer is a linear readout, like the perceptron's, with the same weights at every position - Training adjusts the embedding table, the output layer and, later, the matrices of attention, all together, to lower the cross-entropy of the next character

- With all scores equal, every weight is 1/i and attention is the average of two slides back - This softmax is the same function as the output layer's, used differently: there it chooses among 97 characters; here it spreads position i's weight over the positions 1 to i - Scoring with the raw vectors x is the simplest version. The next slides give each position separate vectors for asking (the query), for being matched (the key) and for being read (the value), each made by a learned matrix; nobody gives the model target weights

- This is the attention step alone, without a transformer around it and without learning: the keys, values and query are written by hand, and each description is a short vector of features (red, blue, triangle, circle) - In a transformer, every token plays all three roles at once: it asks with a query, is matched through its key and passes on its value. The next slides build that from the token vectors

- The story: three objects were put away. The red triangle is on the desk, the red circle on the shelf, the blue triangle in the drawer. Each object's description is its key; where it is, its value - The question "where is the red triangle?" is the query. Attention compares the query with every key, turns the matches into weights, and returns the weighted mix of the values: the bars show how much of each location comes back - 1. Case 1, "A useful cue", opens first. Before pressing anything, guess which location comes back most. Press "Reveal computation": desk 90.9%, shelf 4.5%, drawer 4.5%. The red triangle's key matches both "red" and "triangle"; the others match only one of the two - 2. On the right, the slider "Scores multiplied by" sets how strongly the best match wins (6 at the start). Drag it to 0: every object gets 33.3%, the plain average. Drag it back up: desk takes over again. The multiplier is an inverse temperature: a large one makes the softmax pick the best match, 0 makes it the plain average. Dividing by √d_k, later today, is a fixed version of the same knob - 3. Case 2, "An incomplete cue": the question now says only "red". Reveal: desk and shelf tie at 48.8%, since both are red. Drag the slider to 12: still a tie (49.9% each). Tick "Include shape in the query": desk is back to 90.9% - 4. Case 3, "Many distractors": one red triangle among 24 red circles. Reveal: desk gets 45.6%, while the circles, each a weak match, add up to 54.4%. Move "Similar distractors" down to 5 (desk 80.1%), or the slider up to 12 (desk 94.4%). The reason is the softmax's normalization: all weights share a total of 1, so 24 weak matches together can outweigh one strong match. Visual search shows the same set-size effect: a target that shares features with many distractors becomes harder to find as the display grows - 5. Case 4, "No correct source": the red triangle's entry is missing. Reveal: shelf gets 95.3%, a confident answer that is wrong. Tick "Restore the matching source": desk 95.0% - The point: attention can only read what is in its context, and a high weight on one source does not mean the answer is right. "Show the math" lists every score and weight - The objects and locations are made up for the demo; they are not words, a trained model, or data from human memory

- Position i is the i-th token of the window. The same three matrices are used at every position - Separate vectors let a token be found for one reason and pass on something else: a name can be matched on "a name was here" and pass on its letters - d_k is the number of entries in each query and key; dividing by √d_k keeps the scores in a range where the softmax works well (a later slide explains why) - Think of a library catalog: the query is your search, the keys are the index cards, the values are the books. Unlike a catalog, which hands you one book, attention returns a mix of all the values, weighted by the matches

- Each projection is a linear layer without the nonlinearity, applied to every row of X with the same weights. W_Q and W_K have size d_model × d_k, W_V has size d_model × d_v - W_Q, W_K and W_V are fixed once training ends; the queries, keys and values are recomputed for every new input - d_k is the width of queries and keys (they must match, because we take their dot product); d_v is the width of values

- Row i of A holds position i's weights a_ij: nonnegative, summing to 1. Row i of A V is y_i, the new vector of position i - Why √d_k: when the entries of a query and a key are roughly independent, with mean 0 and variance 1, their dot product has variance d_k, a spread of √d_k. Without the division, large scores push the softmax to put almost all weight on one position, and its gradient becomes close to zero - The softmax is the one from the lecture on learning the weights, where it turned a network's scores into class probabilities. Here it turns one row of scores into weights that sum to 1

- In a trained model the query, keys and values come from the learned matrices W_Q, W_K and W_V; here they are made up so the arithmetic fits on a slide - Scores: (1,0)·(0,1) = 0; (1,0)·(2,0) = 2; (1,0)·(1,1) = 1. Divided by √2: 0, 1.414, 0.707 - Softmax: e^0 = 1, e^1.414 = 4.11, e^0.707 = 2.03; their sum is 7.14, so the weights are 0.14, 0.58 and 0.28, which add up to 1 - The key of cat matches the query of sat best, so cat gets the largest weight, and y_3 is pulled toward cat's value. Every other position does the same with its own query, at the same time

- The same three words and numbers as "Attention by hand". The demo updates sat, the last word, unless you click another word at the top. Press Next to go one stage at a time - 1. Input: each word's vector is its word embedding plus its position embedding; for sat, x3 = [1, 1, 0] - 2. Q, K, V: three matrices, W_Q, W_K and W_V, turn each vector into a query, a key and a value. The same three matrices are used for every word - 3. Match: sat's query is compared with every key by a dot product, giving the scores 0, 2 and 1 - 4. Scale and mask: each score is divided by √2, because queries and keys here have 2 numbers; dividing by the square root of their length keeps the scores from growing with the vector length. The mask hides words that come after the one being updated - 5. Softmax: the scores become the weights 0.14, 0.58 and 0.28, which add up to 1 - 6. Gather: the new vector for sat is the weighted sum of the values, (0.28, 0.72) - 7. All positions: every word does the same at the same time - To see the causal mask at work, first click "the" at the top: the is the first word, so with the mask it can read only itself (weight 1). Untick "Causal mask": now the reads cat and sat too. For sat the mask changes nothing, because no word comes after it - Other things to try, one at a time: untick "Divide by √2" (the weights become more uneven); tick "Equal weights" (every allowed word gets the same weight, the plain average); tick "Zero cat's value" (cat still gets the largest weight, but brings nothing back)

- The same three words as "Attention by hand". The magenta arrow is the query of the word chosen under "asking" (sat at the start); the blue arrows are the keys of the, cat and sat. A thicker key arrow means more weight; a gray key is a later word, hidden by the causal mask - 1. The demo opens on the numbers of "Attention by hand": weights 0.14, 0.58 and 0.28, output (0.28, 0.72). The lower plot shows the three values (green) and the output, their weighted average (magenta) - 2. Drag the query toward cat's key: cat's weight grows, and the output moves toward cat's value - 3. Move the query up and to the left, to about (−2, 2): the scores become 2, −4 and 0, so the gets the largest weight - 4. Keep the direction and double the length: the scores double and the weights become more uneven. Unticking "divide by √d_k" does the same - 5. Click cat under "asking": the query becomes cat's own, (1, 1), and sat's key turns gray, because sat comes after cat (its gray arrow sits under cat's query, which happens to be the same vector). Weights 0.33 and 0.67 on the and cat. Untick "causal mask": sat can now be read too

- Read one row at a time. A row is one query word; the colors across it are its weights over the six key words, and they add up to 1 - The scores here were set by hand to make the pattern easy to see. In a trained model they come from W_Q and W_K, which are learned - The right panel is the causal mask, the next slide

- The mask matters because training is parallel: the whole sequence goes in at once, and position t is asked to predict the symbol at t + 1. The mask keeps each position's prediction to the symbols before it, as at generation time - Setting a score to minus infinity gives e^(−∞) = 0 in the softmax, and each row renormalizes over the positions to its left. The first row can only read itself - The diagonal is allowed: every position reads itself and the positions before it

- These are the numbers that Part 1 of the assignment prints for each model, on the same held-out stories. The effective number of choices is e to the power of the loss: the attention head is as uncertain as a choice among about 3.4 equally likely characters, against 97 for a uniform guess - The bigram table is the baseline from the start of the lecture: a model that uses more context is worth something only if it beats it on the same held-out text

- Next lecture: what trained heads do in real models such as GPT-2, then stack attention into a full model, train it to write stories, and look inside it to see what its heads do

- Adeli et al. (2026) and Mehrani & Tsotsos (2023) make a pair: the first shows that DINO-style transformers predict human reaction times in a same-object task from their patch affinities; the second argues that what these models compute is grouping by similarity, not attention in the psychologist's sense. Decide for yourself whether the word is earned - Christian et al. (2026) compare six vision–language models' attention maps with human fixations on the same images; the match depends mostly on the kind of decoder - Barbero et al. (2025) explain why so many heads park their weight on the first token, which the next lecture shows in GPT-2 and in vision transformers: parking it there keeps a deep model from over-mixing its positions - Goldstein et al. (2025) and Cai et al. (2026) are where the model–brain program stands for language: a speech model's layers aligned with the cortical hierarchy during real conversation, and single frontal neurons sorted by what a language model's representations say about each word - Two more that qualify: Lindsay (Front Comput Neurosci 2020, doi:10.3389/fncom.2020.00029), a review of attention across psychology, neuroscience and machine learning; Bricken & Pehlevan (NeurIPS 2021, arXiv:2111.05498), attention as a cerebellar associative memory, a bridge to the memory models of a later lecture - Before posting, skim the existing entries on your paper; your post must add something not already said

- Submitted on Canvas (Minute papers → Minute Paper 16); credit for a thoughtful attempt