query q =
output = (weighted average of the values)
Thicker key arrow: more weight. Gray key: a later word, hidden by the causal mask. "asking" picks whose query is used.
Drag the magenta arrow. Try:
- Make sat read mostly the.
- Keep the direction, double the length. What happens to the weights?
- Ask from cat with the mask on, then off.