Brown University crest
CPSY 1291 Computational Methods for Mind, Brain & Behavior
Lecture 10 · Theme 2: Learning representations: from neurons to deep networks

Training at scale, generalization & characterizing units

Thursday, October 15 · Fall 2026

Reader-friendly version — lecture content with figure descriptions

Last lecture: hidden layers and backpropagation

A fully connected network drawn left to right: inputs x1 to xD, hidden layer 1, hidden layer 2 and an output layer y-hat. Every unit connects to every unit of the next layer; the weight matrices W superscript 1, 2 and 3 label the gaps
  • Hidden layers re-represent the inputs, so a readout can separate classes that no single line separates, as in XOR
  • Backpropagation gives the gradient of the loss for every weight: error at the output, sent backward, error × activity
  • The training loop: forward pass, loss, backward(), step(), repeated over epochs
  • Each step of gradient descent used all the training images and a fixed learning rate η\eta

Today: three questions about a trained network

  • How do we train it on millions of images? Small batches of examples, a learning rate that shrinks, and Adam
  • Does what it learned hold for images it never saw? Generalization: how to measure it honestly, and how to improve it
  • What have its units learned? Measure them the way physiologists measure V1 and IT neurons

Today's three examples

Contour plot of the loss of a linear neuron with two weights, w1 and w2, with a gradient-descent path from a start point to the lowest loss

The two-weight neuron
24 simulated examples

Two panels: a degree-3 polynomial that follows the sine curve through 10 noisy points, and a degree-9 polynomial that passes through every point and swings far off between them; under each, its training and test RMSE

The curve fit
10 noisy points of a sine curve

A grid of grayscale renderings, one row per category (animal, vehicle, face, vegetable or fruit, house, man-made object), each with two objects seen from three of their 24 viewpoints

The IT readout
100 IT sites, six object categories

  • A neuron, a curve, a readout: the same ideas about training and generalization hold for each, and for a deep network such as AlexNet (end of the lecture)

The IT readout, in full

A grid of grayscale renderings, one row per category (animal, vehicle, face, vegetable or fruit, house, man-made object), each with two objects seen from three of their 24 viewpoints
  • Recordings: neurons in monkey inferotemporal cortex (IT), the object-recognition area (Bao et al. 2020), the data of the first two assignments
  • Images: 51 objects × 24 views = 1,224, in six categories (rows of the grid)
  • Input: each image's average response at 100 IT sites: 100 numbers. Output: its category, one of 6 (chance: 1 in 6)
  • Every readout today learns this same mapping, from a logistic regression to a small network
Bao, She, McGill & Tsao Nature 2020

How do we train on a million images?

Last lecture, every step used all the training images. That is slow, and they do not fit in memory at once

One example, all of them, or a few at a time?

  • One step updates every weight once, using the loss averaged over the examples it is computed on
  • Online, one example per step, like the perceptron and the delta rule: cheap steps, but each gradient is very noisy
  • Full batch, all NN examples per step, like last lecture's loop: an exact gradient, but one step per pass, and a million images do not fit in memory
  • Minibatch, BB examples per step (often 32 to 256): each epoch, shuffle the examples and split them into groups of BB, one step per group. Many cheap steps per pass, so far fewer passes
Twelve numbered training examples in order, then the same twelve shuffled, then cut into three batches of four; each batch is one step, and the three steps make one epoch; the next epoch shuffles again
Robbins & Monro Ann Math Stat 1951 · PyTorch DataLoader · Recitation 5

Three tips for training your own network

Shrink the learning rate as training goes · Average the gradients: momentum and Adam · Sweep the learning rate first

Tip 1: shrink the learning rate as training goes

Two contour plots of last lecture's two-weight loss, with identical axes, each showing 600 minibatch updates from the same start using the same batches of two examples. Left, constant step size 0.8: the last 300 steps scatter widely around the star that marks the lowest loss, mean distance 0.12. Right, a step that shrinks as 0.8 divided by 1 plus t over 20: the last 300 steps sit in a tight cluster on the star, mean distance 0.01. Faint trails enter from the start, which lies off the plotted area

  • The two-weight neuron, now with minibatches of 2. A constant rate keeps bouncing; a shrinking one settles. Full batches would not need it
PyTorch learning-rate schedulers

Tip 2: average the gradients (momentum)

Contours of the squared-error loss of the two-weight neuron, weight w1 horizontal and w2 vertical on equal scales, with its second input on a scale five times larger, so the loss is a long narrow valley running along w1. From the same start, 40 steps each at the same learning rate: plain gradient descent zigzags up and down across the valley; momentum swings in a few slow, smooth waves along it toward the lowest loss. Up-and-down travel along w2: 4.4 for plain descent, 0.8 for momentum

  • The two-weight neuron, x2x_2 on a 5× larger scale: a narrow valley. A rate that does not overshoot across it barely moves along it: plain descent zigzags
  • Momentum averages recent gradients (m\mathbf{m}): the up-and-down pushes cancel

Tip 2, continued: Adam, a step size for each weight

The same valley of the two-weight neuron and the same start, w1 horizontal and w2 vertical. Plain gradient descent zigzags faintly in the background; momentum swings in slow waves as on the previous slide; Adam follows one smoother curve down the valley to the lowest loss. Up-and-down travel along w2: plain 4.4, momentum 0.8, Adam 0.6

  • Each weight's step is divided by its typical gradient size: both weights move at a similar pace
  • Steps to 1% of the starting loss, each at its best learning rate: plain more than 40, momentum 8, Adam 6
  • Adam tolerates a wider range of learning rates, but η\eta is still a setting to choose (Tip 3)

Adam: the update

m←β1m+(1−β1) ∇L,v←β2v+(1−β2) (∇L)2,θ←θ−η mv+ϵ\mathbf{m}\leftarrow\beta_1\mathbf{m}+(1-\beta_1)\,\nabla\mathcal{L},\qquad \mathbf{v}\leftarrow\beta_2\mathbf{v}+(1-\beta_2)\,(\nabla\mathcal{L})^2,\qquad \boldsymbol{\theta}\leftarrow\boldsymbol{\theta}-\eta\,\frac{\mathbf{m}}{\sqrt{\mathbf{v}}+\epsilon}

θ: all weights and biases · operations are per weight · Adam's first-step correction omitted

  • m\mathbf{m}, the running average of the gradient (its first moment): which way each weight has been pushed lately, like a rolling ball's velocity
  • v\mathbf{v}, the running average of the squared gradient (its second moment): how large each weight's gradients usually are, whatever their sign
  • Each weight in effect gets its own learning rate, η/v\eta/\sqrt{v}, set by its gradient history. If a weight's recent gradients all point the same way, its step is about η\eta long; if they keep flipping sign, its step is short
  • Defaults β1=0.9\beta_1=0.9, β2=0.999\beta_2=0.999 work on most problems; ϵ\epsilon prevents division by zero
Kingma & Ba ICLR 2015 · Loshchilov & Hutter ICLR 2019 · Wilson et al. NeurIPS 2017 · PyTorch · Recitation 5

Tip 3: sweep the learning rate first

Training loss on a log scale against epoch for four learning rates of one network reading the six object categories from 100 monkey IT sites. At 10 to the minus 5, dotted grey, the loss barely falls (test accuracy 0.18); at 10 to the minus 3, dashed blue, it falls slowly (test 0.62); at 0.1, solid magenta, it falls to about 0.015 (test 0.91); at 3, dash-dotted black, it jumps off the top of the axis in the first epoch and diverges

  • The IT readout with a small network: 100 IT responses → 64 hidden units (ReLU) → 6 outputs. Training loss: cross-entropy
  • Plain gradient steps; four runs differing only in η\eta
  • Too small: the loss barely moves. Too large: it explodes
  • η\eta is a hyperparameter, set by us. Good values span orders of magnitude: sweep on a log scale

Does what a network learned hold for images it never saw?

Generalization: when does a trained network's fit carry over to new data, and how do we make it?

Recall: train on some images, test on others

Two panels for a readout of animate versus inanimate from 480 monkey IT neurons, trained on 102 images and tested on 306. Left: accuracy bars; with true labels, training 1.00 and test 0.96; with shuffled training labels, training 1.00 and test 0.56 on average, below a dashed majority-class line at 0.63. Right: histogram of the test accuracy over 1,000 label shuffles, centred near 0.55, with its 95th percentile marked near 0.67 and the true-label test accuracy, 0.96, far to the right

  • Recall the baselines lecture: a probe (readout) of animate vs inanimate from 480 IT neurons. Fit it on 102 images, test it on 306 others
  • With the true labels: 1.00 on the training images, 0.96 on the test images. Then shuffle the training labels and fit the probe again
Bao, She, McGill & Tsao Nature 2020

A perfect training score can hide poor generalization

Two panels for a readout of animate versus inanimate from 480 monkey IT neurons, trained on 102 images and tested on 306. Left: accuracy bars; with true labels, training 1.00 and test 0.96; with shuffled training labels, training 1.00 and test 0.56 on average, below a dashed majority-class line at 0.63. Right: histogram of the test accuracy over 1,000 label shuffles, centred near 0.55, with its 95th percentile marked near 0.67 and the true-label test accuracy, 0.96, far to the right

  • Shuffled labels were fit perfectly too (1.00), yet scored only 0.56 on new images
  • Zhang et al. (2017), ImageNet: random labels fit for 95% of 1.2 million photos; new photos 0.1%, chance
  • Generalization gap: training accuracy minus test accuracy
  • When does a fit carry over to new data? How do we measure and improve it? (minute paper)
Bao, She, McGill & Tsao Nature 2020 · Zhang, Bengio, Hardt, Recht & Vinyals ICLR 2017

Training fits a function from inputs to outputs

A model neuron with two inputs, x1 and x2, with weights w1 and w2, summed with a bias b into z, passed through the nonlinearity g, giving the output a = g(w1 x1 + w2 x2 + b)
  • A neuron, a readout or a network computes an output from an input: y^=f(x;θ)\hat y = f(\mathbf{x};\textcolor{#B5396B}{\boldsymbol\theta})
  • Training chooses the parameters θ\textcolor{#B5396B}{\boldsymbol\theta} so that y^\hat y matches yy on the training pairs (x,y)(\mathbf{x}, y)
  • Generalization asks whether ff is also right for inputs it never saw
  • With hundreds of inputs we cannot draw ff; with one input we can

Too rigid, or too flexible

Computed: 10 noisy samples of a sine curve, after Bishop (2006), Fig. 1.4

Four panels, each with the same 10 magenta points scattered around a dashed sine-shaped curve, and a blue fitted curve. Degree 0: a flat line, training error 0.63, error on new points 0.76. Degree 1: a falling straight line, 0.48 and 0.56. Degree 3: a curve close to the dashed one, 0.19 and 0.33. Degree 9: a curve through every point that swings far off between and beyond them, training error 0.00, error on new points 0.86. Open green circles: 25 of the new points the error on new points is measured on

  • The curve fit: polynomials of degree dd (d−1d-1 bends), 10 training points (magenta), new points (green)
  • Underfitting: high bias, the error of the average prediction. Overfitting: high variance, the fit changes with the training set
  • RMSE=1N∑i(y^i−yi)2\text{RMSE}=\sqrt{\tfrac{1}{N}\sum_i (\hat y_i - y_i)^2}, in units of yy. Training RMSE keeps falling; test RMSE falls, then rises
Bishop Pattern Recognition and Machine Learning, Springer 2006

Same model, more examples

Computed: the same curve and noise, after Bishop (2006), Fig. 1.6

Four panels of the same degree-9 polynomial fitted to more and more noisy points from the dashed sine-shaped curve. N equals 10: the curve swings far off, error on new points 0.86. N equals 15: close to the dashed curve, 0.34. N equals 30: 0.35. N equals 100: the fitted curve follows the dashed one, training error 0.28, error on new points 0.31. Open green circles: 25 of the new points the error on new points is measured on

  • The same degree-9 polynomial, fitted to 10, 15, 30 and 100 points from the same curve
  • With 10 points it passes through every one; with 100 it cannot, and follows the curve instead
  • Overfitting depends on flexibility relative to the number of examples
Bishop Pattern Recognition and Machine Learning, Springer 2006

How do we choose a model and measure how well it generalizes?

Every choice made while looking at a score can fit that score's images

Three data splits

A horizontal bar split into three parts. Training, the largest part, in magenta: fit the weights. Validation, in green: choose settings and when to stop. Test, in dark grey: report once, at the end

  • Training set: fit the parameters θ\textcolor{#B5396B}{\boldsymbol\theta}, every number set from data (weights, biases, a polynomial's coefficients)
  • Validation set: choose the hyperparameters on it (settings we pick, such as η\eta). Few examples? Cross-validation (next slide)
  • Test set: used once, at the end. Never choose anything on the test set, or its score is too optimistic
  • Often broken: ImageNet results are tuned on the set they report; on a fresh test set, 11–14 points lower
Recht, Roelofs, Schmidt & Shankar ICML 2019 · Recitation 6: held-out data

Cross-validation: each example validates once

Five rows, fold 1 to fold 5, each showing the training data cut into five equal parts: one part in green, a different one in each row, the other four in magenta; each row ends in its own score. A separate dark block on the right is the test set, set aside and used once. Caption: train on the magenta parts, score on the green one; the cross-validation score is the mean of the five scores

  • Few examples: a separate validation set wastes data. Cut the training data into kk parts; train on k−1k-1, score on the one left out, kk times. Leave-one-out: one part per example
  • Choose the setting with the best mean score, then refit on all the training data. The test set stays out of all of it
scikit-learn: cross-validation

Learning curves: when to stop training

Two panels against epoch, 0 to 400, for a network reading out the six object categories from 100 monkey IT sites, trained on 300 images with a fifth of their labels redrawn at random. Left, loss: training loss in magenta falls to near 0; validation loss in green is lowest near epoch 20, about 0.56, then rises to about 0.97. Right, accuracy: training accuracy rises to 1.0; validation accuracy is highest, 0.82, at epoch 8, marked by a dashed line labelled checkpoint kept, and drifts down to about 0.78. A dotted line marks chance, one in six

  • The IT readout, 300 training images, a fifth of their labels redrawn at random: validation turns as the network fits the wrong labels
  • Early stopping: stop when the validation score stops improving. Keep the checkpoint (saved parameters) with the best validation score: accuracy here, epoch 8; by loss it would be 20

Why stop before the bottom of the training loss?

Left: the curve fit's 10 training points and the dashed sine curve, with a degree-9 polynomial trained by gradient descent shown at three moments: after 10 steps, a smooth curve that is still too flat; after 1,467 steps, a curve close to the sine; after 10 to the 10 steps, the curve that passes through every point and swings far off between them. Right: training RMSE and test RMSE against gradient steps on a log axis; training RMSE falls to zero, test RMSE is lowest, 0.33, at step 1,467, marked stop, and rises to 0.86

  • The curve fit, degree 9, trained step by step: early steps fit the broad shape, late steps the noise
  • Stopped at its best step, degree 9 does as well as degree 3 (test RMSE 0.33 for both)

A test accuracy from few images is imprecise

Test accuracy of the running readout, six categories from 100 monkey IT sites, against the number of test images, 25 to 800, on a log axis. Each point has a vertical bootstrap interval. With 25 test images the interval runs from 0.76 to 1.00; with 800, from 0.83 to 0.88. A dashed line marks the accuracy on all 924 other images, 0.86

  • The IT readout as a logistic regression, trained on 300 images; the dashed line is its accuracy on all 924 other images (0.86)
  • The bootstrap (baselines lecture): resample the test images with replacement; the middle 95% of the accuracies is a 95% confidence interval
  • With 25 test images, the accuracy could be anywhere from 0.76 to 1.00; with 800, from 0.83 to 0.88
  • Compare models on enough test images that their intervals do not overlap

Shortcut learning: texture over shape

Reproduced from Geirhos et al. 2019, Fig. 1: three images with ResNet-50's top three answers. (a) A texture image of elephant skin: Indian elephant 81.4 percent, indri 10.3, black swan 8.2. (b) A photograph of a tabby cat: tabby cat 71.1 percent, grey fox 17.3, Siamese cat 3.3. (c) The cat's shape with elephant-skin texture, a texture-shape cue conflict: Indian elephant 63.9 percent, indri 26.4, black swan 9.6

  • Even a precise test accuracy can mislead. Out of distribution (OOD): test images from a different source than the training images
  • Shortcut learning: a cue that predicts the label in training but not in the world (texture, background)
  • Cue-conflict images like (c): people chose shape in 95.9%, ResNet-50 (an ImageNet network) in 22.1%
Geirhos et al. ICLR 2019 · Geirhos et al. Nat Mach Intell 2020

How do we make a network generalize better?

A penalty on the weights, more data, and switching units off

A penalty on the weights

L(θ)  =  Ltrain(θ)⏟fit  +    λ∥θ∥2⏟strength    ∥θ∥2⏟size of the parameters\mathcal{L}(\textcolor{#B5396B}{\boldsymbol\theta}) \;=\; \underbrace{\mathcal{L}_{\text{train}}(\textcolor{#B5396B}{\boldsymbol\theta})}_{\text{fit}} \;+\;\; \underbrace{\lambda\vphantom{\lVert\textcolor{#B5396B}{\boldsymbol\theta}\rVert^2}}_{\text{strength}}\;\;\underbrace{\lVert\textcolor{#B5396B}{\boldsymbol\theta}\rVert^2}_{\text{size of the parameters}}

Computed: the curve fit's 10 points, after Bishop (2006), Fig. 1.7

Four panels of the degree-9 polynomial fitted to the same 10 points, with an increasing penalty. No penalty: through every point, swinging far off, error on new points 0.86. Lambda 10 to the minus 7: a smooth curve close to the dashed one, 0.34. Lambda 10 to the minus 3: smooth, 0.34. Lambda 1: an almost flat line, training error 0.49, error on new points 0.61. Open green circles: 25 of the new points the error on new points is measured on

  • The degree-9 polynomial, its coefficients penalized (weight decay): test RMSE falls from 0.86 to 0.34. Too large a λ\lambda underfits
Bishop Pattern Recognition and Machine Learning, Springer 2006

Without a penalty, a readout's weights keep growing

Two panels for the running readout, six categories from 100 IT sites, as a logistic regression trained on 300 images. Left: the size of the weights against gradient steps on a log axis; without a penalty it keeps rising, to about 10.7; with weight decay it levels off near 3.5. Right: the training cross-entropy on a log axis; without a penalty it falls toward zero, to 0.0015; with weight decay it levels off near 0.11

  • The IT readout (logistic): all 300 training images right; a longer w\textcolor{#B5396B}{\mathbf{w}} only adds confidence
  • A penalty stops it: same accuracy, less confidence. (If the training classes overlapped, w\textcolor{#B5396B}{\mathbf{w}} would stop)

More data, or altered copies of it

Four versions of one photograph of a pink orchid: the original; flipped left to right; cropped and enlarged; and with its colour faded and raised

  • More training examples always help generalization: the same model overfits less (the polynomial with 10 vs 100 points)
  • No more data? Make some. Data augmentation: train on altered copies that keep the label (flipped, cropped, recoloured). AlexNet trained on 2,048 versions of each photo
  • An augmentation states what should not change the label: a flipped orchid is still an orchid
Krizhevsky, Sutskever & Hinton NeurIPS 2012

Regularization: penalties and dropout

Three copies of the same small network: in the first two, a different random half of the hidden units is switched off for one training step each, with their connections removed; in the third, at test time, all units are on

  • With all the data used, regularization: limit how freely the network fits the training data, like the L2 weight penalty
  • Dropout: at each training step, switch off a random half of the hidden units; all are on at test time
  • In practice a mix: L2, dropout, sometimes L1. Each cuts the network's freedom to fit noise
Srivastava et al. JMLR 2014

Try it: overfit, then regularize

  • Last lecture's network, 30 training points, 15% of labels flipped. Train: held-out loss falls, then rises. Then add λ, dropout, or more points

A model of a neuron: regression with a penalty

Two panels for one monkey IT recording site and 512 features of the trained network's late layer. Left: predicted against recorded responses for a ridge fit; training images faint grey, held-out images red, both scattered along the diagonal, R-squared 0.71 on training and 0.69 on held-out images. Right: R-squared against the penalty lambda on a log axis; with almost no penalty, training R-squared is 0.85 and held-out R-squared 0.28; as the penalty grows, training R-squared falls while held-out R-squared rises to meet it near 0.69 at the penalty chosen on the training images, then both fall

Computed: one monkey IT site (Bao et al. 2020), 1,224 object images, 512 late-layer units of ResNet-18; a random fifth held out

  • Predict one IT site from 512 network units: ridge regression, a weighted sum plus the weight penalty. λ\lambda by cross-validation (RidgeCV); R2R^2 on held-out images

Does a trained network see like the visual cortex?

Nobody told AlexNet about V1. Measure its units the way physiologists measure neurons, and compare

What a trained network's first layer learns

First-layer weights of AlexNet shown as small color images: oriented edges, stripe patterns and color blobs
  • AlexNet, trained to label 1.2 million photos, was never shown a receptive field
  • Its first-layer filters, each drawn as a small image: oriented edges, stripes, colour blobs
  • Do these look like the receptive fields of V1 cells? And how would we measure "look like"?
Krizhevsky, Sutskever & Hinton NeurIPS 2012

Real V1 receptive fields look like Gabors

Reproduced from Ringach 2002, Fig. 2: six macaque V1 simple cells in three blocks. Each row shows the measured receptive field (Data), the best-fitting two-dimensional Gabor (Fit) and what is left over (Residual). The fields range from round blobs to elongated stripes of alternating sign; each fit reproduces its field's shape, and the residuals are faint, unstructured speckle. A colour bar runs from minus 1, blue, through 0, green, to plus 1, red

  • First the brain. A cell's receptive field: the input pattern that drives it. Data: six macaque V1 simple cells
  • Daugman (1985): each is a Gabor, a grating seen through a smooth window. Fit: the best Gabor; Residual: what is left. The residual is mostly noise
Ringach J Neurophysiol 2002 · Daugman J Opt Soc Am A 1985 · Jones & Palmer J Neurophysiol 1987

The Gabor function, fitted, and two numbers from it

G(x,y)=A exp⁡ ⁣(−x′22σx2−y′22σy2)cos⁡ ⁣(2πfx′+ϕ)G(x,y)=A\,\exp\!\Big(-\frac{x'^2}{2\sigma_x^2}-\frac{y'^2}{2\sigma_y^2}\Big)\cos\!\big(2\pi f x'+\phi\big)

Left, a toy receptive field with pixel noise, the Gabor fitted to it, and the residual, which is only noise; R-squared 0.87. Right, three noise-free toy Gabors: A with nx 0.20 and ny 0.20, a small spot; B with nx 0.35 and ny 0.50, a few short stripes; C with nx 0.90 and ny 1.20, many long stripes

  • Fit A,σx,σy,f,ϕA,\sigma_x,\sigma_y,f,\phi, orientation and centre by least squares; R2R^2 scores the fit
  • Keep nx=σxfn_x=\sigma_x f, ny=σyfn_y=\sigma_y f: the window's width and length in cycles

Where do V1 simple cells sit?

Scatter of Gabor width nx against length ny, both in cycles, for 93 macaque V1 simple cells: most cluster near the origin, below about 0.6 on both axes, with a few longer fields. Three blue stars mark the toy units A, B and C of the Gabor-fit slide: A and B sit among the cells, C lies beyond most of them
  • Ringach (2002) fit Gabors to 93 simple cells of macaque V1 (cells that respond to a grating only at some phases) and released the fits
  • Most cells sit below 0.6 on both axes (medians 0.32 and 0.41): a window of one or two stripes
  • Toy unit C, with many long stripes, lies beyond most real cells
  • The same two numbers can place any receptive field on this plot, a network unit's included
Ringach J Neurophysiol 2002

Quantified: V1 cells and AlexNet's first layer

The same scatter and axes as the previous slide, with the toy units removed: macaque V1 simple cells as grey squares near the origin, and AlexNet's first-layer filters that a Gabor fits well as open green circles; the filters overlap the cells but sit systematically larger
  • Filled squares: macaque V1 simple cells. Open circles: AlexNet's first-layer filters, each fit by a Gabor
  • The same two numbers place a cell and a network unit on one plot
  • Do the network's filters look like V1 cells, and where do they differ?
Ringach J Neurophysiol 2002 · Krizhevsky, Sutskever & Hinton NeurIPS 2012

What does a unit respond to? Its tuning curve

Top: the weights of four hand-built units, three Gabors at different orientations and window lengths, labelled with orientation selectivity indices 1.00, 1.00 and 0.90, and a centre-surround unit with index 0.00. Bottom: their tuning curves to gratings at 24 orientations, each scaled to its peak; the three Gabors peak at different orientations, the long one narrowly, the short one broadly, and the centre-surround unit is flat

  • Hubel and Wiesel's method: show gratings (stripes) at many orientations and record the response. Here, four units built by hand, so we can check the method
  • Top: each unit's weights, its receptive field: three Gabors and a round centre-surround unit
  • Orientation selectivity index (OSI), (max − min) / (max + min) of the responses: near 1 when the weakest response is near zero. Bandwidth: the half-width of the tuning curve at half its height
  • A and C both have indices near 1, yet C is far more broadly tuned: the index does not measure tuning width

Why rectify before averaging over phase?

Bar chart for one Gabor unit and a grating at its preferred orientation shown at four phases: the weighted sums are plus one, plus one, minus one, minus one and average to zero; after rectifying, the responses are one, one, zero, zero and average to one half. Thumbnails of the four gratings and of the unit's weights sit under the bars

  • To measure a unit's orientation tuning alone, average its response over where the stripes sit (phase). A raw weighted sum flips sign after half a cycle, and the average cancels
  • A neuron's firing rate is never negative: model it as rectified, like a ReLU. A rectified unit keeps its preference, as a real cell does. The Gabor fit itself uses the weights and needs no rectification

Deeper in: how selective is one IT site for objects?

Responses of two IT sites to 51 objects, 24 views each: dots for single views, bars for object means. Site A, object selectivity index 0.60, has object means that differ widely, highest for a swan, shown as a thumbnail; site B, index 0.22, has object means that barely differ compared with the spread of the views

  • Past the first layer, weights no longer look like images: measure selectivity from responses alone. Show each of 51 objects from 24 views (the images of the first assignments); split one IT site's variance into between objects and within an object
  • Object selectivity index = between / total: near 1 when the response depends on which object, near 0 when the object means barely differ compared with the spread of the views
  • Site A: the bars (one per object, its mean over 24 views) differ much more than the views of one object do: 0.60. Site B: the bars are nearly level and the views vary as much: 0.22
  • This ranks units by how they respond. Whether a unit is needed is another measurement: next slide
Bao, She, McGill & Tsao Nature 2020

Is a selective unit needed? Remove it and see

  • Selectivity: how a unit responds. Necessity: what a readout loses without it. Different measurements
  • Ablation: set a unit's activity to zero and measure the drop in a readout's accuracy
  • Control: remove the same number of randomly chosen units. Selective units matter more only if removing them costs more than removing random ones
  • Keep the readout fixed, and you ask whether it relied on those units. Refit it, and you ask whether the remaining units can still separate the classes

Recap

  • Training at scale: minibatches give cheap, noisy gradients; a schedule shrinks the step; Adam adapts it per weight. Sweep the learning rate first, on a log scale
  • Generalization: performance on data the model never saw. Overfitting depends on flexibility relative to the number of examples
  • Splits: train to fit, validate to choose and to stop, test once, with a bootstrap interval
  • Beyond the test set: shortcuts fail on new kinds of images. Large networks can memorize random labels and still generalize on real labels
  • Regularization: weight decay, data augmentation, dropout and early stopping can reduce overfitting. Choose their settings on validation data
  • Characterizing a unit: tuning curves (rectified before averaging over phase), Gabor fits placed beside V1 cells, object selectivity from responses. Ablation with a random control asks whether a unit is needed

Further reading for the research trail

Optional: a few research-trail entries over the semester. Any paper mentioned in any lecture qualifies.

Minute paper

Submit on Canvas → Minute papers → Minute Paper 11 (access code read out in class).

Write three brief points in your own words:

  1. In plain words: how can a readout score 100% on its training images and still do poorly on new ones?
  2. Something you do not yet understand, or a question still open
  3. Another idea you found interesting, and why it matters for brains, behavior or AI

Credit for a thoughtful attempt, not for being correct.

Appendix

Slides cut from the lecture for time: schedules, bias and variance, many dimensions, five-photo results, the PyTorch checkpoint loop

Four ways to shrink the learning rate

Two stacked panels. Top: the learning rate against step for four schedules: constant, step decay that halves it at fixed intervals, 1 over 1 plus t over tau, and cosine decay. Bottom: the training loss for each; the shrinking schedules settle lower and more smoothly than the constant one
  • One readout, trained four times with the same minibatches; only the schedule of η\eta differs. Top: the step size at each step. Bottom: the training loss
  • Constant never changes. Step decay halves η\eta every 200 steps. Cosine lowers it smoothly to near 0. 1/(1+t/τ)1/(1+t/\tau) shrinks it a little at every step
  • Inset, last 200 steps: every shrinking schedule ends steadier than the constant step
  • A common recipe for image networks: start at 0.1 and divide by 10 every 30 epochs, or follow a cosine

Same model, different training sets

Computed: the polynomial example; 20 fits shown, numbers from 2,000 training sets

Three panels, each with 20 thin blue curves fitted to 20 training sets of 10 noisy points, their average in red, and the true curve dashed. Degree 1, too simple: the lines agree with each other but their average misses the curve; the gap between the average and the true curve is shaded and labelled bias. Degree 3, about right: the fits cluster tightly around the true curve. Degree 9, too flexible: the average lies on the true curve but the individual fits swing wildly

  • Draw many training sets from one source (same 10 inputs, new noise each time) and fit each. Thin blue lines: one fit per training set; red: their average; dashed: the true curve
  • Degree 1: the fits agree, but their average misses the curve: bias, the error of the average prediction, the shaded gap f(x)−E[y^]f(x)-\mathbb{E}[\hat y]. Degree 9: right on average, but the fitted curve itself changes from one training set to the next: variance

Bias and variance

E[(y−y^)2]=(f(x)−E[y^])2⏟bias2+E[(y^−E[y^])2]⏟variance+σnoise2,y=f(x)+noise\mathbb{E}\big[(y-\hat y)^2\big] = \underbrace{\textcolor{#B5396B}{\big(f(x)-\mathbb{E}[\hat y]\big)^2}}_{\textcolor{#B5396B}{\text{bias}^2}} + \underbrace{\textcolor{#3F7A6B}{\mathbb{E}\big[(\hat y-\mathbb{E}[\hat y])^2\big]}}_{\textcolor{#3F7A6B}{\text{variance}}} + \sigma^2_{\text{noise}}, \qquad y = f(x) + \text{noise}

Squared error against the degree of the polynomial, 0 to 9. Bias squared in magenta falls from 0.50 at degree 0 to near 0 from degree 3 on. Variance in green rises slowly, then steeply to 0.55 at degree 9. Their sum plus the noise, in dark ink, is U-shaped, lowest at degree 3, 0.13. A dotted line marks the noise, 0.09

Computed: 2,000 training sets

  • ff: the true curve; y^\hat y: the fitted prediction at xx; E\mathbb{E} averages over training sets. σnoise2\sigma^2_{\text{noise}}: the noise variance, 0.302=0.090.30^2 = 0.09 (squared errors here, not RMSE)
  • Bias falls and variance rises with the degree: the best model balances the two, here at degree 3
Geman, Bienenstock & Doursat Neural Comput 1992

Many dimensions, few examples

Accuracy against the number of input dimensions on a log axis, from 1 to 1024, with 100 training points. Training accuracy, in magenta, stays near 1.0 throughout. Accuracy on new points, in green, starts at 1.0 and falls to about 0.58 at 1024 dimensions; the shaded gap between the curves widens. A dashed line marks 100 dimensions, as many as training points

  • Simulated points: the label is the sign of the first input; every other input is random noise. A linear readout, 100 training points
  • Training accuracy stays near 1.0; accuracy on new points falls toward chance
  • The curse of dimensionality (baselines lecture): the examples needed grow quickly with the number of dimensions. With 1,024 weights and 100 examples, the readout fits the training labels partly through the noise dimensions
  • The fix: a representation, or a limit on the readout, that ignores the irrelevant dimensions

Learning a new category from five photos

Reproduced from Sorscher, Ganguli and Sompolinsky 2022, Fig. 1A and B. A: four photographs each of two animals the network was never trained on, a coati and a numbat, and a new photograph marked with a question mark. B: the primate ventral stream from the retina to IT drawn above a deep network trained on ImageNet; each image becomes a point in the network's feature space, the labelled examples of each animal are averaged into a prototype, and a straight boundary halfway between the two prototypes labels the new image

  • ResNet-50 trained on ImageNet, frozen; 1,000 new categories (ImageNet-21k) it never saw
  • 5 photos each; an animal's average is its prototype, and a new photo takes the nearer one's label: 98.6% correct (raw pixels: 66%)
Sorscher, Ganguli & Sompolinsky PNAS 2022

Five photos are enough, in a good representation

Reproduced from Sorscher, Ganguli and Sompolinsky 2022: accuracy of the prototype readout on new categories against the number of examples per category, for the trained network, an untrained network and raw pixels, and the five-photo accuracy read from monkey IT and V4 recordings compared with network layers

  • Pairs of the 1,000 new categories: 98.6% from 5 photos (untrained network 56%, pixels 66%)
  • Monkey IT (168 sites, 64 objects): 84% (AlexNet 80%, ResNet-50 93%). Category geometry predicts the errors
Sorscher, Ganguli & Sompolinsky PNAS 2022 · Majaj, Hong, Solomon & DiCarlo J Neurosci 2015

In PyTorch: keep the best checkpoint

best_acc = -float("inf")                    # best validation accuracy so far
for epoch in range(num_epochs):             # one epoch = one pass over the training set
    model.train()                           # training mode (dropout on)
    for X, y in train_loader:               # one minibatch at a time
        optimizer.zero_grad()               # clear last step's gradients
        loss_fn(model(X), y).backward()     # loss on this batch, then backprop
        optimizer.step()                    # update every weight
    model.eval()                            # evaluation mode (dropout off)
    with torch.no_grad():                   # no gradients needed to score
        correct = sum((model(X).argmax(1) == y).sum().item()
                      for X, y in val_loader)   # count right answers
    val_acc = correct / len(val_loader.dataset)
    if val_acc > best_acc:                  # better than every earlier epoch?
        best_acc = val_acc
        torch.save(model.state_dict(), "best.pt")   # save the weights: a checkpoint
model.load_state_dict(torch.load("best.pt"))        # end: reload the best weights

- Last lecture ended with a network that learns by gradient descent, with the gradient from backpropagation and autograd - One step means one update of every weight, using the gradient of the loss averaged over the examples it was computed on. In last lecture's loop that was the whole training set, every time. That works for a few hundred examples, not for the million images a vision network is trained on - And a low training loss was the only goal. Today asks whether that is the right goal

- The three questions follow one network from training, to testing, to opening it up - The assignment due Sunday uses all three: a readout trained on minibatches with Adam and a weight penalty, its train and test accuracy, and tuning curves, Gabor fits and selectivity of AlexNet units and IT neurons

- Three examples recur through the lecture, so their details are given once, here. They are deliberately different models: a single linear neuron, a polynomial of one input, a readout of recorded IT responses. Learning rates, momentum, overfitting, splits and penalties apply to all three in the same way, and to deep networks: the last section looks inside AlexNet - The two-weight neuron: ŷ = w1 x1 + w2 x2, fit by squared error to 24 examples simulated from weights (1, −0.3) plus a little noise; the contour plot is its loss, from last lecture's slide "Learning the weights: start anywhere, walk downhill". Used for minibatches, the learning-rate schedule, momentum and Adam - The curve fit: y = sin(2πx) plus noise, 10 training points, polynomials of degree 0 to 9 (Bishop 2006). Used for underfitting, overfitting, early stopping and the weight penalty - The IT readout: trial-averaged responses of 100 randomly chosen IT sites to the 1,224 object images of Bao et al. (2020): 51 objects, 24 views each, in six categories (animal, vehicle, face, vegetable or fruit, house, man-made object). A readout names the category; chance is 1 in 6. Used for the learning rate, learning curves, the precision of a test accuracy and the weight penalty

- Bao, She, McGill & Tsao (2020) recorded neurons in macaque IT while the monkeys viewed 51 objects, each rendered from 24 viewpoints, 1,224 images in all. The first two assignments used these recordings - A site is one recording channel. Its response to an image is the firing rate averaged over the repeated presentations of that image - We keep 100 of the recorded sites, chosen at random once (seed 0), and use the same 100 throughout the lecture - The readout's task: from the 100 numbers an image evokes, name its category (animal, vehicle, face, vegetable or fruit, house, man-made object). Readouts differ only in how they map the 100 numbers to the category: a logistic regression (a weighted sum per category, then softmax) or a small network with one hidden layer

- Two changes make training on large data sets possible: compute each step from a small batch of examples, and shrink the step size as training goes - Then Adam, the optimizer the assignment uses, and a measurement of how much the learning rate matters

- Why one example gives a noisy gradient: the gradient of one example's loss points the way that fits that example best. Two images of cats may pull a weight in opposite directions (one is dark, one is light), and an image with an unusual label pulls it somewhere no other image would. The average over all examples is the direction that helps most of them; one example, or a few, give a noisy estimate of it - Three ways to compute a step's gradient. The perceptron and the delta rule updated after every example (online). Last lecture's loop averaged over every training example (full batch). Practice sits in between - Why not full batch: one update per pass through a million images is slow, and the activations of every image must be held at once to compute the gradient, far more than a graphics card's memory. Why not online: one image at a time wastes the graphics card, which processes a batch of images in parallel as fast as one, and each step follows a single example's noisy gradient - Why minibatches converge in fewer epochs: one pass through the data costs the same computation either way, since every example is processed once. Full batch spends that pass on a single update. Minibatches of B spend it on N/B updates: with 50,000 images and B = 50, a thousand small steps instead of one. Each step is noisier, but the many steps usually reach a low loss in a small fraction of the epochs - One step is also called an iteration or an update. With B = 1 the method is the original stochastic gradient descent; PyTorch's optim.SGD takes this step at any batch size - Does the noise of small batches help? Several studies found that small batches reach better test accuracy than very large ones (Keskar et al. 2017; Masters & Luschi 2018). Shallue et al. (2019) found that the difference largely disappears when the learning rate and other settings are tuned separately for each batch size. Small batches are at least as good, and often what fits in memory - In PyTorch: `train_set = TensorDataset(X_train, y_train)`, then `for xb, yb in DataLoader(train_set, batch_size=64, shuffle=True):` - This DataLoader loop goes inside the epoch loop, so one epoch is now many steps. The five lines of the training loop go inside the DataLoader loop, with xb and yb in place of X_train and y_train; the loss is the average over the batch - Parameters are what training learns (weights and biases). Hyperparameters are what we choose before training: the learning rate η, the batch size B, the number of epochs, the width and depth - The assignment trains its hidden-layer probe on minibatches of 64 images (32 when the network learns new objects) - Robbins & Monro (1951) showed that noisy gradient steps converge if the step size shrinks at the right rate, under assumptions about the loss and the noise. The next slide shows schedules that shrink it

- Minibatches make each step cheap but noisy. The next slides are three things practitioners do about that, each shown on a small problem where you can see it work - All three are a line or two of PyTorch: a scheduler, the choice of optimizer, and a loop over learning rates

- The neuron and data are last lecture's (the slide "Learning the weights: start anywhere, walk downhill"): 24 simulated examples made by a linear neuron with weights (1, −0.3) plus a little noise, and the mean squared error as the loss. The contours are that loss; the star is its lowest point, which sits almost exactly on the weights that made the data - Both runs start from the same random weights (off the plotted area, top left) and use the same sequence of minibatches, each 2 examples drawn at random. Only the step size differs: 0.8 throughout (left), or 0.8/(1 + t/20), which halves after 20 steps (right) - Each minibatch gradient points roughly toward the star but with noise, because two examples are a poor sample of 24. The size of a step is the learning rate times the gradient. With a constant learning rate, the last steps are as large as the first ones, so each noisy gradient still throws the weights a long way and they keep jumping around the lowest point: on average 0.12 from it over the last 300 steps. With a shrinking learning rate each jump gets smaller, and the weights settle within 0.01 of it - A network's loss has many regions where it is low and rises all around (valleys, in the picture of the loss as a landscape over the weights). The same idea applies: large noisy steps early move the weights far, small steps late let them settle in one of those regions

- What changed from the previous slide: the data are the same 24 examples, but the second input x2 is multiplied by 5, as if it were recorded in other units. The neuron fits the same outputs with w2 five times smaller, so the lowest loss moves from (1, −0.3) to about (1, −0.06). Nothing else changes; the axes are drawn on equal scales, as before - Why a valley: multiplying an input by 5 makes the loss 5 × 5 = 25 times steeper along its weight. Because the two inputs are also correlated, the loss rises about 50 times faster across the valley than along it. Inputs on very different scales make such valleys, which is one reason inputs are standardized before training - Why the zigzag: the gradient points straight downhill, across the contour lines, so here it points mostly across the valley. With one learning rate for both weights, a step that makes useful progress along the valley is far too large across it and lands on the other side, and the next step comes back - Both runs use the same learning rate, 0.089, the best for plain gradient descent; the only difference is the running average m (β = 0.9). The up-and-down pushes alternate in sign from step to step and largely cancel in the average; the pushes along the valley all agree and add up. Up-and-down travel over the 40 steps falls from 4.4 to 0.8 - Why momentum's path is still wavy: the fast zigzag flips sign every step, so an average over about 10 steps (β = 0.9) cancels it. The slow swing that remains is momentum's own inertia: the averaged velocity carries the weights past the valley floor before turning them back, like a ball rolling in a bowl with friction. A larger β means more inertia and longer, slower swings - Momentum works without Adam: plain gradient descent with momentum, torch.optim.SGD(model.parameters(), lr=0.1, momentum=0.9), is the classic recipe; the standard ResNet training on ImageNet uses exactly that - Full batches, so no minibatch noise: this isolates what the averaging does

- Adam = the running average of the gradient (m, momentum) + a separate scale for each weight (√v, the running size of that weight's gradient). The next slide writes the update - Here Adam uses the largest learning rate for which it does not overshoot the lowest loss along the valley (0.076), to show its path; up-and-down travel 0.6 - The step counts on the slide come from separate runs, each method at the learning rate that reaches 1% of the starting loss fastest: plain descent does not reach it within 40 steps (its rate must stay small or it blows up across the valley), momentum 8, Adam 6. At those fast rates momentum and Adam overshoot once before settling, which is why the figure shows gentler runs - Real network losses have such narrow valleys in many of their millions of directions, which is one reason Adam often needs fewer steps. It still needs a learning rate: two slides on

- Why m over √v works: if a weight's recent gradients all point the same way, m is about as large as √v and the step is about η. If they flip sign from step to step, m averages to near zero while v stays large, so the step is small. The ratio acts like a signal-to-noise ratio for each weight - The name: Adam stands for adaptive moment estimation, from the two running averages. m estimates the mean of the gradient (its first moment), v the mean of its square (its second moment) - Adam usually reaches a low loss in fewer steps than plain gradient descent, but it still needs a learning rate (the next slide) - A schedule changes the step size over time, the same for every weight. Adam changes it for each weight separately and smooths the noisy minibatch gradients. The assignment uses Adam in every training loop you write - ∇ℒ is the gradient of the loss on the current minibatch, one entry per weight. Plain stochastic gradient descent uses it directly, θ ← θ − η∇ℒ. Adam changes only what is done with it, in PyTorch the opt.step() line - Operations on vectors are element by element: (∇ℒ)² squares each weight's gradient, and the division is weight by weight - The update shown omits Adam's start-up correction. m and v start at zero, so they are too small during the first steps. Adam divides them by 1 − β₁ᵗ and 1 − β₂ᵗ, where t counts steps, which matters only early in training - Adam is cheap: two running averages per weight, and no second derivatives (how fast a slope itself changes), which some slower optimizers use - AdamW, Loshchilov & Hutter (2019), with a separate weight decay, is the usual choice for large language models - Adam is not always best. On some image datasets, well-tuned gradient descent with momentum generalizes as well or better, Wilson et al. (2017). Its advantage is that it works with little tuning

- The network: the 100 IT responses to an image go into a hidden layer of 64 units, each a weighted sum followed by a ReLU, then into 6 output units, one per category, with a softmax. Training lowers the cross-entropy between the softmax output and the true category, as in last lecture's training loop - Training set: 918 images; test set: the other 306. Minibatches of 32, 30 epochs, plain gradient steps (no momentum, no Adam), four learning rates - Results, final training loss and test accuracy: 10⁻⁵, the loss barely moves, test 0.18; 10⁻³, slow, test 0.62; 10⁻¹, loss 0.015, test 0.91; 3, the loss passes 10²⁰ in the first epoch and the run is lost - The working range spans orders of magnitude, so try learning rates 10 times apart (10⁻⁴, 10⁻³, 10⁻², …) before fine-tuning. Do it first, because every other setting depends on it - Adam needs a learning rate too. Because it divides each step by the typical size of the gradient, it tolerates larger rates than plain steps do; plain steps show the failure more clearly here

- Generalization: how well what a model learned from its training data holds for new data from the same source. The baselines lecture asked this for readouts of IT; today it is the same question for any trained network, and what to do when the answer is no - Training lowers the loss on the training images, but what we care about is the performance on new images. The next sections measure that performance, show where a good test score can still mislead, and improve it. The last section opens a trained network up

- A reminder of the baselines lecture before going further. The readout is the logistic-regression probe of that lecture; the data are the monkey IT recordings of the first two assignments - Shuffling breaks every link between the images and the labels, so nothing about animacy is left to learn. Ask yourself what the training accuracy and the test accuracy will be before the next slide

- The figure is from the baselines lecture: the animate vs inanimate readout of monkey IT (Bao et al. 2020 recordings), 102 training images, 306 test images, 1,000 shuffles of the training labels - With true labels the generalization gap is 0.04 (1.00 − 0.96); with shuffled labels it is 0.44 (1.00 − 0.56) - Zhang et al. (2017), Table 2: Inception v3 trained on ImageNet (1.2 million photos, 1,000 categories) with no regularization. With the true labels: 99.5% correct on the training photos, 59.8% on new ones. With labels drawn at random: 95.2% on the training photos, 0.1% on new ones, which is chance for 1,000 categories. A network with enough capacity, the number of different labelings it can fit, can memorize any labels; the same network generalizes when the labels are real - Training accuracy alone cannot tell a network that learned the categories from one that memorized the images - Minute paper 11 asks about this slide - The assignment measures the train–test gap of a readout and refits it with shuffled labels - The previous lecture trained networks by backpropagation. Whatever the learning rule, brain or network, the test of what it learned is the same: performance on new data

- Every model of the course so far is such a function. The perceptron and the delta rule map two inputs to one output. The probe of the baselines lecture maps 480 IT responses to "animate" or "inanimate". Last lecture's network maps two inputs to XOR through a hidden layer - Training adjusts θ, the weights and biases, to lower the loss on the training pairs. Nothing in training looks at new inputs, which is why generalization has to be measured separately - The next slides use a function of a single input x, a curve, because then the whole function can be drawn and compared with the points it was fit to

- Computed: y = sin(2πx) plus noise with standard deviation 0.3, at 10 evenly spaced x between 0 and 1. Least-squares polynomials of degree 0, 1, 3 and 9 - Errors are root-mean-square errors (RMSE): the square root of the mean squared error, the loss of the delta rule and of linear regression. Taking the square root puts the error back in the units of y, so 0.3 means the fit misses a typical point by about 0.3 - A polynomial of degree d has at most d − 1 turning points (peaks and troughs) and at most d places where it crosses zero. Degree 1 is a line, degree 3 can bend twice, degree 9 up to eight times - Error on new points: 2,000 new noisy samples from the same curve, not used in the fit - A polynomial of degree 9 has 10 coefficients, one per training point, so it can pass through all 10 points exactly. It fits the noise too, and swings far off between and beyond the points - The best degree here is 3, with error 0.33 on new points. The noise alone gives 0.30, so no curve can do much better - Why a curve? The readouts of the assignments are fitted functions too, with hundreds of weights and a few hundred images, but they cannot be drawn. The polynomial is the smallest case where the whole function, its flexibility (number of coefficients) and its errors can all be seen - The same trade-off decides how strong a penalty the ridge readouts of the next assignment need, and the shuffled-label probe of the last slide is the degree-9 polynomial of a readout: flexible enough to fit any labels - This example and the next follow Bishop (2006), Figs 1.4 and 1.6, the standard textbook illustration

- The previous slide changed the model and kept the data. This one keeps the model and adds data - Computed as before. Error on new points 0.86 with N = 10, 0.34 with 15, 0.35 with 30, 0.31 with 100. Training error rises from 0.00 to 0.28, toward the noise level of 0.30: with many points the curve can no longer fit the noise - A network's flexibility is harder to count than 10 coefficients, but the same trade holds: more training images make the same network overfit less

- Every choice we make while looking at a score (the degree, the number of epochs, the learning rate) can fit that score's particular images - So the images used to choose must be kept apart from the images used to report

- Held-out data has been with us since the early lectures. With choices to make, it now comes in two parts: validation for choosing, test for reporting - Parameters are what training learns (weights and biases). Hyperparameters are what we choose: learning rate, batch size, number of epochs, width, depth, penalty strength - Any step that learns from data, such as PCA or z-scoring, is fit on the training images only. The current assignment fits PCA this way - A train and test curve plotted side by side to look at the gap, as in the current assignment, is exploration: nothing is chosen on it. A curve used to choose an epoch or a penalty must be a validation curve. The next assignment uses all three splits

- The baselines lecture scored readouts of IT this way. Here it chooses a setting: for each candidate (a learning rate, a penalty λ, a degree), run all k folds and average the k validation scores; keep the candidate with the best mean - Every training example is used for validation exactly once and for training k − 1 times, so no example is wasted. The price is k fits per candidate - k = 5 or 10 is common. Leave-one-out sets k to the number of examples: N fits, each on all examples but one. For ridge regression it has a closed form, which is why RidgeCV (the ridge slide, end of the next section) uses it at no extra cost - After choosing, refit once on all the training data with the chosen setting; the test set then scores that one model, once - Anything fit to data inside the loop, such as z-scoring or PCA, is fit on the k − 1 training parts of each fold only

- Computed: the IT recordings of the first two assignments (Bao et al. 2020). A network with one hidden layer of 256 ReLU units, trained with Adam, learning rate 10⁻³, minibatches of 32, reads out the six categories from 100 randomly chosen IT sites. 58 of the 300 training labels were redrawn at random (46 ended up wrong), as happens when images are labelled quickly; the 400 validation images keep their true labels - Validation loss is lowest at epoch 20 (0.561) and is 0.972 at epoch 400. Validation accuracy is highest at epoch 8 (0.820) and is 0.775 at epoch 400. Training accuracy reaches 1.000: the network has memorized the wrong labels too - With all 482 sites the curves barely turn: in so many dimensions the network can memorize the wrong labels without disturbing its answers on other images (the many-dimensions slide, in the appendix). With 100 sites, memorizing them costs accuracy on new images - Loss and accuracy need not agree on the best epoch. The loss also punishes confident wrong answers, so it can rise while the accuracy holds steady. Pick one validation score in advance and keep the checkpoint where it is best - Early stopping is a form of regularization: any change to training that gives up some fit to the training data to do better on new data. Fewer steps leave the weights less time to fit the noise. Regularization is the subject of the next section - The next assignment plots learning curves and keeps the best checkpoint

- Computed: the 10 points of "Too rigid, or too flexible", a degree-9 polynomial, full-batch gradient descent on the mean squared error from all coefficients at 0, learning rate 0.64. Test RMSE on the same 2,000 new points - Gradient descent fits the directions in which the loss is steep first: the broad shape of the data. The directions that bend the curve through each noisy point are shallow and take millions of steps. Stopping in between keeps the shape and leaves the noise unfit - Test RMSE: 0.53 after 10 steps, lowest at step 1,467 (0.33, training RMSE 0.12), then 0.86 after 10¹⁰ steps, the degree-9 fit of "Too rigid, or too flexible" that passes through every point - The steps are on a log axis: the late rise in test RMSE takes 10 million steps or more on this tiny problem. On a network the same turn appears within tens of epochs, as on the previous slide - Stopping early keeps the weights closer to where they started, a limit on how closely they can fit the training data. Later in this lecture this idea gets a name, regularization, and the weight penalty does something similar - In practice you do not see the test curve: the validation set stands in for the new points, as on the previous slide

- This is the law of large numbers at work: as the test set grows, the measured accuracy converges to the true accuracy, the one we would get on every possible new image. How fast it converges comes from the binomial spread (or the central limit theorem): the interval shrinks as 1 over the square root of the number of test images, so four times the images halve its width - Computed: a logistic-regression readout of the six categories from the same 100 IT sites (Bao et al. 2020), trained on 300 images; for each test-set size, one random set of test images from the other 924, resampled 2,000 times. The interval holds the middle 95% of the resampled accuracies - The interval width falls roughly as one over the square root of the number of test images: 0.24 with 25, 0.15 with 100, 0.05 with 800 - To compare two models on the same test images, resample the same images for both and look at the difference (a paired bootstrap)

- Everything so far assumed that test images come from the same source as training images. In the world they often do not - A network trained on ImageNet photographs scores highly on new ImageNet photographs, its test set. Images of another kind can expose what it relies on - ResNet-50: a 50-layer image network trained on ImageNet, 1.2 million photographs in 1,000 categories. The cue-conflict images were made by style transfer, the cat's shape painted with elephant-skin texture - The pictured cat illustrates the conflict. The percentages summarize the whole experiment: across the cue-conflict images, counting only answers that named either the shape category or the texture category, people chose the shape in 95.9% of them, ResNet-50 in 22.1% - Training on stylized images, which removes texture as a cue, raised the network's shape choices - Geirhos et al. (2020) review shortcut learning across tasks: a pneumonia detector that read the hospital's marker in the X-ray, an animal classifier that used the background - A held-out test set can miss a shortcut when the same cue–label relationship holds in the training and test images. When you look at a network's errors, ask which cue it might have used - The next assignment covers parts of two photographs and watches a pretrained network's answer change

- The next slides show three ways: penalize large weights, train on altered copies of the images, and switch off random units during training. Early stopping, from the learning curves, is a fourth - All four are regularization: they give up some fit to the training data to do better on new data

- The polynomial with 10 coefficients and 10 points, from the start of the lecture. Without a penalty the coefficients reach about a million in size; with λ = 10⁻³, about 17 - Large coefficients are what let the curve swing between the points. Keeping them small keeps the curve smooth - θ stands for all the parameters. In the polynomial, θ is all 10 coefficients, the constant term included, and the penalty covers every one of them. λ is the strength of the penalty, a hyperparameter (elsewhere in the course λ is an eigenvalue). Choose λ on validation data, never on the test set - The current assignment adds this penalty to a readout - Bishop (2006), Fig. 1.7, shows the same effect

- Computed: the running readout (six categories, 100 IT sites) as a softmax logistic regression on 300 training images, full-batch gradient descent for 20,000 steps, without a penalty and with λ = 0.01 (bias not penalized). Left: the length of w; right: the training cross-entropy, both against the number of steps. Both classify all 300 training images correctly; on the other 924 images, 0.851 without the penalty and 0.860 with it - Magenta: no penalty. Green: weight decay. A longer w makes the readout more confident - Not every unpenalized readout grows. With cross-entropy, the weights grow without end only when the readout can classify every training image correctly (the classes are separable in the training set): then there is no finite best w. If some training images stay wrong whatever w is, a finite w minimizes the loss and gradient descent settles there. With squared error (linear regression) a finite best always exists. With 100 sites and only 300 images, separating them is easy, which is why it happens here - When every training image is already classified correctly, scaling w up pushes every p̂ for the true class closer to 1, so the cross-entropy keeps falling: there is no finite best w, only a best direction. Without a penalty the norm is 10.7 after 20,000 steps and still rising - With λ = 0.01 the norm levels off at 3.5 and the cross-entropy near 0.11. With 100 inputs and six classes, the penalty changes the direction of w as well as its length - scikit-learn's LogisticRegression adds this penalty by default. A PyTorch loop does not, unless you pass weight_decay to the optimizer - For plain gradient descent (torch.optim.SGD), weight_decay adds weight_decay × θ to each gradient, so every step also shrinks each weight by a fixed fraction. That is the gradient of the penalty with λ = weight_decay / 2 - torch.optim.Adam includes the penalty in its gradient and then rescales that gradient. AdamW shrinks the weights separately, at each step (Loshchilov & Hutter 2019) - By default, PyTorch applies weight_decay to every parameter it is given, biases included

- More data is the most reliable fix for overfitting, when it can be had. Labelled data is expensive, so the next best thing is to make more from what you have - Each epoch shows a different random version of each image, which makes memorizing the training images harder - Flips that would change the label (a 6 and a 9, a b and a d) must be left out. Speech recognizers train on recordings with stretches of time and bands of frequency masked out (SpecAugment, Park et al. 2019); medical-image networks on rotated and re-lit scans - Krizhevsky et al. (2012) trained AlexNet on random 224 × 224 crops of each 256 × 256 photo and their mirror images, which multiplies the training set by 2,048, plus colour changes - Photograph: THINGSplus, public domain (CC0)

- Regularization is any change to training that gives up some fit to the training data to do better on new data. The weight penalty of the previous slides is one; early stopping, from the learning curves, is another - Dropout (Srivastava et al. 2014): each hidden unit is switched off with probability p (often one half) at every training step, a different random set each time. A unit cannot count on any particular other unit, so the network cannot build fragile combinations that fit single training images. At test time all units are on, with their outputs scaled so their average matches training. AlexNet used dropout in its fully connected layers; without it, Krizhevsky et al. (2012) wrote, the network "exhibits substantial overfitting" - L1 penalizes the sum of the absolute values of the weights instead of their squares. It drives many weights to exactly zero, so the model uses fewer inputs - The penalties differ in detail, but they share one effect: they restrict which weight settings training can reach, so the network has fewer ways to fit the noise in its training data - In PyTorch: nn.Dropout(p=0.5) as a layer; model.train() switches it on and model.eval() switches it off. Weight decay is the weight_decay argument of the optimizer

- The same playground as last lecture, with three additions: held-out points (open circles) that training never sees, the held-out loss as a green curve, and a weight penalty and dropout - Default: two moons, 30 training points with 15% of their labels flipped, two hidden layers of 32 ReLU units, learning rate 0.03. The held-out points keep their true labels, so the held-out loss measures how well the boundary matches the real one - Unregularized: the held-out loss is lowest near epoch 100 (about 0.43), then rises to about 1.0 by epoch 5,000 as the boundary bends around the flipped points; held-out accuracy about 75% - With λ = 0.01 the held-out loss stays near 0.44 (accuracy about 82%). Dropout 0.5 brings it down to about 0.27 (about 88%). With 200 training points instead of 30 it falls to about 0.18, and every held-out point is classified correctly - The loss is the cross-entropy of a sigmoid output, as in last lecture's playground. The weight penalty adds λ·w to each weight's gradient, the gradient of (λ/2)‖w‖². Dropout silences each hidden unit with probability p on every training step and is switched off for the held-out loss and the picture - The demo is on the course site with the other demos. Play with it after class

- The weight penalty of this section, applied to a model of a neuron instead of a curve. The assignment fits this model, and the next assignment uses the same ridge readout - An encoding model predicts a neuron's response to each image from a network's features. With the penalty λ‖w‖² added to the squared error, the fit is called ridge regression. It is scored by R² on held-out images - The target is one IT site's response to each of the 1,224 object images of Bao et al. (2020); the features are the 512 units of the trained ResNet-18's late layer. A random fifth of the images is held out and never seen by the fit - The model is a weighted sum of the features plus a bias, fit by least squares: linear regression. Run with a small enough step on a continuous target, the delta rule minimizes the same squared error - λ is chosen by leave-one-out cross-validation within the 980 training images (scikit-learn's RidgeCV). The held-out fifth only scores the chosen fit. The held-out curve on the right is drawn to show what the penalty does; it was not used to choose λ - R² = 1 − (squared error of the prediction) / (squared error of predicting the mean response). 1 is perfect, 0 is no better than the mean, and it can be negative. The baselines lecture used the squared correlation, which is close for a good fit - Right panel: with almost no penalty, 512 weights fit the training images closely (R² 0.85) and predict held-out images poorly (0.28): overfitting. As λ grows, the training fit gets worse and the held-out fit better, up to a point (0.69) - The current assignment fits this model with nested cross-validation: λ is chosen inside each training fold, and the held-out R² comes from the outer folds

- In 1959 Hubel and Wiesel spent hours showing spots of light to a neuron in a cat's visual cortex without a response. It fired when the edge of the glass slide in their projector swept across the screen: the cell responded to an edge at one orientation. That discovery, and the methods that followed, earned them the 1981 Nobel Prize - The last slide predicted a neuron from a network's units. Now the reverse: what do the units of a trained network respond to, measured the way we measure neurons? - The methods are those of physiologists: show stimuli, record the responses, summarize them. In an animal that is all we can do. In a network we can also read the weights, and check one against the other - The same measurements describe a V1 cell, an IT site and a unit of AlexNet. The assignment uses them on network units and on IT sites

- Each small image is one first-layer filter of AlexNet (torchvision's trained weights), its 11 × 11 weights for the three colour channels shown as a colour image. A filter is the unit's template: the image patch that drives it most - Training only asked the network to name objects. Nothing in the objective mentions edges, orientations or V1 - The rest of this section answers the question in three steps: what V1 receptive fields look like, a way to describe any receptive field with a few numbers, and then the comparison, measured - The assignment does exactly this comparison for AlexNet's first layer

- Reproduced from Ringach (2002), Fig. 2 (rows 1–2, 4–5 and 7–8 of the original). Each receptive field was mapped by showing the cell rapid sequences of gratings and averaging the stimulus that preceded each spike (reverse correlation) - Ringach's caption: "In general, 2-dimensional Gabor functions provide a good representation of the shapes of receptive fields in V1" - Daugman (1985) proposed the two-dimensional Gabor as a description of simple-cell receptive fields. Jones & Palmer (1987) mapped cat simple cells and showed that Gabors fit them closely, with residuals that looked like noise - A residual with structure in it, such as a stripe the fit left out, would mean a Gabor does not describe that field - The next slide fits Gabors to toy fields we built, and keeps two numbers from each fit

- For a first-layer unit, the input weights form an image, its receptive field. A deeper unit combines earlier units, so its weights are not an image - Computed on toy units, not on data. Left: a Gabor with added pixel noise, the least-squares Gabor fit, and the residual (the field minus the fit). The residual is only noise, so the fit has captured the structure - SS_res is the sum of the squared residuals. SS_tot is the sum of the squared deviations of the field from its mean. R² is 1 when the residual is zero, and falls as the fit misses more of the field - Here R² measures how well the fit reproduces the field it was fit to. Earlier, for the model of a neuron, R² scored predictions of held-out responses instead - A residual with structure in it, such as a stripe the fit left out, means a Gabor does not describe that field - The Gabor is a cosine grating (frequency f, phase φ) multiplied by a Gaussian window with widths σx across and σy along the stripes; x′ and y′ are the image coordinates rotated to the stripes' orientation. Right: nx = σx·f and ny = σy·f are the window's standard deviations measured in cycles of the stripes (Ringach 2002). Below about 0.25 on both, the window holds only a small part of one cycle: a small spot with little orientation (A). Larger values mean more stripes, and longer ones (C) - Keeping only good fits means the comparison that follows says nothing about the fields a Gabor does not describe - The assignment fits a Gabor to each of AlexNet's first-layer filters

- Ringach (2002) mapped the receptive fields of 93 macaque V1 simple cells, fit a Gabor to each, and released the fits. The slate-grey squares are his cells: 71 of the 93 have both n_x and n_y below 0.6, and the medians are 0.32 and 0.41. The stars are the toy units of the Gabor-fit slide, for reference - nx measures the window across the stripes, ny along them, both as the window's standard deviation in wavelengths of the stripe pattern. A Gaussian window has no hard edge - The assignment places AlexNet's first-layer filters on this plot - Two summary numbers are a weak test of whether two sets of receptive fields are alike. Predicting a neuron's responses from a network is a stronger test: the model of a neuron earlier today was a first example, and the lecture on comparing models with brains takes it further

- The network's points are AlexNet's first-layer filters with a Gabor fit of R² above 0.75, each fit to the filter's mean over its three colour channels - The assignment makes this plot and asks these two questions - Nobody built Gabors into AlexNet: its first layer learned them from labelled photographs by backpropagation. A later lecture shows that a different objective, sparse coding of natural images, also learns them

- In an animal we cannot read the weights. We can only show stimuli and record. With a network unit we can do the same, and check the result against the weights - Hubel & Wiesel (1962) measured orientation tuning this way in cat visual cortex, with bars of light at many orientations - Computed: four 41 × 41 receptive fields built by hand. A, B and C are Gabors (an oriented stripe pattern inside a smooth elliptical, Gaussian window) with long, medium and short windows. D is a centre-surround unit like a retinal ganglion cell. Each is shown full-field gratings at 24 orientations and 4 phases - Top: each unit's weights and its orientation selectivity index (OSI). Bottom: its tuning curve, scaled to its own peak - Bandwidth is the half-width of the tuning curve at half its height above its minimum; the figure does not print it. Compare the narrow curve of A with the broad curve of C - The centre-surround unit is round, so it responds the same at every orientation. Its index is 0 and its curve is flat - A long window covers many points along the stripes, so a small tilt already misaligns them: narrow tuning. A short window tolerates more tilt: broad tuning. Both can still fall to near zero at the orthogonal orientation, which is why both indices are near 1 - The tuning curve follows from the template. It peaks where the grating best matches w - The assignment measures the tuning curves of AlexNet's first-layer units the same way

- Computed: a Gabor filter and a grating at its preferred orientation, at four evenly spaced phases. Responses scaled so the largest is 1 - The weighted sums are +1, +1, −1, −1 and average to 0, although this is the unit's preferred orientation. The cancellation is exact for a grating whose light and dark stripes balance, evenly spaced phases and no bias - After the ReLU they are 1, 1, 0, 0, average 0.5 - Why rectify: to compare a network unit with a recorded cell, the unit should behave like a cell, and a cell's firing rate cannot go below zero. A rectified weighted sum (a ReLU, or a rate that is zero below threshold) is the simplest such model, and AlexNet's first-layer units are followed by a ReLU anyway - What rectification is not needed for: the Gabor fit of the previous slides is fit to the weights, the receptive field itself, with no responses involved - Averaging a tuning curve over phase is an analysis step. A recorded cell's responses cannot cancel this way because they are never negative. That says nothing about how the cell combines its inputs - In V1, a simple cell responds to a grating only at some phases, like the rectified sum here. A complex cell responds at every phase. It behaves as if it pooled rectified simple-cell responses at different phases - The assignment rectifies each AlexNet response before averaging over phase

- Real data: trial-averaged responses of two IT sites to the 1,224 images of Bao et al. (2020), 51 objects in six categories with 24 views each. Each dot is one view, each bar an object's mean - IT sites respond to whole objects. The geometry lecture split the variance of whole populations of units the same way. Here it describes one site - Site A's object means differ a lot compared with the spread of its views: 60% of its variance lies between objects. Site B responds about as differently to two views of one object as to two objects: 22% - Here every object has 24 views, so the total variance equals the variance of the object means plus the mean within-object variance. With unequal numbers of views, each object's part must be weighted by its count - A different measure from the orientation selectivity index, which compares the peak and trough of a tuning curve. This one asks how much of all the variation is about identity - Trial-averaged responses still carry some noise. It adds to both parts, mostly to the within-object part - Analysis of variance (ANOVA) uses the same split but divides each sum of squares by its degrees of freedom (the number of values free to vary once the group means are fixed): F = [SS_between / (G − 1)] / [SS_within / (N − G)], with G objects and N responses. Whether an F is large needs a reference distribution, the F distribution or a shuffle test. With categories as the groups, it measures category selectivity - The assignment computes this index for IT neurons and for network units

- In a network, ablating a unit means setting its output to zero for every image, then measuring the readout's test accuracy again. In an animal, lesions or optogenetic silencing play the same role - A unit can be selective and still not be needed. Take ten identical selective units. With a refitted readout, the remaining nine can replace the removed one. A fixed readout may still lose accuracy, since its weighted sum changes. The reverse also happens: a population can be decoded well when no single unit separates the classes - Without the random control, a drop in accuracy only tells you that removing units hurts. The control tells you whether the selective ones are special - The assignment removes the most selective units, or random ones, in IT and in the network, and refits the readout each time

- Next lecture: the visual system and convolutional networks. Each first-layer filter is slid over the whole image, which is convolution, and the first-layer Gabors meet V1

- Hubel & Wiesel (1962), Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. The classic behind "Measuring a tuning curve" - Ringach (2002), Spatial structure and symmetry of simple-cell receptive fields in macaque primary visual cortex. Goes with "Where V1 simple cells sit" - Sorscher, Ganguli & Sompolinsky (2022), Neural representational geometry underlies few-shot concept learning. Goes with "Generalization happens in a space" - Recht, Roelofs, Schmidt & Shankar (2019), Do ImageNet classifiers generalize to ImageNet? The ranking of networks held on the new test set, but every network lost accuracy: a new sample of a slightly different source is a small distribution shift. Goes with "Shortcut learning" - Geirhos, Jacobsen, Michaelis, Zemel, Brendel, Bethge & Wichmann (2020), Shortcut learning in deep neural networks. Goes with "Shortcut learning" - Before posting, skim the existing entries on your paper; your post must add something not already said

- Submitted on Canvas (Minute papers → Minute Paper 11); credit for a thoughtful attempt

- These slides belong to this lecture but were not presented in class

- The previous slide showed why to shrink the step. This one shows the shapes of schedules in common use, on one small problem - The story: all four runs fall quickly at first, because early steps are large in every schedule. The difference is at the end. With a constant step the loss keeps jumping (the dark trace in the inset); the shrinking schedules let it settle near the lowest possible loss (dashed line) - A minibatch gradient is noisy, and the noise does not shrink as the weights approach the minimum. A constant step keeps moving them around. Too large a step overshoots the valley, and too small a step takes many steps to get there - Computed: logistic regression on simulated, overlapping 2-D classes, minibatches of 8 examples, the cross-entropy loss, trained with each schedule from the same start and the same minibatches. The dashed line is the lowest loss the model can reach on these data - The third schedule written out: η_t = η0/(1 + t/τ), where η0 is the starting learning rate. Here η0 = 0.05 and τ = 100 steps, so the step has halved after 100 steps - Typical values: the classic ImageNet recipe for ResNets starts at 0.1 and divides it by 10 every 30 epochs (He et al. 2016). Cosine decay over the whole run is today's common default, often after a warm-up: a few epochs in which the learning rate grows from near 0 to its starting value, so that the first, large gradients do not throw the weights far off. The 1/(1 + t/τ) schedule is the one convergence proofs use; τ is chosen by hand like η, by trying a few values; a common choice makes the learning rate halve after one to a few epochs - Robbins & Monro's two conditions: the sum of all step sizes is infinite, so the weights can still travel as far as they need to. The sum of their squares is finite, so the noise of the gradients dies out. The 1/(1 + t/τ) schedule meets both. A fixed η fails the second - With extra assumptions about the loss and the gradient noise (for example, a bowl-shaped loss and gradient noise that stays bounded), steps that meet both conditions make stochastic gradient descent converge. For a network's loss, converging means reaching a point where the gradient is zero, which need not be the lowest loss - Schedules used for large networks (warmup, a short period of growing η, then cosine decay over a fixed number of steps) do not meet these conditions. They work well in practice and are tuned by experience

- Still the polynomial. One training set gives one fit. Asking how a method generalizes means asking what it would do with other training sets of the same size - Computed: the polynomial example; 20 training sets for the drawing, 2,000 for the numbers. Degree 1: bias² 0.222, variance 0.017. Degree 3: 0.007 and 0.030. Degree 9: 0.001 and 0.547 - Bias is the error of the average prediction. At each x, average the predictions of all the fits (the red curve, E[ŷ]) and compare with the true curve f(x): the shaded gap is f(x) − E[ŷ]. On the next slide, bias² is that gap squared and averaged over x. Variance is how much the prediction ŷ at a given x changes from one training set to another: the spread of the thin blue curves, not the spread of their errors - A weight penalty, later in this lecture, makes the degree-9 fits less sensitive to their noise: lower variance, at the cost of some bias

- The previous slide showed bias and variance as pictures. For squared error they add up exactly to the expected error on new points - At a fixed input x, bias and variance average over training sets. The expected error on the left also averages over the new point's own noise, which is independent of the training sets and has mean zero. The figure then averages all three terms over the inputs - σ²_noise is the variance of the noise around the true curve (0.3² = 0.09 here): no model can remove it - Computed: 2,000 training sets of 10 points for each degree. The expected error is lowest at degree 3 (0.127): bias² 0.007, variance 0.030, noise 0.090 - Bias here means a systematic error of the average fit, not the bias term b of a neuron - Geman, Bienenstock & Doursat (1992) brought this decomposition to neural networks. Its conclusion, that more flexible models overfit, is the classical view. Very large networks do not always follow it: they can memorize random labels yet generalize on real ones (Zhang et al. 2017, on the shuffled-label slide) - For a classifier scored by accuracy there is no exact decomposition, but the same two kinds of error exist

- The polynomial had one input. A network's hidden layer, or a layer of a trained network used as features, often has hundreds or thousands of units, and a readout on it has as many weights. A lab or an animal may have only a few hundred labelled examples - Computed: points drawn from a D-dimensional Gaussian; the label is the sign of the first coordinate. Logistic regression with almost no penalty, 100 training points, 2,000 new points, 20 repeats. With 1 dimension, 99.6% on new points; with 1,024, 58% - The task already needs only one dimension. The readout fails because it cannot tell which one: with D weights and fewer than D examples, it can fit the training labels using the noise dimensions, and those weights do not carry over to new points - Two fixes: a representation that drops or shrinks the irrelevant dimensions, or a limit on the readout's weights (a penalty, later in this lecture) so that it cannot fit their noise - The same failure as the shuffled-label readout of the baselines lecture (480 neurons, 102 images): more weights than examples - Curse of dimensionality: the number of examples needed grows quickly with the number of dimensions. Another face of it, also from the baselines lecture: in many dimensions, random directions are nearly perpendicular

- The previous slide ended on a fix: a representation that ignores the irrelevant dimensions. This is what such a representation buys, measured - More examples help; so does a better representation. Here the representation is fixed and the examples are few - Why a prototype? The simplest readout, an average of the labelled feature vectors: a neuron could compute it - The experiment, step by step (Sorscher et al. 2022, pp. 2–3 and Methods): train ResNet-50 on the 1,000 ImageNet categories and freeze it. Pick 1,000 new concepts from the larger ImageNet-21k collection, 500 photos each. Represent each photo by the network's 2,048 final features - For a pair of new concepts, average the few labelled feature vectors of each into a prototype. The readout's weight vector is the difference of the two prototypes, and the boundary sits halfway between them - Score on held-out photos of the same two concepts, over all 999,000 ordered pairs: 98.6% with 5 examples per concept, 92.0% with 1 - The theory predicts each pair's error from the geometry of the two object manifolds (geometry lecture): the distance between their centres, their radii and their dimension. Larger dimension can even help - The same analysis on monkey recordings (Majaj et al. 2015; 168 IT and 88 V4 sites, 64 objects): five-shot accuracy about 84% from IT, higher than from V4, and between AlexNet (80%) and ResNet-50 (93%) on the same images - Five examples suffice because the network's representation already separates new animals it was never trained on. The same examples can be too few in one representation and enough in another

- Accuracies are for telling apart two new categories, averaged over 999,000 ordered pairs of 1,000 new ImageNet-21k concepts (Fig. 1C) - The monkey numbers use the recordings of Majaj et al. (2015): 168 IT and 88 V4 sites, 3,200 images of 64 objects; five-shot accuracy from IT about 84%, higher than from V4 (Fig. 5) - The theory predicts each pair's error from the geometry of the two object manifolds (geometry lecture): the distance between their centres, their radii and their dimension - The same examples can be too few in one representation and enough in another: that is why the representation matters more than the readout

- The training loop of last lecture, with a validation pass after every epoch - model.train() and model.eval() switch layers that behave differently during training. One is dropout, which switches off random hidden units during training (later in this lecture). torch.no_grad() skips the bookkeeping for gradients, which validation does not need - model.state_dict() holds every parameter of the network; torch.save writes it to a file. That file is the checkpoint - The test set appears nowhere in the loop. It is used once, after the last line