Comparing networks with the brain

Reader-friendly version of the 47 lecture slides: the lecture content in normal flow, with figure descriptions and data tables where the figure carries data. Open the presentation.

Slide 1

Brown University crest
CPSY 1291 Computational Methods for Mind, Brain & Behavior
Lecture 13 · Theme 2: Learning representations: from neurons to deep networks

Comparing networks with the brain

Thursday, October 29 · Fall 2026

Return to contents

Slide 2

Backpropagation trains a network, and it can also question one

Hold the weights fixed and take the gradient with respect to the image

Return to contents

Slide 3

A network is a function of its weights and of its input

Contour map of the logistic loss of a one-weight network over the weight w (horizontal) and the input x (vertical). A magenta training path moves right along w at fixed x, from w = 0.15 to w = 2.62, loss 0.63 to 0.12. From the trained point a blue attack path moves down along x at fixed w, crosses the dashed decision boundary at x = 0, and the loss rises to 1.98

Contour map of the logistic loss of a one-weight network over the weight w (horizontal) and the input x (vertical). A magenta training path moves right along w at fixed x, from w = 0.15 to w = 2.62, loss 0.63 to 0.12. From the trained point a blue attack path moves down along x at fixed w, crosses the dashed decision boundary at x = 0, and the loss rises to 1.98

Open full-size figure

Computed · the smallest network: one weight, one input, logistic loss

  • A network computes y^=f(x; θ)\hat{\boldsymbol{y}} = f(\textcolor{#3E6D8E}{\boldsymbol{x}};\,\textcolor{#B5396B}{\boldsymbol{\theta}}). Its loss ℓ\ell depends on both. The network in the figure has a single weight, w\textcolor{#B5396B}{w}
  • In training, we keep the image fixed and change the weights to lower the loss, θ←θ−η ∇θ ℓ\textcolor{#B5396B}{\boldsymbol{\theta}} \leftarrow \textcolor{#B5396B}{\boldsymbol{\theta}} - \eta\,\nabla_{\textcolor{#B5396B}{\boldsymbol{\theta}}}\,\ell
  • In an attack, we keep the weights fixed and change the pixels to raise the loss. Backpropagation down to the pixels gives one slope per pixel, ∇x ℓ\nabla_{\textcolor{#3E6D8E}{\boldsymbol{x}}}\,\ell. A few small steps push the image across the decision boundary
  • ResNet-50: 25.6 million weights, 150,528 pixel values (224 × 224 × 3)

Return to contents

Slide 4

Backpropagation, four uses

Backpropagation, four uses — table
use gradient of variable then steps
training loss ℓ\ell weights θ\textcolor{#B5396B}{\boldsymbol{\theta}} step down many
adversarial example loss ℓ\ell pixels x\textcolor{#3E6D8E}{\boldsymbol{x}} step up one, tiny
attribution map logit zcz_c pixels x\textcolor{#3E6D8E}{\boldsymbol{x}} the gradient is the map none
feature visualization one channel's activation pixels x\textcolor{#3E6D8E}{\boldsymbol{x}} step up many, from noise

Return to contents

Slide 5

Training: the gradient flows back to the weights

A schematic network, left to right: the petunia photograph, image x; conv blocks with weights theta; feature maps A of layer l; more conv; the last feature maps A of layer L, channels k; class scores z; and at the right, the objective box. Gray arrows carry the forward pass; a red arrow carries the gradient backward. Here the objective is the loss of z and the label y; the red arrow branches up into every conv block; the image is fixed and the weights change, theta gets theta minus eta times the gradient of the loss

Schematic, one frame of six with an identical layout. Left to right along the top: the petunia photograph labeled image x, two conv blocks marked with the weights theta in magenta, a small stack of three feature maps labeled A superscript l, a third conv block, a stack of four feature maps labeled A superscript L, channels c, a column of five class scores labeled z, and at the right a red-outlined box labeled objective at the top. Gray arrows carry the forward pass left to right. A red gradient arrow drops from the objective box and runs right to left along the bottom of the network. Training. Objective: loss l of z and y. Tags: image x fixed (closed lock); weights theta changes (open lock, bold). The red arrow runs back to the first conv block and sends an upward branch into each of the three conv blocks. Below the conv blocks: theta gets theta minus eta times the gradient of the loss with respect to theta; change the weights.

Open full-size figure

Return to contents

Slide 6

An adversarial example steps the pixels up the loss

A schematic network, left to right: the petunia photograph, image x; conv blocks with weights theta; feature maps A of layer l; more conv; the last feature maps A of layer L, channels k; class scores z; and at the right, the objective box. Gray arrows carry the forward pass; a red arrow carries the gradient backward. Here the objective is the loss, to go up; the weights are fixed and the red arrow runs all the way back to the pixels; below, the petunia unchanged to the eye, and x gets x plus epsilon times the sign of the gradient of the loss with respect to x

Schematic, one frame of six with an identical layout. Left to right along the top: the petunia photograph labeled image x, two conv blocks marked with the weights theta in magenta, a small stack of three feature maps labeled A superscript l, a third conv block, a stack of four feature maps labeled A superscript L, channels c, a column of five class scores labeled z, and at the right a red-outlined box labeled objective at the top. Gray arrows carry the forward pass left to right. A red gradient arrow drops from the objective box and runs right to left along the bottom of the network. Adversarial example. Objective: loss l of z and y, go UP. Tags: image x changes; weights theta fixed. The red arrow runs all the way back to the photograph and ends at a stop bar under it. Result below the photograph: the same petunia, which looks unchanged, beside x gets x plus epsilon times the sign of the gradient of the loss with respect to x; change the image a little: one step.

Open full-size figure

Return to contents

Slide 7

Adversarial examples

Goodfellow et al. 2015, Figure 1: a photograph of a panda, classified panda with 57.7 percent confidence, plus 0.007 times a colorful noise pattern, the sign of the gradient of the loss with respect to the pixels, equals an image that looks identical to the panda but is classified gibbon with 99.3 percent confidence

Goodfellow et al. 2015, Figure 1: a photograph of a panda, classified panda with 57.7 percent confidence, plus 0.007 times a colorful noise pattern, the sign of the gradient of the loss with respect to the pixels, equals an image that looks identical to the panda but is classified gibbon with 99.3 percent confidence.

Open full-size figure

Return to contents

Slide 8

Adversarial training: learn from the worst case

min⁡θ    average over images⏟training    max⁡∥δ∥≤ε  ℓ(f(x+δ; θ), y)⏟attack\min_{\textcolor{#B5396B}{\boldsymbol{\theta}}}\;\;\underbrace{\text{average over images}}_{\text{training}}\;\;\underbrace{\max_{\|\boldsymbol{\delta}\|\le\varepsilon}\;\ell\big(f(\textcolor{#3E6D8E}{\boldsymbol{x}}+\boldsymbol{\delta};\,\textcolor{#B5396B}{\boldsymbol{\theta}}),\,y\big)}_{\text{attack}}

Return to contents

Slide 9

Try it: attack the flower network

Return to contents

Slide 10

Attribution maps show which pixels a decision used

Occlusion, saliency and two fixes for it, Grad-CAM, and how to test a map

Return to contents

Slide 11

Occlusion: cover a region and watch the score

Three pairs side by side, each a test photograph with a gray square over part of it and, to its right, a map of the probability of the correct class as the square moves: a Pomeranian dog, low only where the square covers the dog's face; a racing car's wheel, low over the wheel; a couple with an Afghan hound, low over the dog and not over the people's faces

Zeiler and Fergus (2014), Figure 7, three pairs side by side. Each pair is a test photograph with a gray square drawn on it, and to its right a map of the probability of the correct class as the square is moved over every position: red is high, blue is low. Pair 1, true label Pomeranian: a fluffy dog holding a green toy; the map is red everywhere except a blue patch where the square covers the dog's face. Pair 2, true label car wheel: the side of a racing car; the map is blue over the wheel on the right. Pair 3, true label Afghan hound: a man and a woman with a long-haired dog; the map is blue over the dog and red over the man's face. The point: covering the object, not its surroundings, lowers the network's probability.

Open full-size figure

Return to contents

Slide 12

Try it: cover a region, watch five scores

Return to contents

Slide 13

Saliency: the gradient at the pixels

The petunia photograph, left, and its saliency map for the class morning glory, right: a speckle of bright pixels spread over the whole flower, densest at the dark center

Two panels of equal size. Left: a Flowers102 test photograph of a magenta petunia, which the course's flower network (ResNet-50 with a linear readout) calls morning glory with probability 0.63. Right: the saliency map for morning glory over a gray copy of the photograph: bright speckle over the whole flower, densest at the dark center and along petal edges.

Open full-size figure

Computed · a petunia the network calls morning glory (p = 0.63)

𝐬  =  ∣ ∂zc∂𝐱 ∣\textcolor{#B23A2E}{\mathbf{s}} \;=\; \left|\,\frac{\partial \textcolor{#3F7A6B}{z_c}}{\partial \textcolor{#3E6D8E}{\mathbf{x}}}\,\right|

  • Which pixels, if changed a little, would change the logit zcz_c the most?
  • zc\textcolor{#3F7A6B}{z_c}: the logit (score before the softmax) of the predicted class; 𝐱\textcolor{#3E6D8E}{\mathbf{x}}: the image
  • 𝐬\textcolor{#B23A2E}{\mathbf{s}}: the saliency map, one value per pixel
  • We use the predicted class, to explain what the network did. When the network is wrong, compare with the map for the correct class

Return to contents

Slide 14

Saliency: backpropagate the logit to the pixels

A schematic network, left to right: the petunia photograph, image x; conv blocks with weights theta; feature maps A of layer l; more conv; the last feature maps A of layer L, channels k; class scores z; and at the right, the objective box. Gray arrows carry the forward pass; a red arrow carries the gradient backward. Here the objective is the class score z_c, one cell of z filled red; weights fixed; the red arrow runs back to the pixels; below, the petunia's saliency map, the absolute value of the partial derivative of z_c with respect to x, read the slope at each pixel, no step

Schematic, one frame of six with an identical layout. Left to right along the top: the petunia photograph labeled image x, two conv blocks marked with the weights theta in magenta, a small stack of three feature maps labeled A superscript l, a third conv block, a stack of four feature maps labeled A superscript L, channels c, a column of five class scores labeled z, and at the right a red-outlined box labeled objective at the top. Gray arrows carry the forward pass left to right. A red gradient arrow drops from the objective box and runs right to left along the bottom of the network. Saliency map. Objective: class score z_c, with one class-score cell filled red and labeled z_c. Tags: image x fixed; weights theta fixed. The red arrow runs back to the pixels. Result below the photograph: the saliency map of the petunia, speckled bright dots over a darkened flower, beside the absolute value of the partial derivative of z_c with respect to x; read the slope at each pixel, no step.

Open full-size figure

Return to contents

Slide 15

The chain rule reaches the pixels in one more link

∂zc∂wjk  =  δj xk⏟training: error×input∂zc∂xk  =  ∑jδj wjk⏟saliency: error×weight, summed over the units pixel k feeds\underbrace{\frac{\partial z_c}{\partial \textcolor{#B5396B}{w}_{jk}} \;=\; \delta_j\,\textcolor{#3E6D8E}{x}_k}_{\text{training: error} \times \text{input}} \qquad\qquad\qquad \underbrace{\frac{\partial z_c}{\partial \textcolor{#3E6D8E}{x}_k} \;=\; \sum_j \delta_j\,\textcolor{#B5396B}{w}_{jk}}_{\text{saliency: error} \times \text{weight, summed over the units pixel } k \text{ feeds}}

Return to contents

Slide 16

Invisible noise changes the saliency map

Three panels. First, the petunia's saliency map for morning glory, logit 10.69. Second, the saliency map after adding a fixed noise pattern of about 2.6 gray levels to every pixel, logit 11.26: the speckle has rearranged, with bright clusters in new places. Third, the correlation between the map of the noisy image and the photograph's map, against the size of the noise from 0 to 5 gray levels: it falls from 1 to 0.38 at 2.6 gray levels, marked, and to about 0.2 at 5

Three panels. First, the petunia's saliency map for morning glory, logit 10.69. Second, the saliency map after adding a fixed noise pattern of about 2.6 gray levels to every pixel, logit 11.26: the speckle has rearranged, with bright clusters in new places. Third, the correlation between the map of the noisy image and the photograph's map, against the size of the noise from 0 to 5 gray levels: it falls from 1 to 0.38 at 2.6 gray levels, marked, and to about 0.2 at 5

Open full-size figure

Return to contents

Slide 17

SmoothGrad: average the maps of many noisy copies

The petunia photograph, left, and its SmoothGrad map for morning glory, right: smoother than the saliency map, with the dark center and the petal edges standing out

Two panels of equal size. Left: the same petunia photograph. Right: the SmoothGrad map for morning glory, the average of 50 saliency maps of noisy copies: smoother than the plain saliency map, with the dark center and the petal edges standing out and less speckle on the background.

Open full-size figure

Computed · 50 noisy copies · same network and photograph

𝐬ˉ  =  1n∑i=1n∣ ∂zc∂(𝐱+ϵi) ∣\bar{\mathbf{s}} \;=\; \frac{1}{n}\sum_{i=1}^{n}\left|\,\frac{\partial \textcolor{#3F7A6B}{z_c}}{\partial (\textcolor{#3E6D8E}{\mathbf{x}}+\boldsymbol{\epsilon}_i)}\,\right|

  • SmoothGrad makes 50 noisy copies of the photograph, each with its own noise ϵi\boldsymbol{\epsilon}_i, and averages their saliency maps
  • The jumps differ from one copy to the next, so they average out. What remains is the part of the map that all the copies share
  • You will compute saliency and SmoothGrad maps in the assignment

Return to contents

Slide 18

Saturation: the slope is flat where the detail mattered

The logit for morning glory along a path of images from a blurred copy of the petunia, left, to the photograph, right, with both images shown at the bottom. The logit rises steeply from 2.8 over the first third of the path to a peak near 11.5 at the middle, then declines slightly to 10.69 at the photograph, where a dashed tangent is flat: slope at the photograph 0.07. A double arrow marks the rise of 7.9 between the blurred copy and the photograph

The logit for morning glory along a path of images from a blurred copy of the petunia, left, to the photograph, right, with both images shown at the bottom. The logit rises steeply from 2.8 over the first third of the path to a peak near 11.5 at the middle, then declines slightly to 10.69 at the photograph, where a dashed tangent is flat: slope at the photograph 0.07. A double arrow marks the rise of 7.9 between the blurred copy and the photograph

Open full-size figure

Computed · same network and photograph · blurred copy: Gaussian blur, sd 10 pixels

  • We go step by step from a blurred copy of the photograph to the photograph itself. The logit rises from 2.8 to about 11 in the first third of the way, then stays about flat
  • At the photograph, the slope is only 0.07. The gradient says the detail does not matter, yet adding it raised the logit by 7.9
  • This is saturation. Once the network has enough evidence, more evidence adds nothing, just as a neuron firing at its maximum rate cannot respond more to a stronger stimulus

Return to contents

Slide 19

Integrated gradients: add up the slopes along the path

The petunia photograph, left, and its integrated-gradients map for morning glory, right: credit concentrated on the dark center and along the petal veins and edges, with less speckle than saliency

Two panels of equal size. Left: the petunia photograph. Right: the integrated-gradients map for morning glory, computed from a blurred-copy baseline in 32 steps: credit concentrated on the dark center and along the petal veins and edges, with less speckle than the plain saliency map.

Open full-size figure

Computed · baseline: a blurred copy · 32 steps

IG  =  (𝐱−𝐱0) ⊙ 1m∑j=1m∂zc∂𝐱∣𝐱0+jm(𝐱−𝐱0)\textcolor{#B23A2E}{\mathrm{IG}} \;=\; (\textcolor{#3E6D8E}{\mathbf{x}}-\mathbf{x}_0)\,\odot\,\frac{1}{m}\sum_{j=1}^{m}\frac{\partial \textcolor{#3F7A6B}{z_c}}{\partial \mathbf{x}}\Big|_{\mathbf{x}_0+\frac{j}{m}(\textcolor{#3E6D8E}{\mathbf{x}}-\mathbf{x}_0)}

  • Integrated gradients computes the gradient at mm images along the path from a baseline 𝐱0\mathbf{x}_0 to the photograph. It multiplies each pixel's average slope by how far that pixel moved, 𝐱−𝐱0\mathbf{x}-\mathbf{x}_0. Slope times distance is that pixel's share of the rise
  • Now the steep part of the path counts. The credits of all pixels add up to the whole rise of the logit, 7.9 here
  • You have to choose the baseline. We use a blurred copy here, and the assignment's bonus uses a black image

Return to contents

Slide 20

Grad-CAM: stop at the last feature maps

A schematic network, left to right: the petunia photograph, image x; conv blocks with weights theta; feature maps A of layer l; more conv; the last feature maps A of layer L, channels k; class scores z; and at the right, the objective box. Gray arrows carry the forward pass; a red arrow carries the gradient backward. Here the objective is the class score z_c; the red arrow stops at the last feature maps A of layer L, where each channel k carries a weight alpha k, for example 0.8, 0.1, minus 0.3, 0.5; below, alpha k is the average over positions of the partial derivative of the class score z_c with respect to channel k, and the map for class c is ReLU of the sum over k of alpha k times channel k, drawn as a coarse heat map over the petunia, brightest at its center

Schematic, one frame of six with an identical layout. Left to right along the top: the petunia photograph labeled image x, two conv blocks marked with the weights theta in magenta, a small stack of three feature maps labeled A superscript l, a third conv block, a stack of four feature maps labeled A superscript L, channels c, a column of five class scores labeled z, and at the right a red-outlined box labeled objective at the top. Gray arrows carry the forward pass left to right. A red gradient arrow drops from the objective box and runs right to left along the bottom of the network. Grad-CAM. Objective: class score z_c, one class-score cell red. Tags: image and weights fixed. The red arrow stops at the last feature maps A superscript L, marked by a stop bar under that stack; each of the four channels carries a red number, its weight alpha_c: 0.8, 0.1, minus 0.3, 0.5. Below: alpha_c equals the average over positions of the partial derivative of z_c with respect to channel c of A superscript L; map equals ReLU of the sum over c of alpha_c times that channel. Beside the formulas, a coarse 7 by 7 heat map over the petunia, brightest at the flower's center.

Open full-size figure

Return to contents

Slide 21

Remove a region and run the network again

Five versions of the petunia photograph, each with the probability of morning glory: the photograph, 0.63; a gray square over the dark center, 0.15; the same square inpainted, filled with smooth pink continued from its border, 0.00; the brightest region of the SmoothGrad map, an irregular patch over the center and the petal bases, filled gray, 0.05; the same region inpainted, 0.00. Each removed region is outlined with a dashed line

Five versions of the petunia photograph, each with the probability of morning glory: the photograph, 0.63; a gray square over the dark center, 0.15; the same square inpainted, filled with smooth pink continued from its border, 0.00; the brightest region of the SmoothGrad map, an irregular patch over the center and the petal bases, filled gray, 0.05; the same region inpainted, 0.00. Each removed region is outlined with a dashed line

Open full-size figure

Return to contents

Slide 22

Test the map against the same shape elsewhere

The petunia photograph, left, with the brightest region of its SmoothGrad map outlined: the dark center and the petal bases above it; right, the same photograph with that region inpainted, filled with smooth pink

Two panels of equal size. Left: the petunia photograph with a solid outline around the brightest region of its SmoothGrad map for morning glory: the dark center and the bases of the petals above it, about 7% of the image. Right: the same photograph after that region is inpainted, filled with colors continued smoothly from its border, so the dark center becomes a smooth pink patch outlined with a dashed line.

Open full-size figure

Computed · region: the brightest 6% of the SmoothGrad map, grown to 7%

Dot plot of the probability of morning glory: 0.00 after inpainting the map's region, against 30 same-shape controls between 0.50 and 0.82, around the original 0.63

Dot plot. Horizontal axis: probability of morning glory from 0 to 1; a dotted vertical line marks the original 0.63. Top row: 30 gray dots, one per control in which a region of the same shape was inpainted at a random place that misses the map's region, spread from 0.50 to 0.82. Bottom row: one blue dot at 0.00 for the inpainted map region.

Open full-size figure
  • If the map is right, removing its region should lower the score more than removing the same shape anywhere else
  • Inpainting the map's region drops the probability from 0.63 to 0.00. As a control, we inpaint the same shape at 30 places that miss the region. The probability stays between 0.50 and 0.82. Only the region the map picked matters
  • This test works for any map. In the assignment, you will measure the share of a map's top pixels that fall on the flower

Return to contents

Slide 23

Test every map: delete pixels in its order

Left, the petunia photograph labelled true petunia, predicted morning glory, p = 0.633. Right, deletion curves: the probability of morning glory against the fraction of pixels replaced by a blurred copy, pixels removed in the order each map ranks them, 10 steps. Occlusion falls fastest, area 0.04, then Grad-CAM 0.08, saliency 0.18 and integrated gradients 0.21; the dashed random map stays high for most of the way, area 0.58

Left, the petunia photograph labelled true petunia, predicted morning glory, p = 0.633. Right, deletion curves: the probability of morning glory against the fraction of pixels replaced by a blurred copy, pixels removed in the order each map ranks them, 10 steps. Occlusion falls fastest, area 0.04, then Grad-CAM 0.08, saliency 0.18 and integrated gradients 0.21; the dashed random map stays high for most of the way, area 0.58

Open full-size figure

Return to contents

Slide 24

What does a single unit respond to?

Find the image that drives one unit hardest

Return to contents

Slide 25

Feature visualization: one channel at the top

A schematic network, left to right: the petunia photograph, image x; conv blocks with weights theta; feature maps A of layer l; more conv; the last feature maps A of layer L, channels k; class scores z; and at the right, the objective box. Gray arrows carry the forward pass; a red arrow carries the gradient backward. Here the objective box is wired to the feature maps A of layer l, and everything after them is grayed out; one channel k is outlined red; objective: one channel's mean activation, one over H_l W_l times the sum over i, j of A l, i, j, k; weights fixed; the red arrow runs back to the pixels; below, a real feature visualization of a ResNet-50 channel, and x gets x plus eta times the gradient, many steps, start from noise

Schematic, one frame of six with an identical layout. Left to right along the top: the petunia photograph labeled image x, two conv blocks marked with the weights theta in magenta, a small stack of three feature maps labeled A superscript l, a third conv block, a stack of four feature maps labeled A superscript L, channels c, a column of five class scores labeled z, and at the right a red-outlined box labeled objective at the top. Gray arrows carry the forward pass left to right. A red gradient arrow drops from the objective box and runs right to left along the bottom of the network. Feature visualization. The objective now attaches to the intermediate stack A superscript l: a dark line rises from that stack, runs over the top of the canvas and drops into the objective box, and everything after A superscript l (third conv block, A superscript L, z) is grayed out. The front channel of A superscript l is outlined red and labeled c. Objective: one channel's mean activation, 1 over H_l W_l times the sum over i, j of A superscript l at i, j, c. Tags: image x changes; weights fixed. The red arrow runs back to the pixels. Result below the photograph: a feature-visualization image from a ResNet-50 channel, a colorful repeating texture, beside x gets x plus eta times the gradient with respect to x; many steps, start from noise.

Open full-size figure

Return to contents

Slide 26

Feature visualization: optimize the image

We keep the weights fixed and change the image until one channel kk of layer ll, averaged over its Hl×WlH_l \times W_l positions, responds as strongly as possible:

𝐱⋆  =  arg⁡max⁡𝐱  1HlWl∑i,jAi,j,k[l](𝐱)  −  penalty(𝐱)\textcolor{#3E6D8E}{\mathbf{x}}^\star \;=\; \arg\max_{\textcolor{#3E6D8E}{\mathbf{x}}}\; \textcolor{#3F7A6B}{\frac{1}{H_l W_l}\sum_{i,j} A^{[l]}_{i,j,k}(\mathbf{x})} \;-\; \text{penalty}(\textcolor{#3E6D8E}{\mathbf{x}})

Return to contents

Slide 27

AlexNet: oriented edges, then textures, then object parts

AlexNet feature visualizations in three groups of four. First layer, the filters: oriented stripes and color blobs. Middle layer, conv3: repeating textures of dots and cells. Last layer, conv5: object-like parts, a coral-like surface with holes, rows of eye-like spots, shells

AlexNet feature visualizations in three groups of four. First layer, the filters: oriented stripes and color blobs. Middle layer, conv3: repeating textures of dots and cells. Last layer, conv5: object-like parts, a coral-like surface with holes, rows of eye-like spots, shells

Open full-size figure

Return to contents

Slide 28

ResNet-50: edges, textures, then sharper parts

ResNet-50 feature visualizations in three groups of four. First layer, the 7 by 7 filters: color-opponent blobs and an oriented blue-orange edge. Middle, layer3: textures and arch-like repeated shapes. Last, layer4: screws and bolts, building facades, traffic seen from above

ResNet-50 feature visualizations in three groups of four. First layer, the 7 by 7 filters: color-opponent blobs and an oriented blue-orange edge. Middle, layer3: textures and arch-like repeated shapes. Last, layer4: screws and bolts, building facades, traffic seen from above

Open full-size figure

Return to contents

Slide 29

ViT-B/16: no oriented filters in the first layer

ViT-B/16 feature visualizations in three groups of four. First layer, the 16 by 16 patch filters: noisy, colored, with no oriented stripes. Middle, block 5: repeating scale and wave textures. Last, block 11: diffuse, speckled patterns with small dark marks, no clear object parts

ViT-B/16 feature visualizations in three groups of four. First layer, the 16 by 16 patch filters: noisy, colored, with no oriented stripes. Middle, block 5: repeating scale and wave textures. Last, block 11: diffuse, speckled patterns with small dark marks, no clear object parts

Open full-size figure

Return to contents

Slide 30

DINOv2, trained without labels: textures and parts too

DINOv2 ViT-B/14 feature visualizations in three groups of four. First layer, the 14 by 14 patch filters: noisy, no oriented stripes. Middle, block 5: woven, ridged and pebbled textures. Last, block 11: dense surfaces with eye-like spots, fur-like and ridged textures

DINOv2 ViT-B/14 feature visualizations in three groups of four. First layer, the 14 by 14 patch filters: noisy, no oriented stripes. Middle, block 5: woven, ridged and pebbled textures. Last, block 11: dense surfaces with eye-like spots, fur-like and ridged textures

Open full-size figure

Return to contents

Slide 31

ResNet-50's last block: MEIs and top photographs

Three ResNet-50 layer4 units, one per row: the unit's MaCo feature visualization, then its six most strongly activating ImageNet validation images. Unit 43: a helmet, a triceratops skull, a thatched roof, a burrito, a thatched cottage, axes. Unit 179: a bell, screws, a perforated bone, a bolt, a birdhouse, a shark. Unit 309: sleeping bags, volcanoes and mountain peaks, a tent, a yurt

Three ResNet-50 layer4 units, one per row. In each row, the unit's MACO feature visualization on the left, labeled with its number, then the six ImageNet validation images that drive it most strongly. Unit 43: a bronze helmet, a triceratops skull, a thatched roof, a burrito, a thatched cottage, and two axes on grass. Unit 179: a bell, small metal screws, a perforated bone, a bolt and a screw, a birdhouse, and a shark. Unit 309: piled sleeping bags, a volcano above a town, a volcano over a lake, a volcano in cloud, a tent below snowy peaks, and a yurt. The point: unit 309 responds to conical shapes, while unit 179 has no single label.

Open full-size figure

Return to contents

Slide 32

A shortcut: "tench" includes the fingers holding it

Two concepts of ResNet-50 for the class tench, from ObjectLENS. Left, concept 2: a synthesized image of fish bodies with pink fingers wrapped around them, beside three ImageNet crops that drive it, each a hand gripping a fish. Right, concept 6: a synthesized image of fish heads, beside three crops of tench heads and faces

Two concepts of ResNet-50 for the class tench, from ObjectLENS. Left, concept 2: a synthesized image of fish bodies with pink fingers wrapped around them, beside three ImageNet crops that drive it, each a hand gripping a fish. Right, concept 6: a synthesized image of fish heads, beside three crops of tench heads and faces

Open full-size figure

Return to contents

Slide 33

A Spider-Man neuron, after the Halle Berry neuron

Seven image tiles that drive one unit of CLIP: photographs of two people in Spider-Man costumes and of a real spider in its web; a pencil drawing of the mask, a Spider-Man sticker and a drawing of a spider; and a sign reading The Amazing Spider-Man

Seven image tiles that drive one unit of CLIP: photographs of two people in Spider-Man costumes and of a real spider in its web; a pencil drawing of the mask, a Spider-Man sticker and a drawing of a spider; and a sign reading The Amazing Spider-Man

Open full-size figure

Return to contents

Slide 34

A Golden Gate Bridge feature in a language model

Example inputs that activate the Golden Gate Bridge feature of Claude 3 Sonnet, from Templeton et al. (2024): three photographs of the bridge, at sunset, from below and with the Space Shuttle flying over it, and four lines of text in which the words Golden Gate Bridge are highlighted

Example inputs that activate the Golden Gate Bridge feature of Claude 3 Sonnet, from Templeton et al. (2024): three photographs of the bridge, at sunset, from below and with the Space Shuttle flying over it, and four lines of text in which the words Golden Gate Bridge are highlighted

Open full-size figure
  • Templeton et al. (2024) found millions of features in the language model Claude 3 Sonnet. Each feature is a direction in the model's activity. One responds to the Golden Gate Bridge in photographs, in English and in other languages
  • If you turn that feature up, the model brings the bridge into every answer. In any network, we can ask what drives a unit and what happens when we drive it
  • Try at home: browse the features of open language models at neuronpedia.org

Return to contents

Slide 35

Is a network a good model of the ventral stream?

Score a network's blocks against V1, V4 and IT recordings, with tools you already have

Return to contents

Slide 36

Scoring a network against the brain: tools you know

Sixteen photographs of everyday objects from the THINGS database, each labeled with its concept, alligator to boat in alphabetical order, for example an axe in snow, a banana, a beaver and a sailing boat

Sixteen photographs from the THINGS database in two rows of eight, each with its concept name below: alligator, altar, ashtray, axe, bamboo, banana, beachball, bean; then beaver, bed, beer, bench, bike, blind, boa, boat. They are ordinary photographs of single objects or materials, some on plain backgrounds (an ashtray and a beach ball on gray) and some in natural settings (an alligator on rocks, a sailboat on a beach at low tide, pink and white feather boas). These are 16 of the 100 TVSD test photographs (Papale et al., 2025): the images each shown 30 times, on which the encoding models are scored.

Open full-size figure

TVSD (Papale et al. 2025): 16 of the 100 test photographs

  • A readout is a ridge regression from a block's units to one site's responses, fit on training photographs and scored on new ones
  • The noise ceiling divides each site's score by how well the site predicts itself across repeats, so 1 is the best possible
  • A block's linear predictivity is the median score over an area's sites. RSA compares dissimilarity matrices, with no weights to fit
  • Data, as in the assignment: TVSD, about 22,000 THINGS photographs, sites in V1, V4 and IT of two monkeys

Return to contents

Slide 37

Later blocks predict later areas

Median noise-corrected correlation of ridge encoding models against the 16 blocks of an ImageNet ResNet-50, for TVSD V1 in blue, V4 in green and IT in magenta; monkey F solid, monkey N dashed; light bootstrap bands; a dot marks each curve's peak. V4 peaks at layer3.3 in both monkeys, about 0.68 and 0.71; IT peaks later, at layer4.0 in F, 0.76, and layer3.4 in N, 0.81; V1 stays near 0.64 and peaks at different blocks in the two monkeys; every curve falls in the last blocks

Computed: ImageNet-trained ResNet-50 on the TVSD test photographs (Papale et al., 2025). Line plot. Horizontal axis: the 16 blocks of ResNet-50 in order, grouped as layer1 (blocks 0 to 2), layer2 (0 to 3), layer3 (0 to 5) and layer4 (0 to 2). Vertical axis: median noise-corrected correlation over electrodes, 0 to 0.8. Color gives the area, V1 blue, V4 green, IT magenta; line style gives the monkey, F solid, N dashed; shaded bands are 95% bootstrap intervals; a dot marks each curve's peak. V1 starts highest, near 0.64, and stays nearly flat until layer4. IT starts lowest, near 0.5, and rises through layer2 and layer3. V4 peaks at layer3 block 3 in both monkeys, about 0.68 in F and 0.71 in N. IT peaks later and higher: 0.76 at layer4 block 0 in F, 0.81 at layer3 block 4 in N. Every curve drops in the last blocks of layer4. Later areas are best predicted by deeper blocks, V4 before IT.

Open full-size figure

Return to contents

Slide 38

Better AI has not meant better models of the brain

Past human-level accuracy, better ImageNet networks are not better models of IT, and the recordings disagree about what is

Return to contents

Slide 39

Past human-level accuracy, the trend reverses

Scatter of neural alignment, 0.2 to 1.0, against ImageNet multi-label accuracy, 0.5 to 1.0, for several hundred models colored by family: CNNs trained on ImageNet-1K in red, adversarially trained CNNs in yellow, convolutional vision transformers in green, vision transformers in blue, with internet-scale and vision-and-language variants in lighter shades; a gray band marks human accuracy near 0.93 to 0.97. Two dashed lines: alignment rises with accuracy up to about 0.9, then falls; purple stars mark recent models SigLIP-2, DINOv3 ViT-L and ViT-H, I-JEPA, vJEPA2 and EVA-02, the most accurate of which sit lowest

From Linsley, Feng and Serre (2026). Scatter plot of neural alignment, how well a network's features predict monkey IT responses (vertical, 0.2 to 1.0), against ImageNet multi-label accuracy (horizontal, about 0.45 to 1.0), several hundred networks drawn as colored squares. Legend: CNNs trained on ImageNet-1K red, on internet-scale images pink, adversarially trained yellow; convolutional vision transformers (ConvViT) on ImageNet-1K dark green, internet-scale light green; vision transformers (ViT) on ImageNet-1K blue, internet-scale light blue, internet-scale vision and language gray. A gray vertical band labeled Human accuracy covers about 0.92 to 0.97. Two dashed lines meet near the band: one rising from about 0.5 at the left to about 0.7, one falling from there to about 0.6 at accuracy 1.0. Yellow adversarial CNNs sit at the left, accuracy 0.48 to 0.7; most red and blue networks cluster at 0.75 to 0.97 accuracy and 0.45 to 0.7 alignment. Purple stars mark recent models: SigLIP-2 and DINOv3 ViT-L near 0.65, DINOv3 ViT-H near 0.56, I-JEPA near 0.49, EVA-02 near 0.48 at the highest accuracy, and vJEPA2 lowest, near 0.29. Alignment rises with accuracy up to about human accuracy, then falls.

Open full-size figure

Return to contents

Slide 40

Four datasets, one cohort: the picture in 2026

Four scatter plots of alignment score against ImageNet multilabel accuracy for 363 classifiers, CNNs as blue dots, transformers orange, self-supervised and image-text models as open circles, adversarially trained models as triangles, each with a dashed noise ceiling. Arcaro IT: rank correlation plus 0.37 overall, minus 0.38 among the 84 models above 92.7 percent. Triple-N IT: minus 0.08 overall, plus 0.20 at the top. TVSD IT: plus 0.73 overall, plus 0.65 at the top. Human feature use: minus 0.77 overall, minus 0.37 at the top

Four scatter plots from Serre 2026, Figure 4, of alignment score against ImageNet multilabel accuracy, 50 to 100 percent, for 363 image classifiers: CNNs as blue dots, transformers as orange dots, self-supervised and image-text models as open circles, adversarially trained models as triangles; a dashed line marks each dataset's ceiling. Panel a, Arcaro IT: rank correlation plus 0.37 over all models, minus 0.38 among the 84 above 92.7 percent. Panel b, Triple-N IT: minus 0.08 and plus 0.20. Panel c, TVSD IT: plus 0.73 and plus 0.65. Panel d, human feature use: minus 0.77 and minus 0.37; two harmonized models, VGG-16 and ViT-B/16, sit well above the cloud.

Open full-size figure

Reproduced from Serre, Annu Rev Vis Sci (in submission), Fig. 4 (draft); Triple-N and TVSD scores computed for the review

  • 363 networks, four targets. For IT, we use each network's best-fitting layer. Human feature use is the agreement with the regions people paint as important in ClickMe, a web game
  • Arcaro IT and ClickMe turn down past about 93% accuracy. TVSD still rises (rank correlation ρ = +0.73). Triple-N, a newer IT dataset, is flat (ρ = −0.08)
  • The answer depends on which recordings you use. To discuss: if higher accuracy no longer gives a better fit to the brain, what would?
Serre Annu Rev Vis Sci (in submission) · Linsley, Feng & Serre Trends Cogn Sci 2026 · Li et al. Nat Neurosci 2026

Return to contents

Slide 41

Final projects: ask the models

Serre Annu Rev Vis Sci (in submission) · Golan, Raju & Kriegeskorte PNAS 2020 · Fel et al. NeurIPS 2022

Return to contents

Slide 42

Recap

Return to contents

Slide 43

Further reading for the research trail

Optional: a few entries over the semester; any paper from any lecture qualifies.

Return to contents

Slide 44

Minute paper

Submit on Canvas → Minute papers → Minute Paper 14 (access code read out in class).

The minute paper asks you to recall, in your own words, what you just heard. Retrieving it is what strengthens the memory; copying a slide does not. It is not a quiz.

A trend I have noticed: papers that discuss slides I never reached, and papers written while I was still lecturing. Both miss the point. From today, a paper about material not covered in class, or written before the class is over, loses credit.

Write three brief points:

  1. One idea from today, restated in your own words
  2. Something you do not yet understand, or a question still open
  3. Another idea you found interesting, and why it matters for brains, behavior or AI

Return to contents

Slide 45

Going further

Return to contents

Slide 46

Decoding: a readout from brain to stimulus

Eight polar plots, one per grating orientation shown (horizontal, then in steps of 22.5 degrees). In each, a black line marks the orientation shown and a gray outline shows how often the readout chose each orientation; every outline is elongated along the black line. Label: participant S2, V1 and V2, 400 voxels

Reproduced from Kamitani and Tong (2005), Figure 2a. Eight polar plots in two rows of four, one per grating orientation shown: horizontal, then rotating in steps of 22.5 degrees through vertical to 157.5 degrees. A small black-and-white grating at each plot's top left shows the orientation. Dashed radial lines mark the eight possible orientations; a black line marks the orientation shown; a gray outline shows how often a readout of the fMRI responses chose each orientation, reaching farther along orientations chosen more often. In every plot the outline is elongated along the black line: the readout mostly chose the orientation shown. A label reads S2, V1/V2, 400 voxels. Orientation can be read out from the pattern across many voxels, although each voxel is only weakly tuned.

Open full-size figure

Reproduced from Kamitani & Tong (2005), Fig. 2a

Return to contents

Slide 47

Decoding dreams

Return to contents