=========================================================================== CPSY 1291 — RECITATION 7: Exam 1 review (Lectures 1-8, and backprop from 9) TA-led, 80 minutes. Optional. Week 7 — MUST be held before Thu 10/22. SCOPE: Exam 1 covers L01-L08 plus the four steps of backpropagation from L09 (forward pass, output error, error sent backward, gradient = error x input). Nothing else from L09 (softmax, the training loop, autograd, transfer learning) and nothing from L10 onward. FORMAT: this is a PROBLEM SESSION, not a lecture. The slides are prompts. Put a problem up, give the room 3-4 minutes, take answers from students before revealing. Resist the urge to talk through the summary slides — they exist so the room can check itself, and the handout carries the full version. BUDGET: about 15 min on the framing and the three summary slides (slides 2-5), about 49 min on problems, 4 min on the cold list, and 7 min of open questions at the end. If you fall behind, cut problems 5 and 6 — not the backprop problem and not the open-question block, which is what the students who came actually came for. DO NOT: predict what is on the exam, or rank topics by likelihood. Say you do not know. Point at the learning objectives in the syllabus instead. MATERIAL: handout-07-midterm-review.pdf has the full compressed summary of L01-L08, the four backprop steps, and sixteen problems with worked answers. Tell them at the start that it exists so nobody transcribes the board. ============================================================================
2 min. Say the format immediately: this is a problem session. You will put questions up and the room answers them. Fifteen minutes of summary, then an hour of problems. Say the scope once, precisely: Lectures 1-8, plus the four steps of backpropagation from Lecture 9. Nothing after that. Say the thing about the handout: it has the full compressed summary and sixteen worked problems, so nobody needs to transcribe anything. If asked what is on the exam: you do not know. Point at the syllabus learning objectives. Do not speculate — it is unfair to whoever is not in the room.
3 min. This is the most useful framing you can give them, so give it first. Illustrate with one example they know: PCA. The object is the eigendecomposition of the covariance. What it is for is finding directions of greatest variance. When it misleads you: it is linear, it depends on the units of each feature, and the direction with most variance need not be the one that separates the groups. All three are examinable, and only the first is what students revise.
5 min. Read the bullets, do not expand them. This slide is a checklist for the room to audit itself against — anyone who cannot expand a bullet knows what to revise tonight. Ask for a show of hands on which bullet is least familiar, and spend two minutes on whatever wins. Do not spend six.
5 min. Same treatment. The two most misunderstood items on this slide are "PR counts how evenly the variance is spread, not how much information there is" and "high shattering and high CCGP pull against each other" — flag both explicitly, because both are natural exam questions and both are usually answered wrong.
4 min. Say the scope again: the four steps, on a network small enough to do by hand. Not the softmax, not the training loop, not autograd. Remind them that with a sigmoid output and the cross-entropy, the output error is simply prediction minus target: the sigmoid's slope cancels the loss's denominator, as in logistic regression. Problem 10 is this calculation on a new input.
4 min. Answers: both are 0. Cosine because the vectors point in exactly the same direction. Correlation because after centering each row both become proportional to (-1, 0, 1). Part 3 is the real question: these units differ only in gain. Cosine discards gain, correlation discards gain and offset, so neither can tell them apart; Euclidean keeps both and can. Which one you want is a modelling decision: choose by what should not count. Follow-up if the room is fast: what if the second unit were (3, 2, 1)?
4 min. Answers: the triangle inequality, since 0.9 > 0.4. Cosine distance and correlation distance both permit it; they also fail identity (a brighter copy of an image sits at cosine distance 0 from it). Euclidean distance satisfies all four requirements. Push: is that a defect? Only if you needed the requirement. MDS assumes the matrix is close to a table of distances between points, which is why it matters which measure built the RDM.
4 min. Answers: (1) the MDS solution is not unique — shift, rotate or reflect it and every distance, and the stress, is unchanged — so "the horizontal axis" is not a property of the data. Only relative positions mean anything. (2) Independent evidence: correlate a direction in the map with an external animacy rating, or train a readout for animacy and test it on held-out stimuli. The lecture's line is worth quoting: if a story about your map depends on which way is "up", it is a story about the plot.
5 min. Answers: (1) rank 2; PR = (4+4)^2 / (16+16) = 64/32 = 2. (2) rank 2; PR = 8^2 / (49+1) = 64/50 = 1.28. (3) Both populations use two directions, but in the second one direction carries almost all the variance. PR counts how evenly the variance is spread: if k directions share it equally, PR = k. It does not measure how much information the population carries. Follow-up: why not just report the rank of a real recording? Because noise makes every real recording full rank (480 IT neurons: rank 480).
5 min. Answers: (1) neither; the random start is set by the seed, and another seed gives another map. (2) Claim only what holds across seeds and across a perplexity sweep. Sizes and gaps in a t-SNE map are not distances in the data, so "far apart" and "tight" are not findings. To check a grouping, go back to the response space: are an image's nearest neighbors there the same images? Does a readout trained on some images separate the groups on others? Land the lecture's distinction: a map can be stable (same neighbors run to run) and still unfaithful (different neighbors from the original space).
5 min. Answers: (1) 14. (2) The two XOR splits, which put opposite corners together. (3) A unit whose responses are NOT a weighted sum of the other two: the product x1 x2, or an AND unit. It raises the rank to 3 and all 16 splits become separable. Adding x1 + x2 does nothing: the rank stays 2. Connect it to the recordings: this is why nonlinear mixed selectivity makes a population high-dimensional, and why a high shattering dimensionality means almost any grouping is readable.
5 min. Answers: (1) more neurons than training images: in that many dimensions the points are nearly perpendicular and any labelling can be fit. Getting the training images right is memorizing them. (2) Accuracy on held-out test images, read against a baseline: the shuffled-label score on test images (0.56 in the lecture, chance) and the majority-class baseline. The same holds for deep networks, which memorize ImageNet with shuffled labels (Zhang et al. 2017).
5 min. Answers: (1) A, at 0.30 / 0.35 ≈ 0.86 of its ceiling; B is at 0.50. A raw score cannot be read without its ceiling. (2) The shuffled floor (a permutation test: shuffle one system's image order 1,000 times and beat the 95th percentile) and the same area in another brain. Then ask: how is the ceiling obtained? Split-half reliability: how well two halves of the repeats agree with each other.
5 min. Answers: (1) ŷ = 0.5·2 − 1·1 = 0. Error y − ŷ = 1. Δw = η (y − ŷ) x = 0.1 · 1 · (2, 1) = (0.2, 0.1), so w becomes (0.7, −0.9). (2) ŷ = 1.4 − 0.9 = 0.5: halfway to the target, and the squared-error loss falls from 0.5 to 0.125. Then the conceptual half: the perceptron uses the same update with a thresholded output, so it changes w only on mistakes; logistic regression uses it with σ(z). One recipe: learning rate × error × input.
7 min. Answers, two decimals: Step 1. z1 = 0.50 − 0.40 = 0.10, so a1 = 0.10. z2 = 0.30 + 0.80 = 1.10, so a2 = 1.10. Output z = 0.70·0.10 − 0.60·1.10 = −0.59, ŷ = σ(−0.59) = 0.36. Loss −log(1 − 0.36) = 0.44. Step 2. δ_out = ŷ − y = 0.36: the output was too high. Step 3. δ1 = 0.36 × 0.70 × 1 = 0.25 (unit 1 excites the output: decrease it). δ2 = 0.36 × (−0.60) × 1 = −0.21 (unit 2 inhibits the output: increase it). Step 4. Output weights: 0.36 × 0.10 = 0.04 and 0.36 × 1.10 = 0.39. Unit 1's weights: 0.25 × (1, 1) = (0.25, 0.25). Unit 2's weights: −0.21 × (1, 1) = (−0.21, −0.21). Biases: 0.36 (output), 0.25 and −0.21 (hidden). The point to land: each hidden error is the output error sent back through that unit's outgoing weight, so its sign follows the sign of the weight. One step with η = 1 lowers ŷ to 0.11.
4 min. Read them out. This is the "night before" list and it is in the handout. Each of these has appeared in a lecture and an assignment or a recitation — which is the reason they are on the list, and worth saying, because it tells them the list is not arbitrary.
7 min, and protect it. This is what the people who came actually came for. If nobody speaks, prime the room with a question of your own: "who can tell me why a readout needs held-out images?" — then let the discussion go where it wants. Close by saying that Assignment 3 is released Thu 10/22, the day of the exam, and is due Thu 11/5; and that recitation 8 covers comparing representations — RSA, decoding, and permutation tests.