13 The Elephant-Goldfish Model
These notes are based on Dave Rensin’s essay “Elephants, Goldfish and the New Golden Age of Software Engineering” (Medium, April 2026), adapted for this course. Where you see quoted prompts, they are Rensin’s field-tested originals.
13.1 Learning Objectives
By the end of this lecture you should be able to:
Explain the Elephant-Goldfish Model (EGM)—
what the Elephant and the Goldfish each represent, and why “feed the Elephant, test it against the Goldfish” produces trustworthy design documents. Run Phase 1 (Growing the Elephant): load context, enforce the No Code Rule, break a model out of a sycophantic spiral, and elicit a first-draft technical proposal from the AI.
Build a Phase 2 design document with the four canonical sections—
Problem, Technical Plan, Alternatives, Detailed Implementation— iteratively rather than in one shot, and explain what each section guards against. Execute the Goldfish Protocol: the comprehension test, the critic review, and the implementation-readiness check, and state the pass criterion for each.
Implement with guardrails (Phase 4): drive code generation from the finished design doc, recover from crashed sessions in seconds, and run a “mean” code review.
Apply recursive summarization to bootstrap a README hierarchy for a large existing codebase, including the human-verification step and the roll-up order.
Articulate the thesis “design is the new code”—
why design judgments must shift left into documents when an AI writes the implementation, and what this means for your career.
13.2 Motivation: Mass-Producing Our Mistakes
13.2.1 The terrifying realization
Everything in this course so far has made you faster: agents that edit multiple files, run tests, and commit. Rensin, a Distinguished Engineer at Google, built an internal tool this way that grew to 100,000+ users in months, with contributions from dozens of engineers. And then he names the danger that comes with that speed:
If we aren’t careful, we aren’t just writing code faster. We are mass-producing our mistakes.
If the rate of bad code being generated exceeds the rate of productive code, your project eventually collapses under its own unreadable, unmaintainable slop. Velocity without a high signal-to-noise ratio is negative productivity. The Elephant-Goldfish Model is a discipline for keeping the signal high while keeping the speed.
13.2.2 The three “aha” moments
The model crystallized from three painful experiences, worth internalizing as principles:
Aha #1—
Write down intent. An hour into a complex AI session, the model crashed. The new session was a blank slate; re-bootstrapping by pasting hundreds of lines of code was tedious and token-heavy. If he had maintained a clearly written document of intent and current progress, recovery would have been a 30-second operation: “Read this doc. Read the code it references. Tell me where we are, and let’s continue.” Aha #2—
Models need guardrails. With context restored, the AI immediately “ran off the rails, sprinting in directions I never asked for, hallucinating APIs.” AI models are eager to please, which means they will happily sprint off a cliff unless you build very strict guardrails. Aha #3—
sizeof(docs) << sizeof(code). Starting the next feature, he realized he never needed to feed the AI source code at all— the design docs were enough. It is much cheaper, faster, and more reliable to give an AI context through plain-English design documents than to force it to interpret tens of thousands of lines of raw code.
Connect #3 to the context-window discussion in the Claude Code lecture: a design doc is a lossy but high-signal compression of the code. The context window is a scarce resource; docs are how you spend it well.
13.2.3 Design is the new code
Historically, design judgment lived in two places: design docs and
the code itself—
Rensin’s prediction: either models produce so much code that humans
can’t review it all, or they eventually skip code and emit binaries.
Either way the code becomes opaque—
Design is the new code. The job is intent. The job is architecture. The job is design judgment.
This is the lecture’s thesis. Everything below is machinery for producing design documents rigorous enough to bear that weight.
13.2.4 Warm-up: the interrogator pattern (buying a car)
Before the full model, the essay’s Part 1 teaches its core move with a non-coding example. Want to buy a car? The toy usage is one big research prompt (“here’s what I want, produce the world’s best report”). You get an impressive-looking report and no confidence. The tool usage has three steps:
Victory loves preparation. Ask the model to act as your interrogator, not your researcher: “I will tell you what I’ve come up with so far and your job is to keep asking me clarifying questions until I tell you to stop... question my assumptions and force me to clarify.” Spend 15–20 minutes answering. Argue with it—
but argue to learn, not to win. Your one paragraph becomes 2–3 pages. Trust but verify. Same session: “Adopt the persona of an expert skeptical fact checker... I will only get to ask them to perform this research once... let’s create our acceptance criteria.” Another 15–20 minutes yields a 2–3 page grading rubric. (Recognize this from the testing lecture: acceptance criteria before the work, not after.)
Profit. Open a new session in research mode and hand it both artifacts: the thorough description and the evaluation criteria.
Save the description, the criteria, and the output—
Keep this pattern in mind—
13.3 Running Example: Recurring Tasks in TerpTasks
TerpTasks (our task-manager CLI from the Claude Code lectures) gets
its biggest feature yet: recurring tasks—
Where does the recurrence rule live—
on the Task, or as a separate schedule object? When you complete a recurring task, what exactly happens? Does the next instance appear immediately? At midnight? What if you complete it late?
Our repo has a hard convention: timezone-aware UTC everywhere. “Every Friday” is a local-time concept. (Feel the edge cases sharpening their knives.)
Does sorted_tasks()—
performance-sensitive, per CLAUDE.md— now need to generate future instances on the fly?
A feature with genuine design tension is exactly where EGM earns its keep. We will grow an Elephant for it, distill a design doc, test it against Goldfish, and only then let anything write code.
13.4 The Model: Elephants and Goldfish
Formal definition. The Elephant-Goldfish Model is an operating framework for AI-assisted development built on two kinds of AI sessions:
The Elephant—
“an elephant never forgets.” The heavily-prompted, context-rich session you build up over hours (or days), plus the design document it helps you write. The Elephant holds the institutional knowledge: the micro-decisions, the arguments you had with the model, the entire history of the feature. The Goldfish—
a brand-new, completely fresh AI session with zero memory. It knows exactly what is put in front of it, and nothing else.
The process for every new feature: Feed the Elephant; test it against the Goldfish.
In plain English: you co-write a design doc inside one long,
rich conversation. But a doc that “works” in that conversation
might secretly depend on things said in the chat that never made it
into the doc. So you hand only the doc to an amnesiac fresh
session and check whether it understands. If the Goldfish
gets it, the document—
Intuition. This is a serialization test. Programmers
know the pattern: an object graph “works” in memory, but the real
test of your save-format is whether a fresh process can reload it.
The Elephant is the live process; the design doc is the serialized
file; the Goldfish is the fresh process. If deserialization fails,
your file format is missing fields—
It’s also, note the symmetry, exactly what your future self and future sessions are: goldfish. Every /clear creates one. Every teammate who joins is one. Designing for the Goldfish is designing for everyone who wasn’t in the room.
PHASE 1 PHASE 2 PHASE 3 PHASE 4 |
Grow the Elephant Teach the Elephant Goldfish Protocol Implementation |
+----------------+ +----------------+ +----------------+ +----------------+ |
| context loading| | design doc, | | fresh sessions | | code from doc | |
| No Code Rule |--->| 4 sections, |--->| comprehension |--->| follow plan | |
| sycophant check| | built section | | critic review | | exactly; mean | |
| first proposal | | by section | | readiness | | code review | |
+----------------+ +----------------+ +-------+--------+ +----------------+ |
long, rich the doc IS fails? add to doc crash? reload |
conversation the artifact and loop back ---> doc, continue |
Common misconceptions.
“The Elephant is the model’s context window.” The Elephant is the session plus the design document. The doc is the part that survives; the chat is scaffolding (“mostly token noise”).
“The Goldfish is a worse model.” Same model, same capability—
different information. That’s the entire point: it isolates what the document alone communicates. “This replaces the Explore→Plan→Implement→Verify workflow.” It deepens it. That workflow fits a bug fix in one session; EGM is for features where the plan itself deserves days of scrutiny and the doc outlives the session.
Practical implication. The unit of review shifts. Your
teammates (and instructor) review your design doc, not your
chat logs—
13.5 Phase 1: Growing the Elephant (No Code Yet)
Goal: co-design the feature and its evaluation criteria before a single line of code is written.
13.5.1 Step 1— Context loading
Open a fresh session and point it at the relevant design docs or source tree. Ask it to describe, at a high level, what it learned.
Why? Because if it misunderstands your system on minute one,
it’s going to write terrible code on minute ten. Correct its
understanding immediately. (Starting a brand-new project? Skip
this step—
Worked example. For recurring tasks:
> Read CLAUDE.md, the README files, and src/terptasks/. Give me a |
high-level description of this system: its purpose, its main |
components, and the conventions it follows. Don't propose anything |
yet -- I just want to check your understanding. |
The reply mostly lands, but calls the store “a SQLite-backed task
database.” It’s in-memory (SQLite was exercise E13—
13.5.2 Step 2— The “No Code” Rule
Engineers love to write code. AI loves to write code. You
have to actively fight this urge—
“I do not want you to create code. We are not going to create code. Resist your impulse to create code. Instead, we are going to have a design discussion. I am about to describe a feature. I want you to ask me clarifying questions and challenge my assumptions. Do not just accept what I say.”
Then describe the feature and talk to it like a human for 20–30 minutes. Really argue. These models are excellent interrogators; make it push on the description and the edges of the problem. If it hallucinates, point it at the file that corrects it.
Why so adamant? Three reinforcing reasons. (1) Code
generated now would be built on an unexamined problem
statement—
Worked example. We describe recurring tasks in a paragraph. The Elephant starts interrogating:
AI: When a user completes the June 6 instance of "submit timesheet |
every Friday" on June 9 (late), when is the next due date -- |
June 13 (schedule-anchored) or June 16 (completion-anchored)? |
Us: Schedule-anchored. Timesheets are due Fridays regardless. |
AI: Should "every 3 days" watering behave the same way? If I water |
plants two days late, anchoring to schedule means the next |
watering is in 1 day. Is that what a user wants? |
Us: ...huh. No. That one should anchor to completion. |
AI: Then recurrence needs a per-rule anchoring policy. Also: 'every |
Friday' is local-time; your CLAUDE.md mandates UTC datetimes. |
Where does the timezone conversion live? |
Twenty minutes in, our one-paragraph feature has become three pages
of sharpened requirements—
13.5.3 Step 3— The Sycophant Challenge
AI models are trained to be helpful, which often makes them insufferable sycophants. If the AI stops asking hard questions and just says “Great idea!”, force it back into critic mode:
“You are not being helpful. Your highest and best use is to challenge my thinking.”
Or the Part 1 variant: “When you agree with me you are not being helpful. You are most helpful when you challenge my thinking and force me to think about the edges of my problem.”
Why does the spiral happen? Recall the LLM lecture: these models are tuned with human feedback to be “friendly and helpful,” and raters reward agreement. Flattery is a training artifact, not an assessment of your idea. Treat “You’re a genius!” as a warning light, not a compliment.
Pro-tip (Rensin’s): simply asking “Why do you think
that?” is your best friend—
Worked micro-example. Mid-discussion, the Elephant chirps:
“Storing the full recurrence expansion for a year ahead is a great
approach!”—
13.5.4 Step 4— The First Draft Technical Proposal
Once the problem description is complete, start debating
solutions—
“Based on your understanding of the codebase and what we discussed, I would like you to give me a first draft proposal of a technical implementation to actually make this feature happen. I’m not looking for code. I want prose from you that demonstrates your understanding of my system. Short blocks of pseudocode are fine if you think that will help, but I would strongly prefer clearly written text and block diagrams.”
Why AI-first? Two asymmetric benefits. If it drafts, the
draft reveals its actual understanding of your
system—
The first draft will be at least a little wrong. Ask clarifying
questions (“Why did you choose that pattern? I thought the system
worked like X...”). When it hallucinates, point at the correcting
file (“Nope—
This back-and-forth might take hours—
13.6 Phase 2: Teaching the Elephant (The Design Document)
Now we create the Markdown document that will serve as the absolute source of truth and the guardrails for the code.
The cardinal rule: do not one-shot it. Models have
output-token limits, and a doc generated in one breath will be
shallow where you most need depth. Build it iteratively in
the same chat session—
The four sections, in order:
# |
| Section |
| Contents |
| Guards against |
1 |
| The Problem |
| plain-English description of the business problem; 3-5 sentences, casual-reader level |
| solving the wrong problem; readers bouncing off |
2 |
| The Technical Plan |
| jargon-light prose: the big components and how they fit together; block diagram if it helps |
| architecture that exists only in your head |
3 |
| Alternatives |
| ideas considered and ruled out in Phase 1, with reasons; every major rejected decision |
| future hallucinations and arguments -- re-proposing what you already rejected |
4 |
| Detailed Implementation |
| longest section: step-by-step plan; every single file created or changed, with rationale |
| the AI going off the rails during implementation |
Why Alternatives is the sleeper section. Novices skip
it—
Why “every single file” in section 4. This is the
difference between a plan and a vibe. If the doc says models.py
(add Recurrence, extend Task), store.py
(spawn-next-on-complete in complete()), recurrence.py
(new—
Worked example—
+---------+ completes +---------------+ |
user ---->| cli.py |-------------->| store.py | |
+---------+ | complete() | |
+------+--------+ |
| if task.recurrence |
v |
+---------------+ |
all storage is UTC; | recurrence.py | |
local-time math is | next_due() | |
ONLY here --------------->| (tz-aware) | |
+------+--------+ |
| new Task |
v |
+---------------+ |
| store.add() | |
+---------------+ |
Then Alternatives (four entries, each one line of what and one of why not), then the Detailed Implementation with the file list above, function signatures, and the ordering of changes. Each section gets argued over before the next begins. Total: a couple of hours, ~6 pages.
Save this file to your tree
(docs/designs/recurring-tasks.md). It is the first artifact of
your new feature—
13.7 Phase 3: The Goldfish Protocol
How do you know your design doc is actually good—
13.7.1 Step 5— The Comprehension Test
Start a brand-new, empty session. Give it the doc and say:
“Read this document and the files it references. Tell me what it’s trying to accomplish, and how my system currently works as it relates to this feature.”
Pass criterion: the Goldfish can explain your system
based only on that document and the files it references. If
it can’t, your doc is missing context—
Worked example. Our first Goldfish explains recurrence
beautifully—
13.7.2 Step 6— The Critic Review
Start another Goldfish:
“Assume the role of an expert technical reviewer. Read this design doc and all the files it references. Tell me all the things I missed, all the faulty assumptions, all the edge cases I’m missing, and things I should have considered but did not. Every mistake and ambiguity you find makes you more helpful and useful.”
Note the last sentence—
Worked example. Our first critic returns seven findings.
Two are gold: (1) what happens to the recurrence when a user
deletes a recurring task—
13.7.3 Step 7— Implementation Readiness
One more fresh Goldfish, one more persona:
“You are an experienced software engineer experienced with our codebase. Read this document and the files it references and tell me: Does it absolutely have all the information you would require to successfully implement this feature in your first pass?”
If it asks questions, answer them in the doc—
Finally, show it to a real human for review and approval.
The Goldfish Protocol raises the floor; it does not replace human
judgment. (Working solo? Then use your best judgment—
Misconception check. “Three tests, three passes,
done.” Each test loops until it passes—
13.8 Phase 4: Implementation
13.8.1 Step 8— Coding with Guardrails
Now, and only now, do you write code. Hand the finalized design doc to the AI:
“Read this design doc and the files it references. Implement the feature as described. Follow the plan exactly.”
Because the doc explicitly lists every file and every change, the AI
produces much higher quality code—
Everything from the Claude Code lectures still applies inside
this step: permissions and hooks still gate the tools; tests still
run; you still review diffs; the deviation rule is now crisp—
13.8.2 Step 9— The “Mean” Code Review
When the code is done, feed it back to the AI and tell it to find every way the code is terrible. Rensin’s actual prompt:
“I have a strong intuition that this code is of poor quality. Please tear it to shreds and tell me all the ways it sucks.”
Tell it to flag any place you’ve gone ten lines without a comment.
Demand strict readability. Rinse and repeat until the
critiques are trivial—
Why the theatrical hostility? Same trick as the critic
review: the sycophantic default, asked “is this code okay?”, says
yes. Assert the code is bad and the agreeable move becomes
finding flaws. You are steering the model’s eagerness-to-please so
that pleasing you is the rigorous behavior. And the
readability bar isn’t cosmetic: in a design-is-the-new-code world,
humans mostly read code during incidents—
Worked example. Our mean review of recurrence.py
yields: a DST-unsafe shortcut (timedelta(days=7)—
13.9 Bootstrapping Reality: The README Hierarchy
“That’s nice for new projects, Dave, but my codebase is
millions of lines. I can’t train an elephant for a monolith.”—
13.9.1 Peanuts and hay: recursive summarization
Feed an AI the entire jungle of your source code at once and it
chokes—
Start at the leaves of your source tree. Assign each developer a set of directories.
Generate the peanuts. Point the AI at each leaf directory: “Read the files in this directory and produce a new file named README.md. This file should (a) explain the purpose of this directory and the files contained in it and (b) enumerate each file in the directory and a short description of its function.”
Human verification. The AI will be about 50% wrong in the leaf directories, because it has only the code to rely on—
it sees what, not why. The assigned engineer spends 5–10 minutes fixing each file. Roll up the hay. Move up one level: “Please read all the README.md files in all the subdirectories below me, then read the code in just this directory and create a file here named README.md...” (same (a)/(b) requirements). Note what this does: the level-up summary is built from verified summaries plus only the local code—
never the whole subtree. Continue upward, level by level, until you hit the root of the project.
ROOT README ^ pass 3: reads child READMEs |
/ \ + root-level code only |
src/README tests/README ^ pass 2: reads child READMEs |
/ \ + this dir's code only |
terptasks/README utils/README ^ pass 1 (leaves): reads code; |
human fixes ~50% error, 5-10 min each |
Why does the error rate fall as you rise? Leaf READMEs are
written from raw code—
The result is a hierarchy of context. Open a fresh session at the
project root, feed the Elephant just the READMEs, and it
understands your codebase almost instantly—
How long do you keep them? The READMEs are scaffolding: they
only need to exist until every code file is referenced (and lightly
explained) in at least one design doc. At 100% coverage you can
delete them—
13.9.2 Bonus: instant onboarding
A hierarchy of READMEs plus design docs makes onboarding new
engineers nearly frictionless: load the whole doc tree into a
NotebookLM-style tool, scope it to the newcomer’s subsystem, and
hand it over. Their first job: use it to learn enough to
write a design doc that passes the Goldfish test for their
first feature. When they can, they’re ready to contribute—
It reaches beyond engineers. PMs with a little coding background
routinely show up with a feature idea; Rensin hands them a notebook
and asks for a design doc. A few days later one arrives
~70–80% correct; one or two review rounds later it meets the
bar—
13.10 GitHub-Ready Mini-Project: the EGM Starter Kit
A committable overlay for your terptasks fork that operationalizes this lecture: the prompt library, the design-doc template, the finished worked design doc, and a script that builds the README hierarchy using headless Claude Code (tying in the CI material from the Claude Code lecture).
13.10.1 Structure
terptasks/ |
|- docs/ |
| |- designs/ |
| | |- TEMPLATE.md # the 4-section skeleton |
| | +- recurring-tasks.md # the worked design doc |
| +- egm/ |
| |- PROMPTS.md # every prompt from this lecture |
| +- CHECKLIST.md # phase gates, printable |
|- scripts/ |
| +- grow_readmes.sh # recursive summarization, leaves->root |
+- .claude/ |
+- commands/ |
|- goldfish-comprehension.md # Step 5 as a slash command |
|- goldfish-critic.md # Step 6 |
|- goldfish-ready.md # Step 7 |
+- mean-review.md # Step 9 |
13.10.2 docs/designs/TEMPLATE.md
# Design: <feature name>
Status: DRAFT | GOLDFISH-TESTED | APPROVED | IMPLEMENTED
Author: <you> + Elephant session(s) <dates>
Goldfish log: comprehension PASS <date> / critic converged <date, N rounds>
/ readiness PASS <date> / human approval <who, date>
## 1. The Problem
<!-- 3-5 sentences, plain English, casual-reader level. No jargon. -->
## 2. The Technical Plan
<!-- Jargon-light prose: the big components and how they fit together.
Include a block diagram of major components if it makes sense. -->
## 3. Alternatives
<!-- Every major idea considered and REJECTED in Phase 1, with why.
These are guardrails against future hallucinations and arguments. -->
<!-- format: - REJECTED: <idea> -- <one-line reason> -->
## 4. Detailed Implementation
<!-- Longest section. Step-by-step plan. Enumerate EVERY file that will
be created or changed, and the rationale for each. -->
### Files to be created or changed
| File | Change | Why |
|------|--------|-----|
### Step-by-step plan
<!-- ordered, each step small enough to verify -->
## References
<!-- files a fresh session must read: CLAUDE.md, relevant READMEs, ... -->13.10.3 scripts/grow_readmes.sh
Recursive summarization automated with headless mode (claude -p,
from the Claude Code lecture). Leaves first, then upward—
#!/bin/bash
# grow_readmes.sh -- build the README hierarchy, leaves -> root.
# Usage: ./scripts/grow_readmes.sh <source-root>
# After the leaf pass, STOP and hand-verify each README (expect ~50%
# error there); then re-run to continue upward. Tracks done dirs in
# .readme_done so the script is safely resumable.
set -euo pipefail
ROOT="${1:?usage: grow_readmes.sh <source-root>}"
DONE_LIST=".readme_done"
touch "$DONE_LIST"
# All directories under ROOT (skipping hidden/cache), deepest first,
# so children are always summarized before their parents.
find "$ROOT" -type d \( -name '.*' -o -name '__pycache__' \) -prune -o \
-type d -print | awk '{ print gsub(/\//,"/"), $0 }' | \
sort -rn | cut -d' ' -f2- | while read -r dir; do
grep -qxF "$dir" "$DONE_LIST" && continue # already done
ls "$dir"/*.py >/dev/null 2>&1 || continue # skip non-source dirs
if find "$dir" -mindepth 1 -type d | grep -q .; then
PROMPT="Read all the README.md files in the subdirectories below this
directory, then read the code in just this directory. Create README.md here
that (a) explains the purpose of this directory and its files and (b)
enumerates each file with a short description of its function."
else
PROMPT="Read the files in this directory and produce README.md that (a)
explains the purpose of this directory and its files and (b) enumerates each
file with a short description of its function."
fi
echo ">>> summarizing $dir"
( cd "$dir" && claude -p "$PROMPT" --allowedTools "Read,Glob,Write" )
echo "$dir" >> "$DONE_LIST"
# After finishing all leaves, pause for the human-verification pass.
if ! find "$dir" -mindepth 1 -type d | grep -q .; then
echo " leaf done -- verify $dir/README.md (expect ~50% wrong)"
fi
done
echo "All READMEs grown. Spot-check the upper levels; they should be" \
"much more accurate than the leaves."Design decisions worth teaching: depth-first ordering makes the
roll-up invariant hold; –allowedTools "Read,Glob,Write" is
least privilege (no Bash, no Edit—
13.10.4 .claude/commands/goldfish-critic.md
(The other three commands follow the same pattern.)
---
description: Run a Goldfish critic review of a design doc (Phase 3, Step 6)
argument-hint: <path-to-design-doc>
---
IMPORTANT: Answer only from the document and the files it references.
Do not assume any prior conversation exists.
Assume the role of an expert technical reviewer. Read $ARGUMENTS and all
the files it references. Tell me all the things the design missed, all the
faulty assumptions, all the edge cases it does not handle, and things the
author should have considered but did not. Every mistake and ambiguity you
find makes you more helpful and useful. Rank findings: blocker /
should-fix / nit-pick.One honest caveat, teach it explicitly: a slash command runs in your
current session—
13.10.5 Run it
git clone <your-fork-url> terptasks && cd terptasks
chmod +x scripts/grow_readmes.sh
./scripts/grow_readmes.sh src # pass 1: leaves; then verify by hand
./scripts/grow_readmes.sh src # pass 2+: rolls up (resumable)
claude # grow an Elephant for your feature
# ... Phase 1-2 produce docs/designs/<feature>.md ...
/clear # amnesia first!
/goldfish-comprehension docs/designs/recurring-tasks.md
/clear
/goldfish-critic docs/designs/recurring-tasks.md # loop until nit-picks
/clear
/goldfish-ready docs/designs/recurring-tasks.md
# implement: "Read this design doc... follow the plan exactly."
/mean-review src/terptasks/recurrence.py13.11 Practice Exercises
13.11.1 Basic
E1. In your own words: what does the Elephant hold that the Goldfish doesn’t, and why is that gap exactly what the Goldfish Protocol measures? Use the serialization analogy from The Model: Elephants and Goldfish (or invent a better one).
E2. Classify each prompt to the phase and step it belongs to: (a) “Why do you think that?”; (b) “Resist your impulse to create code”; (c) “Tear it to shreds”; (d) “Does it absolutely have all the information you would require...?”; (e) “Read the files in this directory and produce README.md...”.
E3. Your AI just replied “That’s a brilliant architecture—
you’ve thought of everything!” Write, verbatim, what you say next, and explain why the model fell into this mode (connect to the RLHF discussion from the LLM lecture). E4. State the pass criterion for each of the three Goldfish tests, and what you do with a failure in each case (where does the fix go?).
E5. Why must README generation start at the leaves and roll upward? What specifically breaks if you generate the root README first?
13.11.2 Intermediate
E6. Run Phase 1 for a real TerpTasks feature (e.g., task snoozing or shared task lists): context loading, No Code Rule, at least one deployed sycophant challenge, and an AI-first technical proposal. Deliverable: the transcript, annotated with where the model’s interrogation changed your design.
E7. Continue E6 through Phase 2: build the four-section design doc iteratively (one section per prompt, visibly corrected between sections). Deliverable: the doc plus a paragraph on what the Alternatives section will prevent, with a concrete imagined future argument it forecloses.
E8. Run the full Goldfish Protocol on your E7 doc (fresh sessions—
show the /clears). Log every comprehension failure and critic finding, classify critic findings as valuable/clarifying/nit, and report your personal “30% number.” Loop until the critic converges to nits. E9. Run grow_readmes.sh (or do it manually) on a codebase you did not write—
a leaf pass over an open-source project’s subdirectory. Measure the leaf error rate yourself: what fraction of AI claims about each file were wrong or missing the why? Compare with Rensin’s ~50%. E10. Implement your E7 feature via Step 8 (“follow the plan exactly”), then kill the session halfway through and recover in a fresh one using only the doc. Time both the recovery and what re-bootstrapping without the doc would have required. Then run the mean code review until it converges.
13.11.3 Advanced
E11. Calibrating the critic. Seed a design doc with three planted flaws of different kinds (a faulty assumption, a missing edge case, a contradiction with a referenced file). Run five independent Goldfish critics. Which flaws are found reliably? Which persona/prompt tweaks improve recall without drowning you in nits? Report precision/recall per critic.
E12. EGM vs. vibe-coding, controlled. Pick two features of comparable size. Build one by direct prompting (“just implement it”), one by full EGM. Compare: wall-clock time, review effort, defects found by the mean review and by tests, and—
one week later— how well a fresh session (and you) can explain each implementation. Write up which costs EGM front-loads and which it eliminates. E13. The comprehension debt audit. Rensin claims a human “can’t reasonably be expected to take responsibility for a system they don’t understand.” Take an AI-heavy project you’ve already built this semester. Audit it: for each module, could you pass an oral exam on why it works the way it does? Write the design doc retroactively for the worst module, Goldfish-test it, and reflect: what did writing-after-the-fact cost you compared to writing-before?
13.12 Summary
13.12.1 Key takeaways
Speed without discipline mass-produces mistakes. If bad code outpaces good, you are going backwards, quickly.
sizeof(docs) << sizeof(code)—
plain-English design docs are the cheapest, most reliable way to give an AI (or a human) context. Design is the new code. Feed the Elephant (long, argumentative, code-free design sessions), test against the Goldfish (fresh sessions that see only the doc). The doc must carry the knowledge, because every future session—
and colleague— is a goldfish. Fight the urge to code—
yours and the model’s. The No Code Rule, the sycophant challenge, and AI-first proposals keep the design phase honest. The four-section doc (Problem / Technical Plan / Alternatives / Detailed Implementation with every file enumerated) is built iteratively, never one-shot. Alternatives is guardrails; the file list is the leash.
Three Goldfish, three personas: can you understand it → can you break it → could you build it. Loop each until it passes; then get a human.
Implementation is the easy part when the doc is done: follow the plan exactly, recover from crashes in seconds, then demand a mean review until critiques go trivial.
Any codebase can be bootstrapped in about a week: README hierarchy by recursive summarization, leaves→root, with mandatory human verification at the (50%-wrong) leaves.
13.12.2 Terminology
Term |
| Meaning |
Elephant |
| the context-rich design session plus the design doc it produces |
Goldfish |
| a fresh zero-memory session; sees only what you hand it |
No Code Rule |
| explicit prohibition on code generation during design |
sycophant challenge |
| prompt that pushes a flattering model back into critic mode |
sycophantic spiral |
| agreement loop caused by helpfulness training |
AI-first proposal |
| having the AI draft the design to expose its understanding |
Alternatives section |
| rejected ideas + reasons; guardrails against re-proposal |
comprehension test |
| Goldfish explains the system from the doc alone |
critic review |
| adversarial Goldfish hunts flaws; ~30% of findings valuable |
implementation readiness |
| Goldfish-as-engineer: could you build this first-pass? |
mean code review |
| 'tear it to shreds' review; repeat until critiques are trivial |
recursive summarization |
| README hierarchy built leaves-to-root from verified summaries |
shift left |
| moving design judgment into documents before code exists |
13.12.3 Common mistakes
Letting code appear during Phase 1 (the conversation’s gravity shifts to fixing it).
Proposing the first design yourself—
you inherit your own blind spots, blessed by a sycophant. Rushing the technical-proposal argument (“you will be sorry if you do”).
One-shotting the design doc; skipping the Alternatives section.
Running “Goldfish” tests in the same session (no amnesia = no test).
Patching the Goldfish’s misunderstanding in chat instead of in the doc.
Stopping the critic loop after one round instead of at convergence-to-nits.
Skipping human verification of leaf READMEs, then trusting the roll-up.
Treating a passed Goldfish Protocol as replacing human review.
13.12.4 Connections
Backward: EGM is the feature-scale big sibling of the Explore→Plan→Implement→Verify workflow from How to Effectively Use Claude Code; CLAUDE.md and design docs are both “durable context” ((part "Claude Code Files")); the sycophantic spiral is RLHF from the LLM lecture wearing a party hat; acceptance-criteria-first is the testing lecture’s thesis.
Forward: Rensin’s Part 3 previews coming topics—
we are all managers now (managing agent fleets with outcomes + guardrails, the “look-away duration” metric), sustained concentration as a trainable muscle, and Jevons’ Paradox for developer productivity. His career advice is the course’s: your value is judgment and taste, scaled. “Hard like a hill, not hard like a mountain— so just start walking.”
13.13 Instructor Notes
Suggested duration: 75 minutes lecture + a multi-day lab (EGM
cannot be honestly experienced in 30 minutes—
Segment |
| Time |
motivation: slop, three ahas, design-is-the-new-code |
| 12 min |
car-buying interrogator pattern, live if possible |
| 8 min |
the model + serialization intuition |
| 8 min |
Phase 1 live demo on recurring tasks |
| 15 min |
Phase 2: the four sections, why Alternatives |
| 10 min |
Phase 3: three Goldfish live |
| 12 min |
Phase 4 + README hierarchy |
| 10 min |
kit + lab kickoff |
| remainder |
Where students struggle.
They cannot resist the code. The single hardest behavior change. In lab, make Phase 1 transcripts a graded deliverable and mark any generated code in them as an automatic revision.
They argue to win, not to learn—
dismissing the model’s pushback instead of mining it. The E6 annotation requirement (“where did interrogation change your design?”) forces the right posture. Fake goldfish. Students run “fresh” tests in the same session. Demand visible /clears in transcripts.
They fix the Goldfish, not the doc—
clarifying in chat and moving on. Hammer the rule: answers go in the document. Alternatives feels like busywork until the first time a fresh session re-proposes a rejected idea. The E7 “imagined future argument” makes this concrete before it bites.
Live demonstrations (in order of impact).
The sycophant challenge, live. Pitch the model a mediocre design until it flatters; deploy “your highest and best use is to challenge my thinking”; watch the register change. Reliably lands.
A comprehension-test failure. Bring a deliberately gutted design doc; let the Goldfish confabulate the missing part confidently; then show the fixed doc passing. Teaches both the test and why confident ≠ correct.
“Why do you think that?” snapping a hallucination loop, if you can bait one.
Run one leaf of grow_readmes.sh on an unfamiliar repo and have the class spot the ~50% leaf errors together.
Quiz seeds. (1) What are the Elephant and the Goldfish, and what property of the design doc does the Goldfish Protocol test that the Elephant session cannot? (2) Why must the AI propose the first technical design? Give both reasons. (3) Name the four design-doc sections in order; for two of them, state what future failure they guard against. (4) Give the three Goldfish tests’ personas and pass criteria in one line each. (5) In the README hierarchy: where is the AI most wrong, why there, and what is the mandated remedy? (6) Explain sizeof(docs) << sizeof(code) and connect it to the context window.
Homework ideas. E6–E8 as a three-week arc (design → doc →
protocol) on a real feature, graded on transcripts and doc quality
rather than code; E9 as a one-week measurement exercise with a
class-wide error-rate histogram; E12 as a final-project option—
Based on Dave Rensin, “Elephants, Goldfish and the New Golden
Age of Software Engineering”
(drensin.medium.com, April
2026). Quoted prompts are from the essay; the TerpTasks worked
examples and the starter kit are course additions. Read the
original—