On this page:
13.1 Learning Objectives
13.2 Motivation:   Mass-Producing Our Mistakes
13.2.1 The terrifying realization
13.2.2 The three “aha” moments
13.2.3 Design is the new code
13.2.4 Warm-up:   the interrogator pattern (buying a car)
13.3 Running Example:   Recurring Tasks in Terp  Tasks
13.4 The Model:   Elephants and Goldfish
13.5 Phase 1:   Growing the Elephant (No Code Yet)
13.5.1 Step 1—Context loading
13.5.2 Step 2—The “No Code” Rule
13.5.3 Step 3—The Sycophant Challenge
13.5.4 Step 4—The First Draft Technical Proposal
13.6 Phase 2:   Teaching the Elephant (The Design Document)
13.7 Phase 3:   The Goldfish Protocol
13.7.1 Step 5—The Comprehension Test
13.7.2 Step 6—The Critic Review
13.7.3 Step 7—Implementation Readiness
13.8 Phase 4:   Implementation
13.8.1 Step 8—Coding with Guardrails
13.8.2 Step 9—The “Mean” Code Review
13.9 Bootstrapping Reality:   The README Hierarchy
13.9.1 Peanuts and hay:   recursive summarization
13.9.2 Bonus:   instant onboarding
13.10 Git  Hub-Ready Mini-Project:   the EGM Starter Kit
13.10.1 Structure
13.10.2 docs/  designs/  TEMPLATE.md
13.10.3 scripts/  grow_  readmes.sh
13.10.4 .claude/  commands/  goldfish-critic.md
13.10.5 Run it
13.11 Practice Exercises
13.11.1 Basic
13.11.2 Intermediate
13.11.3 Advanced
13.12 Summary
13.12.1 Key takeaways
13.12.2 Terminology
13.12.3 Common mistakes
13.12.4 Connections
13.13 Instructor Notes
9.1

13 The Elephant-Goldfish Model🔗

    13.1 Learning Objectives

    13.2 Motivation: Mass-Producing Our Mistakes

      13.2.1 The terrifying realization

      13.2.2 The three “aha” moments

      13.2.3 Design is the new code

      13.2.4 Warm-up: the interrogator pattern (buying a car)

    13.3 Running Example: Recurring Tasks in TerpTasks

    13.4 The Model: Elephants and Goldfish

    13.5 Phase 1: Growing the Elephant (No Code Yet)

      13.5.1 Step 1—Context loading

      13.5.2 Step 2—The “No Code” Rule

      13.5.3 Step 3—The Sycophant Challenge

      13.5.4 Step 4—The First Draft Technical Proposal

    13.6 Phase 2: Teaching the Elephant (The Design Document)

    13.7 Phase 3: The Goldfish Protocol

      13.7.1 Step 5—The Comprehension Test

      13.7.2 Step 6—The Critic Review

      13.7.3 Step 7—Implementation Readiness

    13.8 Phase 4: Implementation

      13.8.1 Step 8—Coding with Guardrails

      13.8.2 Step 9—The “Mean” Code Review

    13.9 Bootstrapping Reality: The README Hierarchy

      13.9.1 Peanuts and hay: recursive summarization

      13.9.2 Bonus: instant onboarding

    13.10 GitHub-Ready Mini-Project: the EGM Starter Kit

      13.10.1 Structure

      13.10.2 docs/designs/TEMPLATE.md

      13.10.3 scripts/grow_readmes.sh

      13.10.4 .claude/commands/goldfish-critic.md

      13.10.5 Run it

    13.11 Practice Exercises

      13.11.1 Basic

      13.11.2 Intermediate

      13.11.3 Advanced

    13.12 Summary

      13.12.1 Key takeaways

      13.12.2 Terminology

      13.12.3 Common mistakes

      13.12.4 Connections

    13.13 Instructor Notes

These notes are based on Dave Rensin’s essay “Elephants, Goldfish and the New Golden Age of Software Engineering” (Medium, April 2026), adapted for this course. Where you see quoted prompts, they are Rensin’s field-tested originals.

13.1 Learning Objectives🔗

By the end of this lecture you should be able to:

  1. Explain the Elephant-Goldfish Model (EGM)—what the Elephant and the Goldfish each represent, and why “feed the Elephant, test it against the Goldfish” produces trustworthy design documents.

  2. Run Phase 1 (Growing the Elephant): load context, enforce the No Code Rule, break a model out of a sycophantic spiral, and elicit a first-draft technical proposal from the AI.

  3. Build a Phase 2 design document with the four canonical sections—Problem, Technical Plan, Alternatives, Detailed Implementation—iteratively rather than in one shot, and explain what each section guards against.

  4. Execute the Goldfish Protocol: the comprehension test, the critic review, and the implementation-readiness check, and state the pass criterion for each.

  5. Implement with guardrails (Phase 4): drive code generation from the finished design doc, recover from crashed sessions in seconds, and run a “mean” code review.

  6. Apply recursive summarization to bootstrap a README hierarchy for a large existing codebase, including the human-verification step and the roll-up order.

  7. Articulate the thesis “design is the new code”—why design judgments must shift left into documents when an AI writes the implementation, and what this means for your career.

13.2 Motivation: Mass-Producing Our Mistakes🔗

13.2.1 The terrifying realization🔗

Everything in this course so far has made you faster: agents that edit multiple files, run tests, and commit. Rensin, a Distinguished Engineer at Google, built an internal tool this way that grew to 100,000+ users in months, with contributions from dozens of engineers. And then he names the danger that comes with that speed:

If we aren’t careful, we aren’t just writing code faster. We are mass-producing our mistakes.

If the rate of bad code being generated exceeds the rate of productive code, your project eventually collapses under its own unreadable, unmaintainable slop. Velocity without a high signal-to-noise ratio is negative productivity. The Elephant-Goldfish Model is a discipline for keeping the signal high while keeping the speed.

13.2.2 The three “aha” moments🔗

The model crystallized from three painful experiences, worth internalizing as principles:

  1. Aha #1—Write down intent. An hour into a complex AI session, the model crashed. The new session was a blank slate; re-bootstrapping by pasting hundreds of lines of code was tedious and token-heavy. If he had maintained a clearly written document of intent and current progress, recovery would have been a 30-second operation: “Read this doc. Read the code it references. Tell me where we are, and let’s continue.”

  2. Aha #2—Models need guardrails. With context restored, the AI immediately “ran off the rails, sprinting in directions I never asked for, hallucinating APIs.” AI models are eager to please, which means they will happily sprint off a cliff unless you build very strict guardrails.

  3. Aha #3—sizeof(docs) << sizeof(code). Starting the next feature, he realized he never needed to feed the AI source code at all—the design docs were enough. It is much cheaper, faster, and more reliable to give an AI context through plain-English design documents than to force it to interpret tens of thousands of lines of raw code.

Connect #3 to the context-window discussion in the Claude Code lecture: a design doc is a lossy but high-signal compression of the code. The context window is a scarce resource; docs are how you spend it well.

13.2.3 Design is the new code🔗

Historically, design judgment lived in two places: design docs and the code itself—the trail of micro-decisions a human leaves in the logic. When an AI writes the code, the human makes no micro-decisions. Unless we force those judgments to shift left into a design document before the code exists, the system becomes incomprehensible to the human who is responsible for it.

Rensin’s prediction: either models produce so much code that humans can’t review it all, or they eventually skip code and emit binaries. Either way the code becomes opaque—and “when the code becomes opaque, the only artifact that matters is the design.”

Design is the new code. The job is intent. The job is architecture. The job is design judgment.

This is the lecture’s thesis. Everything below is machinery for producing design documents rigorous enough to bear that weight.

13.2.4 Warm-up: the interrogator pattern (buying a car)🔗

Before the full model, the essay’s Part 1 teaches its core move with a non-coding example. Want to buy a car? The toy usage is one big research prompt (“here’s what I want, produce the world’s best report”). You get an impressive-looking report and no confidence. The tool usage has three steps:

  1. Victory loves preparation. Ask the model to act as your interrogator, not your researcher: “I will tell you what I’ve come up with so far and your job is to keep asking me clarifying questions until I tell you to stop... question my assumptions and force me to clarify.” Spend 15–20 minutes answering. Argue with it—but argue to learn, not to win. Your one paragraph becomes 2–3 pages.

  2. Trust but verify. Same session: “Adopt the persona of an expert skeptical fact checker... I will only get to ask them to perform this research once... let’s create our acceptance criteria.” Another 15–20 minutes yields a 2–3 page grading rubric. (Recognize this from the testing lecture: acceptance criteria before the work, not after.)

  3. Profit. Open a new session in research mode and hand it both artifacts: the thorough description and the evaluation criteria.

Save the description, the criteria, and the output—not the full chat, which is “mostly token noise.” Over time you accumulate a searchable archive of your own reasoning: “Why did we decide this? What was our thought process about X?” Future You compounds on Current You.

Keep this pattern in mind—Phase 1 below is exactly this, aimed at software.

13.3 Running Example: Recurring Tasks in TerpTasks🔗

TerpTasks (our task-manager CLI from the Claude Code lectures) gets its biggest feature yet: recurring tasks“submit timesheet every Friday,” “water plants every 3 days.” It looks small and is not:

  • Where does the recurrence rule live—on the Task, or as a separate schedule object?

  • When you complete a recurring task, what exactly happens? Does the next instance appear immediately? At midnight? What if you complete it late?

  • Our repo has a hard convention: timezone-aware UTC everywhere. “Every Friday” is a local-time concept. (Feel the edge cases sharpening their knives.)

  • Does sorted_tasks()performance-sensitive, per CLAUDE.mdnow need to generate future instances on the fly?

A feature with genuine design tension is exactly where EGM earns its keep. We will grow an Elephant for it, distill a design doc, test it against Goldfish, and only then let anything write code.

13.4 The Model: Elephants and Goldfish🔗

Formal definition. The Elephant-Goldfish Model is an operating framework for AI-assisted development built on two kinds of AI sessions:

  • The Elephant“an elephant never forgets.” The heavily-prompted, context-rich session you build up over hours (or days), plus the design document it helps you write. The Elephant holds the institutional knowledge: the micro-decisions, the arguments you had with the model, the entire history of the feature.

  • The Goldfisha brand-new, completely fresh AI session with zero memory. It knows exactly what is put in front of it, and nothing else.

The process for every new feature: Feed the Elephant; test it against the Goldfish.

In plain English: you co-write a design doc inside one long, rich conversation. But a doc that “works” in that conversation might secretly depend on things said in the chat that never made it into the doc. So you hand only the doc to an amnesiac fresh session and check whether it understands. If the Goldfish gets it, the document—not the conversation—carries the knowledge.

Intuition. This is a serialization test. Programmers know the pattern: an object graph “works” in memory, but the real test of your save-format is whether a fresh process can reload it. The Elephant is the live process; the design doc is the serialized file; the Goldfish is the fresh process. If deserialization fails, your file format is missing fields—no matter how well the original process runs.

It’s also, note the symmetry, exactly what your future self and future sessions are: goldfish. Every /clear creates one. Every teammate who joins is one. Designing for the Goldfish is designing for everyone who wasn’t in the room.

      PHASE 1                PHASE 2               PHASE 3              PHASE 4

 Grow the Elephant     Teach the Elephant     Goldfish Protocol      Implementation

+----------------+    +----------------+    +----------------+    +----------------+

| context loading|    | design doc,    |    | fresh sessions |    | code from doc  |

| No Code Rule   |--->| 4 sections,    |--->| comprehension  |--->| follow plan    |

| sycophant check|    | built section  |    | critic review  |    | exactly; mean  |

| first proposal |    | by section     |    | readiness      |    | code review    |

+----------------+    +----------------+    +-------+--------+    +----------------+

      long, rich            the doc IS         fails? add to doc        crash? reload

      conversation          the artifact       and loop back --->       doc, continue

Common misconceptions.

  • “The Elephant is the model’s context window.” The Elephant is the session plus the design document. The doc is the part that survives; the chat is scaffolding (“mostly token noise”).

  • “The Goldfish is a worse model.” Same model, same capability—different information. That’s the entire point: it isolates what the document alone communicates.

  • “This replaces the Explore→Plan→Implement→Verify workflow.” It deepens it. That workflow fits a bug fix in one session; EGM is for features where the plan itself deserves days of scrutiny and the doc outlives the session.

Practical implication. The unit of review shifts. Your teammates (and instructor) review your design doc, not your chat logs—and mostly not even your code. The doc is the first artifact of the feature and, in Rensin’s words, “is now your new source code.”

13.5 Phase 1: Growing the Elephant (No Code Yet)🔗

Goal: co-design the feature and its evaluation criteria before a single line of code is written.

13.5.1 Step 1—Context loading🔗

Open a fresh session and point it at the relevant design docs or source tree. Ask it to describe, at a high level, what it learned.

Why? Because if it misunderstands your system on minute one, it’s going to write terrible code on minute ten. Correct its understanding immediately. (Starting a brand-new project? Skip this step—there’s nothing to load.)

Worked example. For recurring tasks:

> Read CLAUDE.md, the README files, and src/terptasks/. Give me a

  high-level description of this system: its purpose, its main

  components, and the conventions it follows. Don't propose anything

  yet -- I just want to check your understanding.

The reply mostly lands, but calls the store “a SQLite-backed task database.” It’s in-memory (SQLite was exercise E13—maybe it saw the branch). We correct now: “No—read store.py again; storage is an in-memory list.” Thirty seconds here saves an architecture built on a hallucinated database. Notice this is the previous lecture’s README hierarchy paying rent: the agent bootstrapped from docs, not from 2,000 lines of raw source (sizeof(docs) << sizeof(code)).

13.5.2 Step 2—The “No Code” Rule🔗

Engineers love to write code. AI loves to write code. You have to actively fight this urge—from both parties. Rensin’s prompt:

“I do not want you to create code. We are not going to create code. Resist your impulse to create code. Instead, we are going to have a design discussion. I am about to describe a feature. I want you to ask me clarifying questions and challenge my assumptions. Do not just accept what I say.”

Then describe the feature and talk to it like a human for 20–30 minutes. Really argue. These models are excellent interrogators; make it push on the description and the edges of the problem. If it hallucinates, point it at the file that corrects it.

Why so adamant? Three reinforcing reasons. (1) Code generated now would be built on an unexamined problem statement—fast slop. (2) The moment code exists, the conversation’s gravity shifts to fixing the code instead of questioning the design. (3) You’d be using the model as a researcher when what you need is the interrogator from the warm-up.

Worked example. We describe recurring tasks in a paragraph. The Elephant starts interrogating:

AI: When a user completes the June 6 instance of "submit timesheet

    every Friday" on June 9 (late), when is the next due date --

    June 13 (schedule-anchored) or June 16 (completion-anchored)?

Us: Schedule-anchored. Timesheets are due Fridays regardless.

AI: Should "every 3 days" watering behave the same way? If I water

    plants two days late, anchoring to schedule means the next

    watering is in 1 day. Is that what a user wants?

Us: ...huh. No. That one should anchor to completion.

AI: Then recurrence needs a per-rule anchoring policy. Also: 'every

    Friday' is local-time; your CLAUDE.md mandates UTC datetimes.

    Where does the timezone conversion live?

Twenty minutes in, our one-paragraph feature has become three pages of sharpened requirements—including a design distinction (schedule- vs. completion-anchored recurrence) we had not conceived of when we started. That is the Elephant growing.

13.5.3 Step 3—The Sycophant Challenge🔗

AI models are trained to be helpful, which often makes them insufferable sycophants. If the AI stops asking hard questions and just says “Great idea!”, force it back into critic mode:

“You are not being helpful. Your highest and best use is to challenge my thinking.”

Or the Part 1 variant: “When you agree with me you are not being helpful. You are most helpful when you challenge my thinking and force me to think about the edges of my problem.”

Why does the spiral happen? Recall the LLM lecture: these models are tuned with human feedback to be “friendly and helpful,” and raters reward agreement. Flattery is a training artifact, not an assessment of your idea. Treat “You’re a genius!” as a warning light, not a compliment.

Pro-tip (Rensin’s): simply asking “Why do you think that?” is your best friend—it often snaps the model out of a hallucination loop, because it forces the model to produce reasons where none existed.

Worked micro-example. Mid-discussion, the Elephant chirps: “Storing the full recurrence expansion for a year ahead is a great approach!”—moments after it raised the sorted_tasks() performance concern. We deploy the challenge; it reverses: pre-expanding creates unbounded list growth and stale instances when rules change; lazy generation of the next instance on completion is safer. The flattery was noise. The pushback had content.

13.5.4 Step 4—The First Draft Technical Proposal🔗

Once the problem description is complete, start debating solutions—and it is very important that the AI proposes the first design:

“Based on your understanding of the codebase and what we discussed, I would like you to give me a first draft proposal of a technical implementation to actually make this feature happen. I’m not looking for code. I want prose from you that demonstrates your understanding of my system. Short blocks of pseudocode are fine if you think that will help, but I would strongly prefer clearly written text and block diagrams.”

Why AI-first? Two asymmetric benefits. If it drafts, the draft reveals its actual understanding of your system—errors surface as visibly wrong sentences you can correct. If you draft and it reacts, you’ll almost certainly miss your own blind spots—and the sycophancy bias means it will tend to bless whatever you wrote.

The first draft will be at least a little wrong. Ask clarifying questions (“Why did you choose that pattern? I thought the system worked like X...”). When it hallucinates, point at the correcting file (“Nope—read store.py”). Argue until you settle on a solid approach.

This back-and-forth might take hours—some of Rensin’s sessions lasted 2–3 days. Do not rush this step; you will be sorry if you do. For our feature, the settled shape after a long session: a Recurrence dataclass (frequency, anchoring policy, local timezone) attached optionally to Task; completion of a recurring task spawns the next instance lazily; all stored datetimes stay UTC with conversion isolated in one module.

13.6 Phase 2: Teaching the Elephant (The Design Document)🔗

Now we create the Markdown document that will serve as the absolute source of truth and the guardrails for the code.

The cardinal rule: do not one-shot it. Models have output-token limits, and a doc generated in one breath will be shallow where you most need depth. Build it iteratively in the same chat sessionone section at a time, reviewing and correcting each before moving on.

The four sections, in order:

#

  

Section

  

Contents

  

Guards against

1

  

The Problem

  

plain-English description of the business problem; 3-5 sentences, casual-reader level

  

solving the wrong problem; readers bouncing off

2

  

The Technical Plan

  

jargon-light prose: the big components and how they fit together; block diagram if it helps

  

architecture that exists only in your head

3

  

Alternatives

  

ideas considered and ruled out in Phase 1, with reasons; every major rejected decision

  

future hallucinations and arguments -- re-proposing what you already rejected

4

  

Detailed Implementation

  

longest section: step-by-step plan; every single file created or changed, with rationale

  

the AI going off the rails during implementation

Why Alternatives is the sleeper section. Novices skip it—“why document what we’re not doing?” But three sessions from now, a fresh Goldfish will look at your lazy-generation design and helpfully suggest... pre-expanding a year of instances. Without section 3 it’s a plausible idea you must re-litigate (and a future session might silently “improve” toward it). With section 3, one sentence ends it: rejected—unbounded growth, stale instances on rule change; see Alternatives. You are writing guardrails against the model’s future eagerness, and minutes for your future self.

Why “every single file” in section 4. This is the difference between a plan and a vibe. If the doc says models.py (add Recurrence, extend Task), store.py (spawn-next-on-complete in complete()), recurrence.py (new—all timezone conversion lives here), cli.py (add repeat, tz flags), tests/test_recurrence.py (new—the late-completion table from Phase 1)—then an implementing session that starts touching sorted_tasks() is visibly off-plan, and you can say “follow the doc.” An enumerated file list converts “trust me” into a checklist. This is CLAUDE.md-style thinking at feature scale: write it down once, enforce it everywhere.

Worked example—building it iteratively. We prompt: “Write ONLY section 1, The Problem, 3–5 sentences, casual-reader level.” It returns five clean sentences; we trim jargon (“recurrence cadence” → “how often it repeats”). Then section 2, which arrives with a block diagram:

          +---------+   completes   +---------------+

user ---->|  cli.py |-------------->| store.py      |

          +---------+               |  complete()   |

                                    +------+--------+

                                           | if task.recurrence

                                           v

                                    +---------------+

          all storage is UTC;       | recurrence.py |

          local-time math is        |  next_due()   |

          ONLY here --------------->|  (tz-aware)   |

                                    +------+--------+

                                           | new Task

                                           v

                                    +---------------+

                                    | store.add()   |

                                    +---------------+

Then Alternatives (four entries, each one line of what and one of why not), then the Detailed Implementation with the file list above, function signatures, and the ordering of changes. Each section gets argued over before the next begins. Total: a couple of hours, ~6 pages.

Save this file to your tree (docs/designs/recurring-tasks.md). It is the first artifact of your new feature—your new source code.

13.7 Phase 3: The Goldfish Protocol🔗

How do you know your design doc is actually good—and not just riding on the context built up in your Elephant session? You test it against a Goldfish. Three tests, three fresh sessions, three different personas.

13.7.1 Step 5—The Comprehension Test🔗

Start a brand-new, empty session. Give it the doc and say:

“Read this document and the files it references. Tell me what it’s trying to accomplish, and how my system currently works as it relates to this feature.”

Pass criterion: the Goldfish can explain your system based only on that document and the files it references. If it can’t, your doc is missing context—add the details and repeat until it passes. Do not skip this loop. Documents that pass the Goldfish test become important protections against slop.

Worked example. Our first Goldfish explains recurrence beautifully—then says “when a recurring task is completed, cli.py computes the next instance.” Wrong layer: the doc never states that spawning lives in store.complete(); that fact lived only in the Elephant’s chat history. This is exactly the serialization bug the test exists to find. One sentence added to the Technical Plan; next Goldfish nails it. (Step by step: run test → find what the Goldfish got wrong → ask “what in the doc made you think that?” → patch the doc, not the Goldfish → fresh Goldfish → repeat.)

13.7.2 Step 6—The Critic Review🔗

Start another Goldfish:

“Assume the role of an expert technical reviewer. Read this design doc and all the files it references. Tell me all the things I missed, all the faulty assumptions, all the edge cases I’m missing, and things I should have considered but did not. Every mistake and ambiguity you find makes you more helpful and useful.”

Note the last sentence—it pre-inverts the sycophancy gradient: the persona is rewarded for finding faults, so agreement is now the unhelpful move. Rensin’s field data: about 30% of its suggestions are highly valuablemore than enough to make this worthwhile. Update the doc accordingly, then rinse and repeat with fresh critics until the recommendations consistently fall into the category of “nit-pick.” That convergence—blockers, then real questions, then font choices—is your signal the design space is explored.

Worked example. Our first critic returns seven findings. Two are gold: (1) what happens to the recurrence when a user deletes a recurring task—orphan or cascade? (never discussed!); (2) DST: “every Friday 9am” in America/New_York shifts in UTC twice a year—your next_due() had better use the tz-database, not timedelta(days=7). Three are useful clarifications; two are nits. We fold in the five, run a second critic: one moderate finding. Third: all nits. Converged.

13.7.3 Step 7—Implementation Readiness🔗

One more fresh Goldfish, one more persona:

“You are an experienced software engineer experienced with our codebase. Read this document and the files it references and tell me: Does it absolutely have all the information you would require to successfully implement this feature in your first pass?”

If it asks questions, answer them in the docnot in the chat. Rinse and repeat until there are no important missing items or unresolved ambiguities. Notice the progression of the three personas: can you understand it?can you break it?could you build it? Comprehension, adversarial, constructive.

Finally, show it to a real human for review and approval. The Goldfish Protocol raises the floor; it does not replace human judgment. (Working solo? Then use your best judgment—but the doc that survived three Goldfish is already something a solo developer almost never has: an externally validated design.)

Misconception check. “Three tests, three passes, done.” Each test loops until it passesand a big critic finding may send you back to the Elephant to re-argue the design. The diagram in The Model: Elephants and Goldfish has that back-arrow for a reason.

13.8 Phase 4: Implementation🔗

13.8.1 Step 8—Coding with Guardrails🔗

Now, and only now, do you write code. Hand the finalized design doc to the AI:

“Read this design doc and the files it references. Implement the feature as described. Follow the plan exactly.”

Because the doc explicitly lists every file and every change, the AI produces much higher quality code—it is executing a plan, not improvising one. And the doc pays its crash-insurance dividend (Aha #1): if the session spins out or crashes mid-implementation, you open a new session, hand it the design doc, tell it to read the doc and all referenced files, and you are immediately back in business. Thirty seconds, not an hour. The doc even makes implementation parallelizable—its file list partitions cleanly into independent work.

Everything from the Claude Code lectures still applies inside this step: permissions and hooks still gate the tools; tests still run; you still review diffs; the deviation rule is now crisp— any file not in the doc’s list is a conversation, not a commit. Either the doc was wrong (update it—it’s the source of truth) or the model drifted (rein it in).

13.8.2 Step 9—The “Mean” Code Review🔗

When the code is done, feed it back to the AI and tell it to find every way the code is terrible. Rensin’s actual prompt:

“I have a strong intuition that this code is of poor quality. Please tear it to shreds and tell me all the ways it sucks.”

Tell it to flag any place you’ve gone ten lines without a comment. Demand strict readability. Rinse and repeat until the critiques are trivial—or disappear.

Why the theatrical hostility? Same trick as the critic review: the sycophantic default, asked “is this code okay?”, says yes. Assert the code is bad and the agreeable move becomes finding flaws. You are steering the model’s eagerness-to-please so that pleasing you is the rigorous behavior. And the readability bar isn’t cosmetic: in a design-is-the-new-code world, humans mostly read code during incidents—comment density is for the 2 a.m. responder (who is also, of course, a goldfish).

Worked example. Our mean review of recurrence.py yields: a DST-unsafe shortcut (timedelta(days=7)exactly what the critic warned about, caught again at the code layer); a naked datetime.now() violating the CLAUDE.md UTC rule; two uncommented 15-line stretches; and one genuinely unfair complaint. Fix, re-review; round two finds style nits; round three, nothing. Ship it—through the normal human PR process.

13.9 Bootstrapping Reality: The README Hierarchy🔗

“That’s nice for new projects, Dave, but my codebase is millions of lines. I can’t train an elephant for a monolith.” Actually, you can, and it takes about a week, almost no matter how large the codebase.

13.9.1 Peanuts and hay: recursive summarization🔗

Feed an AI the entire jungle of your source code at once and it chokes—loses context, gets confused, hallucinates. Instead you feed it peanuts and hay: small, highly nutritious, compressed bites of context. Those peanuts are README.md files, and the AI does the heavy lifting of harvesting them through recursive summarization:

  1. Start at the leaves of your source tree. Assign each developer a set of directories.

  2. Generate the peanuts. Point the AI at each leaf directory: “Read the files in this directory and produce a new file named README.md. This file should (a) explain the purpose of this directory and the files contained in it and (b) enumerate each file in the directory and a short description of its function.”

  3. Human verification. The AI will be about 50% wrong in the leaf directories, because it has only the code to rely on—it sees what, not why. The assigned engineer spends 5–10 minutes fixing each file.

  4. Roll up the hay. Move up one level: “Please read all the README.md files in all the subdirectories below me, then read the code in just this directory and create a file here named README.md...” (same (a)/(b) requirements). Note what this does: the level-up summary is built from verified summaries plus only the local code—never the whole subtree.

  5. Continue upward, level by level, until you hit the root of the project.

                  ROOT README            ^  pass 3: reads child READMEs

                 /           \              + root-level code only

          src/README      tests/README   ^  pass 2: reads child READMEs

          /       \                         + this dir's code only

terptasks/README  utils/README           ^  pass 1 (leaves): reads code;

                                            human fixes ~50% error, 5-10 min each

Why does the error rate fall as you rise? Leaf READMEs are written from raw code—maximum inference, maximum error (hence mandatory human verification there, where it’s cheap). Upper levels summarize already-verified text, so mistakes don’t compound; they wash out. You are building a pyramid where every layer rests on human-checked material. This is also why the order is non-negotiable: leaves first.

The result is a hierarchy of context. Open a fresh session at the project root, feed the Elephant just the READMEs, and it understands your codebase almost instantly—reliably picking out the major subsystems and design patterns—at a fraction of the context cost of reading source. (This is the doc-stack our Phase 1 context-loading step consumed.)

How long do you keep them? The READMEs are scaffolding: they only need to exist until every code file is referenced (and lightly explained) in at least one design doc. At 100% coverage you can delete them—or keep them, your call. A new project run rigorously under EGM from day one never needs them at all.

13.9.2 Bonus: instant onboarding🔗

A hierarchy of READMEs plus design docs makes onboarding new engineers nearly frictionless: load the whole doc tree into a NotebookLM-style tool, scope it to the newcomer’s subsystem, and hand it over. Their first job: use it to learn enough to write a design doc that passes the Goldfish test for their first feature. When they can, they’re ready to contribute—no weeks of shadowing, no reverse-engineering opaque legacy code.

It reaches beyond engineers. PMs with a little coding background routinely show up with a feature idea; Rensin hands them a notebook and asks for a design doc. A few days later one arrives ~70–80% correct; one or two review rounds later it meets the bar—and the code built from it “won’t be slop, which means it won’t be a waste of time.” The design doc is the interface through which non-engineers safely contribute engineering work. Come for the speed; stay for the comprehension.

13.10 GitHub-Ready Mini-Project: the EGM Starter Kit🔗

A committable overlay for your terptasks fork that operationalizes this lecture: the prompt library, the design-doc template, the finished worked design doc, and a script that builds the README hierarchy using headless Claude Code (tying in the CI material from the Claude Code lecture).

13.10.1 Structure🔗

terptasks/

|- docs/

|  |- designs/

|  |  |- TEMPLATE.md                 # the 4-section skeleton

|  |  +- recurring-tasks.md          # the worked design doc

|  +- egm/

|     |- PROMPTS.md                  # every prompt from this lecture

|     +- CHECKLIST.md                # phase gates, printable

|- scripts/

|  +- grow_readmes.sh                # recursive summarization, leaves->root

+- .claude/

   +- commands/

      |- goldfish-comprehension.md   # Step 5 as a slash command

      |- goldfish-critic.md          # Step 6

      |- goldfish-ready.md           # Step 7

      +- mean-review.md              # Step 9

13.10.2 docs/designs/TEMPLATE.md🔗

# Design: <feature name>

Status: DRAFT | GOLDFISH-TESTED | APPROVED | IMPLEMENTED
Author: <you> + Elephant session(s) <dates>
Goldfish log: comprehension PASS <date> / critic converged <date, N rounds>
              / readiness PASS <date> / human approval <who, date>

## 1. The Problem
<!-- 3-5 sentences, plain English, casual-reader level. No jargon. -->

## 2. The Technical Plan
<!-- Jargon-light prose: the big components and how they fit together.
     Include a block diagram of major components if it makes sense. -->

## 3. Alternatives
<!-- Every major idea considered and REJECTED in Phase 1, with why.
     These are guardrails against future hallucinations and arguments. -->
<!-- format:  - REJECTED: <idea> -- <one-line reason> -->

## 4. Detailed Implementation
<!-- Longest section. Step-by-step plan. Enumerate EVERY file that will
     be created or changed, and the rationale for each. -->

### Files to be created or changed
| File | Change | Why |
|------|--------|-----|

### Step-by-step plan
<!-- ordered, each step small enough to verify -->

## References
<!-- files a fresh session must read: CLAUDE.md, relevant READMEs, ... -->

13.10.3 scripts/grow_readmes.sh🔗

Recursive summarization automated with headless mode (claude -p, from the Claude Code lecture). Leaves first, then upward—depth order is computed, not hoped for:

#!/bin/bash
# grow_readmes.sh -- build the README hierarchy, leaves -> root.
# Usage: ./scripts/grow_readmes.sh <source-root>
# After the leaf pass, STOP and hand-verify each README (expect ~50%
# error there); then re-run to continue upward. Tracks done dirs in
# .readme_done so the script is safely resumable.
set -euo pipefail

ROOT="${1:?usage: grow_readmes.sh <source-root>}"
DONE_LIST=".readme_done"
touch "$DONE_LIST"

# All directories under ROOT (skipping hidden/cache), deepest first,
# so children are always summarized before their parents.
find "$ROOT" -type d \( -name '.*' -o -name '__pycache__' \) -prune -o \
     -type d -print | awk '{ print gsub(/\//,"/"), $0 }' | \
     sort -rn | cut -d' ' -f2- | while read -r dir; do

  grep -qxF "$dir" "$DONE_LIST" && continue        # already done
  ls "$dir"/*.py >/dev/null 2>&1 || continue        # skip non-source dirs

  if find "$dir" -mindepth 1 -type d | grep -q .; then
    PROMPT="Read all the README.md files in the subdirectories below this
directory, then read the code in just this directory. Create README.md here
that (a) explains the purpose of this directory and its files and (b)
enumerates each file with a short description of its function."
  else
    PROMPT="Read the files in this directory and produce README.md that (a)
explains the purpose of this directory and its files and (b) enumerates each
file with a short description of its function."
  fi

  echo ">>> summarizing $dir"
  ( cd "$dir" && claude -p "$PROMPT" --allowedTools "Read,Glob,Write" )
  echo "$dir" >> "$DONE_LIST"

  # After finishing all leaves, pause for the human-verification pass.
  if ! find "$dir" -mindepth 1 -type d | grep -q .; then
    echo "    leaf done -- verify $dir/README.md (expect ~50% wrong)"
  fi
done
echo "All READMEs grown. Spot-check the upper levels; they should be" \
     "much more accurate than the leaves."

Design decisions worth teaching: depth-first ordering makes the roll-up invariant hold; allowedTools "Read,Glob,Write" is least privilege (no Bash, no Edit—it can only read code and write the README); the done-list makes it resumable; and the explicit pause honors the mandatory human-verification step instead of automating past it.

13.10.4 .claude/commands/goldfish-critic.md🔗

(The other three commands follow the same pattern.)

---
description: Run a Goldfish critic review of a design doc (Phase 3, Step 6)
argument-hint: <path-to-design-doc>
---
IMPORTANT: Answer only from the document and the files it references.
Do not assume any prior conversation exists.

Assume the role of an expert technical reviewer. Read $ARGUMENTS and all
the files it references. Tell me all the things the design missed, all the
faulty assumptions, all the edge cases it does not handle, and things the
author should have considered but did not. Every mistake and ambiguity you
find makes you more helpful and useful. Rank findings: blocker /
should-fix / nit-pick.

One honest caveat, teach it explicitly: a slash command runs in your current session—a true Goldfish needs /clear (or a fresh window) first. The command encodes the persona; you must supply the amnesia. CHECKLIST.md says exactly that in bold.

13.10.5 Run it🔗

git clone <your-fork-url> terptasks && cd terptasks
chmod +x scripts/grow_readmes.sh
./scripts/grow_readmes.sh src          # pass 1: leaves; then verify by hand
./scripts/grow_readmes.sh src          # pass 2+: rolls up (resumable)
claude                                 # grow an Elephant for your feature
# ... Phase 1-2 produce docs/designs/<feature>.md ...
/clear                                 # amnesia first!
/goldfish-comprehension docs/designs/recurring-tasks.md
/clear
/goldfish-critic docs/designs/recurring-tasks.md   # loop until nit-picks
/clear
/goldfish-ready docs/designs/recurring-tasks.md
# implement: "Read this design doc... follow the plan exactly."
/mean-review src/terptasks/recurrence.py

13.11 Practice Exercises🔗

13.11.1 Basic🔗
  1. E1. In your own words: what does the Elephant hold that the Goldfish doesn’t, and why is that gap exactly what the Goldfish Protocol measures? Use the serialization analogy from The Model: Elephants and Goldfish (or invent a better one).

  2. E2. Classify each prompt to the phase and step it belongs to: (a) “Why do you think that?”; (b) “Resist your impulse to create code”; (c) “Tear it to shreds”; (d) “Does it absolutely have all the information you would require...?”; (e) “Read the files in this directory and produce README.md...”.

  3. E3. Your AI just replied “That’s a brilliant architecture—you’ve thought of everything!” Write, verbatim, what you say next, and explain why the model fell into this mode (connect to the RLHF discussion from the LLM lecture).

  4. E4. State the pass criterion for each of the three Goldfish tests, and what you do with a failure in each case (where does the fix go?).

  5. E5. Why must README generation start at the leaves and roll upward? What specifically breaks if you generate the root README first?

13.11.2 Intermediate🔗
  1. E6. Run Phase 1 for a real TerpTasks feature (e.g., task snoozing or shared task lists): context loading, No Code Rule, at least one deployed sycophant challenge, and an AI-first technical proposal. Deliverable: the transcript, annotated with where the model’s interrogation changed your design.

  2. E7. Continue E6 through Phase 2: build the four-section design doc iteratively (one section per prompt, visibly corrected between sections). Deliverable: the doc plus a paragraph on what the Alternatives section will prevent, with a concrete imagined future argument it forecloses.

  3. E8. Run the full Goldfish Protocol on your E7 doc (fresh sessions—show the /clears). Log every comprehension failure and critic finding, classify critic findings as valuable/clarifying/nit, and report your personal “30% number.” Loop until the critic converges to nits.

  4. E9. Run grow_readmes.sh (or do it manually) on a codebase you did not write—a leaf pass over an open-source project’s subdirectory. Measure the leaf error rate yourself: what fraction of AI claims about each file were wrong or missing the why? Compare with Rensin’s ~50%.

  5. E10. Implement your E7 feature via Step 8 (“follow the plan exactly”), then kill the session halfway through and recover in a fresh one using only the doc. Time both the recovery and what re-bootstrapping without the doc would have required. Then run the mean code review until it converges.

13.11.3 Advanced🔗
  1. E11. Calibrating the critic. Seed a design doc with three planted flaws of different kinds (a faulty assumption, a missing edge case, a contradiction with a referenced file). Run five independent Goldfish critics. Which flaws are found reliably? Which persona/prompt tweaks improve recall without drowning you in nits? Report precision/recall per critic.

  2. E12. EGM vs. vibe-coding, controlled. Pick two features of comparable size. Build one by direct prompting (“just implement it”), one by full EGM. Compare: wall-clock time, review effort, defects found by the mean review and by tests, and—one week later—how well a fresh session (and you) can explain each implementation. Write up which costs EGM front-loads and which it eliminates.

  3. E13. The comprehension debt audit. Rensin claims a human “can’t reasonably be expected to take responsibility for a system they don’t understand.” Take an AI-heavy project you’ve already built this semester. Audit it: for each module, could you pass an oral exam on why it works the way it does? Write the design doc retroactively for the worst module, Goldfish-test it, and reflect: what did writing-after-the-fact cost you compared to writing-before?

13.12 Summary🔗

13.12.1 Key takeaways🔗
  1. Speed without discipline mass-produces mistakes. If bad code outpaces good, you are going backwards, quickly.

  2. sizeof(docs) << sizeof(code)plain-English design docs are the cheapest, most reliable way to give an AI (or a human) context. Design is the new code.

  3. Feed the Elephant (long, argumentative, code-free design sessions), test against the Goldfish (fresh sessions that see only the doc). The doc must carry the knowledge, because every future session—and colleague—is a goldfish.

  4. Fight the urge to codeyours and the model’s. The No Code Rule, the sycophant challenge, and AI-first proposals keep the design phase honest.

  5. The four-section doc (Problem / Technical Plan / Alternatives / Detailed Implementation with every file enumerated) is built iteratively, never one-shot. Alternatives is guardrails; the file list is the leash.

  6. Three Goldfish, three personas: can you understand it → can you break it → could you build it. Loop each until it passes; then get a human.

  7. Implementation is the easy part when the doc is done: follow the plan exactly, recover from crashes in seconds, then demand a mean review until critiques go trivial.

  8. Any codebase can be bootstrapped in about a week: README hierarchy by recursive summarization, leaves→root, with mandatory human verification at the (50%-wrong) leaves.

13.12.2 Terminology🔗

Term

  

Meaning

Elephant

  

the context-rich design session plus the design doc it produces

Goldfish

  

a fresh zero-memory session; sees only what you hand it

No Code Rule

  

explicit prohibition on code generation during design

sycophant challenge

  

prompt that pushes a flattering model back into critic mode

sycophantic spiral

  

agreement loop caused by helpfulness training

AI-first proposal

  

having the AI draft the design to expose its understanding

Alternatives section

  

rejected ideas + reasons; guardrails against re-proposal

comprehension test

  

Goldfish explains the system from the doc alone

critic review

  

adversarial Goldfish hunts flaws; ~30% of findings valuable

implementation readiness

  

Goldfish-as-engineer: could you build this first-pass?

mean code review

  

'tear it to shreds' review; repeat until critiques are trivial

recursive summarization

  

README hierarchy built leaves-to-root from verified summaries

shift left

  

moving design judgment into documents before code exists

13.12.3 Common mistakes🔗
  • Letting code appear during Phase 1 (the conversation’s gravity shifts to fixing it).

  • Proposing the first design yourself—you inherit your own blind spots, blessed by a sycophant.

  • Rushing the technical-proposal argument (“you will be sorry if you do”).

  • One-shotting the design doc; skipping the Alternatives section.

  • Running “Goldfish” tests in the same session (no amnesia = no test).

  • Patching the Goldfish’s misunderstanding in chat instead of in the doc.

  • Stopping the critic loop after one round instead of at convergence-to-nits.

  • Skipping human verification of leaf READMEs, then trusting the roll-up.

  • Treating a passed Goldfish Protocol as replacing human review.

13.12.4 Connections🔗
  • Backward: EGM is the feature-scale big sibling of the Explore→Plan→Implement→Verify workflow from How to Effectively Use Claude Code; CLAUDE.md and design docs are both “durable context” ((part "Claude Code Files")); the sycophantic spiral is RLHF from the LLM lecture wearing a party hat; acceptance-criteria-first is the testing lecture’s thesis.

  • Forward: Rensin’s Part 3 previews coming topics— we are all managers now (managing agent fleets with outcomes + guardrails, the “look-away duration” metric), sustained concentration as a trainable muscle, and Jevons’ Paradox for developer productivity. His career advice is the course’s: your value is judgment and taste, scaled. “Hard like a hill, not hard like a mountain—so just start walking.”

13.13 Instructor Notes🔗

Suggested duration: 75 minutes lecture + a multi-day lab (EGM cannot be honestly experienced in 30 minutes—the phases have real wall-clock time).

Segment

  

Time

motivation: slop, three ahas, design-is-the-new-code

  

12 min

car-buying interrogator pattern, live if possible

  

8 min

the model + serialization intuition

  

8 min

Phase 1 live demo on recurring tasks

  

15 min

Phase 2: the four sections, why Alternatives

  

10 min

Phase 3: three Goldfish live

  

12 min

Phase 4 + README hierarchy

  

10 min

kit + lab kickoff

  

remainder

Where students struggle.

  • They cannot resist the code. The single hardest behavior change. In lab, make Phase 1 transcripts a graded deliverable and mark any generated code in them as an automatic revision.

  • They argue to win, not to learndismissing the model’s pushback instead of mining it. The E6 annotation requirement (“where did interrogation change your design?”) forces the right posture.

  • Fake goldfish. Students run “fresh” tests in the same session. Demand visible /clears in transcripts.

  • They fix the Goldfish, not the docclarifying in chat and moving on. Hammer the rule: answers go in the document.

  • Alternatives feels like busywork until the first time a fresh session re-proposes a rejected idea. The E7 “imagined future argument” makes this concrete before it bites.

Live demonstrations (in order of impact).

  1. The sycophant challenge, live. Pitch the model a mediocre design until it flatters; deploy “your highest and best use is to challenge my thinking”; watch the register change. Reliably lands.

  2. A comprehension-test failure. Bring a deliberately gutted design doc; let the Goldfish confabulate the missing part confidently; then show the fixed doc passing. Teaches both the test and why confident ≠ correct.

  3. “Why do you think that?” snapping a hallucination loop, if you can bait one.

  4. Run one leaf of grow_readmes.sh on an unfamiliar repo and have the class spot the ~50% leaf errors together.

Quiz seeds. (1) What are the Elephant and the Goldfish, and what property of the design doc does the Goldfish Protocol test that the Elephant session cannot? (2) Why must the AI propose the first technical design? Give both reasons. (3) Name the four design-doc sections in order; for two of them, state what future failure they guard against. (4) Give the three Goldfish tests’ personas and pass criteria in one line each. (5) In the README hierarchy: where is the AI most wrong, why there, and what is the mandated remedy? (6) Explain sizeof(docs) << sizeof(code) and connect it to the context window.

Homework ideas. E6–E8 as a three-week arc (design → doc → protocol) on a real feature, graded on transcripts and doc quality rather than code; E9 as a one-week measurement exercise with a class-wide error-rate histogram; E12 as a final-project option—the write-ups make excellent discussion material for the “we are all managers now” closing session.

Based on Dave Rensin, “Elephants, Goldfish and the New Golden Age of Software Engineering” (drensin.medium.com, April 2026). Quoted prompts are from the essay; the TerpTasks worked examples and the starter kit are course additions. Read the original—Part 3’s predictions are worth your time.