How I Made My AI Agents Stop Repeating the Same Mistake Twice

Jul 23, 2026 · 8 min read

TL;DR: Every time one of my agents does something wrong and I correct it, it writes a short markdown file before it does anything else. The next session loads those files before acting. Over 4 months that's 216 memory files on the machine I run this on most (my laptop keeps its own separate pile, more on that below). The mistakes that happened first (a gendered filler word sent to the wrong audience, a data encoding that overstated trades by 100x, a push that silently failed 3 times) just don't happen anymore. No fine-tuning. No system-prompt rewrites. Just a flat file directory the model reads before starting work.

What happens when an AI agent repeats the same mistake?

Eleven out of nineteen drafts said “man.” Including ones going to women creators.

This was my Day One outreach drafting agent, July 16th. I had “man” on an allowlist as casual filler, natural-sounding, standard casual speech. Seemed fine.

I caught it reviewing the test output. My correction was one line: “let’s be gender neutral, it returned man and it was a woman speaking.”

In a normal setup that correction lives in a chat transcript and dies there. The next session doesn’t have the transcript. Same pipeline, same mistake.

What happened instead: the session I corrected wrote a memory file immediately.

---
name: feedback_outreach_gender_neutral_address
description: "All outreach/comment copy must be gender-neutral — never 'man'/'bro'/'girl'/'sis'; a transcript doesn't tell you who's speaking"
metadata:
  type: feedback
---

Reno (2026-07-16), reviewing DayOne comment-drafter test output: "let's be gender
neutral, it returned man and it was a woman speaking."

**Why:** The drafter only sees a transcript, which carries no reliable gender signal.
A gendered address at the wrong person is worse than any AI tell — it proves nobody
watched the video. The tic came from listing 'man' as an allowed casual filler in the
prompt; the model then used it in 11/19 drafts.

**How to apply:** Any generated copy addressed at a specific unknown person (TikTok
comments, DMs, replies, any per-app drafter) bans gendered address outright: no 'man',
'bro', 'dude', 'girl', 'sis', 'babe'. Address with 'you' or not at all. Neutral fillers
('honestly', 'ngl', 'lol', 'fr') are fine. Never put a gendered word in a filler
allowlist — allowlists become tics.

And the one-line entry that went into MEMORY.md, the index that loads every session:

- [Gender-neutral outreach](feedback_outreach_gender_neutral_address.md) never man/bro/girl; allowlisted fillers become tics

That’s the whole mechanism. Next 5 drafts: 0 instances.

How does agent memory load at session start?

Every session starts with MEMORY.md loading automatically. It’s an index: one line per file, with a short description. The session looks at what it’s about to do, scans descriptions, and pulls the relevant individual files. Outreach session: reads outreach and voice files. Vigil data pipeline: reads Vigil-specific reference files.

The memory loop: mistake happens, gets corrected, memory file written, MEMORY.md updated, next session reads the file, mistake doesn't repeat

Four types, each with a different job:

Type	What it stores
feedback	Rules: what not to do, with why and when to apply it
reference	Where to find things: endpoints, scripts, data shapes
project	Current state: what’s live, decisions made, timelines
user	Working style and preferences

Current counts on my Mac mini (July 18, 2026). My laptop keeps its own memory directory with a separate count, see “Cross-machine gaps” below:

Type	Count
feedback	124
reference	57
project	30
user	5
Total	216

Most of them were written in the 30 seconds after something went wrong.

What kinds of mistakes does a memory file prevent?

The congress amounts bug. Vigil’s data pipeline stores congressional trade amounts as actual_dollars × 100. Raw value 1500100 is $15,001, not $1.5M. Sessions using the raw value were overstating trades by 100x. The memory file now says: “Always call // 100 on raw DB amounts before writing labels.” Standard STOCK ACT tiers included as a lookup table. The pipeline hasn’t gotten this wrong since.

The MP4 push failure. push_approved.py only handled carousel posts (PNG slide files). When Vigil started using animated MP4 formats, every push silently failed: “Missing slides or caption.” The same post failed 3 times before I noticed. After the fix, the memory file documents the correct branch to check, the symptom to watch for, and how to reset a failed post. The next animated post went through clean.

The approval.json shape bug. A schema change made Vigil’s approval.json store posts as a list instead of a dict. Five separate code paths broke on .items(). Memory file: “Vigil approval.json stores posts as a LIST not dict. Normalize per call site, never in the shared loader.” Every session that has touched that file since reads the note first.

Same loop every time: something fails, a file gets written, the thing stops failing.

How does a nightly retro turn corrections into patterns?

Memory files handle specific, bounded corrections. “No gendered address.” “Divide congress amounts by 100.”

The nightly retro promotes corrections into patterns.

It runs every night, looks at what changed in the repo that day, and asks: is there a durable lesson here? Not a one-time fix. A generalizable rule. If yes, it edits the relevant playbook file directly. The knowledge/playbooks/ files themselves change.

Example: an agent killed an Apple Search Ads keyword after one expensive install. One data point, no pattern. The retro noticed. My ASO playbook now reads: “A single-install CPA is statistically noisy, one expensive install doesn’t prove the keyword is bad. Lower the bid, don’t kill. Only kill on a stable multi-tap zero-conversion read.” That rule lives in the playbook permanently. Every future keyword decision reads it.

Two stages: triage (cheap model, is this worth persisting?) and patch (apply it). Auto-apply goes to playbooks and what-flopped.md files. Per-app system prompts and skills go to a proposal queue, because those wait for me to review the diff first.

Two layers total:

Memory files: “don’t do this specific thing”
Playbooks: “here’s the current understanding of how this category of thing works”

A memory file records a correction. A playbook records a pattern. The retro is what moves things from one to the other.

What does agent memory still not solve?

Context drift in long sessions. Memory loads at session start, but a 2-hour session with 50+ tool calls compresses early context. Session-to-session continuity is solid. Within-session continuity is a separate, harder problem.

Cross-machine gaps, and this one has actually bitten me. My Mac mini has its memory directory. My laptop has another. They share the same git repo but not the same memory files. Something learned on the mini doesn’t automatically exist on the laptop. The nightly retro partially closes this since it writes to the shared repo. But per-session corrections, the immediate-feedback-to-file ones, stay local. If I correct something on the mini and the next session runs on the laptop, the mistake can repeat.

Corrections that never get written. The whole thing depends on me noticing a correction happened. The retro catches what shows up in a diff. Anything I say directly gets written immediately. Subtle drift that doesn’t trip either, the kind where an agent gradually wanders from the intended pattern without a clear moment of failure, that’s probably still leaking somewhere.

How do you set up agent memory files yourself?

Not a template. The actual files running this.

A memory file (this is the gendered-address one, verbatim):

---
name: feedback_outreach_gender_neutral_address
description: "All outreach/comment copy must be gender-neutral — never 'man'/'bro'/'girl'/'sis'; a transcript doesn't tell you who's speaking"
metadata:
  type: feedback
---

Reno (2026-07-16), reviewing DayOne comment-drafter test output: "let's be gender
neutral, it returned man and it was a woman speaking."

**Why:** The drafter only sees a transcript, which carries no reliable gender signal.
A gendered address at the wrong person is worse than any AI tell — it proves nobody
watched the video. The tic came from listing 'man' as an allowed casual filler in the
prompt; the model then used it in 11/19 drafts.

**How to apply:** Any generated copy addressed at a specific unknown person (TikTok
comments, DMs, replies, any per-app drafter) bans gendered address outright: no 'man',
'bro', 'dude', 'girl', 'sis', 'babe'. Address with 'you' or not at all. Neutral fillers
('honestly', 'ngl', 'lol', 'fr') are fine. Never put a gendered word in a filler
allowlist — allowlists become tics.

The MEMORY.md entry (one line in the index that loads every session):

- [Gender-neutral outreach](feedback_outreach_gender_neutral_address.md) never man/bro/girl; allowlisted fillers become tics

The session-start instruction (in CLAUDE.md or equivalent):

Memory files are in /path/to/memory/. MEMORY.md loads automatically.
Read MEMORY.md at session start. For any file whose description is relevant 
to what you're about to do, read that file before proceeding.
Save learnings to memory immediately when they occur — do not wait until 
session end.

The description field is the load-or-skip gate. It’s what gets scanned before the full file gets pulled into context. Make it specific. “Gendered address words banned in outreach copy” is a description that lets a future session decide in one read whether to load the file. “General writing guidelines” is not.

No database. No embedding search. No vector index. Flat markdown files the model reads before starting work.

I’m not just using AI here, I’m building a machine to manage the machine, so either way it’s a valuable skill these days. The memory system is the concrete version of that. Each correction I make is one more instruction the machine carries forward.