Stop Letting AI Agents Wreck Your Codebase: Architecture, Memory, and the Dual-Agent Rule

Three sessions into any real project with an AI coding agent, you will hit "Agentic Codebase Rot":

• Agent Amnesia: Every session starts from scratch. The agent burns tokens re-reading the entire repo, forgets previous architectural choices, and silently overwrites working features.
• Assumption Drift: Given the slightest ambiguity, it guesses rather than verifies.
• Comment Noise: Trivial play-by-play comments (// loop over items) pollute your files, while the actual why remains unrecorded.

While building my notebook-styled portfolio (ngmanhtruong.github.io)—a zero-build, 120-FPS web experience with vector drawing and a serverless headless CMS—I tested a disciplined, AI-first workflow inspired by OpenClaw.

Here is how to keep agents fast, precise, and safe.

1. The Three-Tier Memory System

Don't dump everything into a bloated system prompt. Context gets crowded and instructions get diluted. Instead, separate memory into three distinct layers:

1. Universal Conventions (instructions.md)

Project-neutral rules: Organize by business domain (not tech layers), require a 4-question README.md in every folder, and enforce "Comment WHY, not WHAT". Code already shows what it does; only comments explain the non-obvious rationale and trade-offs.

1. Hard Boundaries (CLAUDE.md)

Project-specific constraints: For this repo, zero build steps, vanilla ES6/CSS only, and strict link security (rel="noopener noreferrer").

1. Active Memory (docs/)

• CODEMAP.md: Structure and entry points (eliminates expensive whole-repo scans).
• DECISIONS.md: Condensed Architecture Decision Records (ADRs). The agent must read past decisions before modifying a domain.
• SESSION_LOG.md: Session handoff notes detailing what changed, trade-offs made, and open tasks.

At the start of every session, the agent reads the memory tier first. In 30 seconds, it regains full context without touching thousands of lines of source code.

2. The Human Stays Chief Architect

AI agents have a default instinct: they love dependencies.

Even in a clean repository, their training biases them toward reaching for external npm packages or bundlers whenever a problem gets tough.

When full-screen Retina drawing triggered an ~180MB uncompressed canvas bitmap and live SVG filters choked GPU rendering on fast scrolls, an unconstrained agent would have pulled in an external canvas library.

As the engineer, you hold the line:

• Replace the giant raster canvas with a lightweight vector SVG overlay using midpoint Bézier interpolation (VRAM dropped from 180MB to 0MB).
• Tile procedural SVG turbulence into a 160x160 hardware pattern (<pattern id="paper-tile">) so the GPU composites it once.

The agent brings typing speed; you make the architectural calls.

3. The Dual-Agent Rule for Security

Never ask the agent that wrote the code: "Is this secure?"

LLMs suffer from confirmation bias and will readily hallucinate reasons to justify their own work. We follow a strict Maker-Checker protocol:

• Self-Scoring: The author agent evaluates its own output against clear security criteria.
• The Independent Auditor: A fresh, separate agent—with no context of the authoring prompt—audits the codebase with an adversarial mindset: finding vulnerabilities, inspecting edge cases, and checking failure modes.

This dual-agent separation caught crucial risks early:

• Whitelisting Cloudflare Worker CORS strictly to trusted production origins.
• Sanitizing client-side markdown parsing against XSS (e.g., escaping raw <feTurbulence> tags in technical notes).
• Implementing a multi-tier fallback cascade (Static Snapshot → localStorage → GitHub API → Local Markdown) to prevent public IP rate-limit lockouts.

Key Takeaway

AI coding agents don't replace engineering discipline—they demand more of it.

To ship resilient software with agents: externalize project memory, enforce architectural constraints, and never let an agent grade its own exam.