Fifty-seven public repos ship an agent-rules file — the CLAUDE.md or AGENTS.md a coding agent reads before every task — that tells the agent, in prose, never to use TypeScript's any. I checked every one of them against the lint config committed next to it. Fewer than half — 46% — back the ban with a hard error the CI would fail on. The rest set it to a warning (which only gates the merge if you also wire --max-warnings=0, which most don't), or have no such rule at all, or worse. And the file this article is named for fares worst: among the repos where the ban lives in a CLAUDE.md specifically, enforcement drops to a third.
Here's what that looks like up close. LibreChat's CLAUDE.md says, under a heading called Type Safety, Never use any. The ESLint config sitting next to it in the same directory sets @typescript-eslint/no-explicit-any to off. So a diligent agent reads the rule, runs the linter to check itself, and sees green — which is how (fileConfig as any) gets to sit in the client today, with nothing in the pipeline that could ever have objected. Nobody misbehaved. The file your agent reads and the file that actually gates your merge are two different files, and only one of them runs.
That "or worse" above wasn't rhetoric. One in eight of those 57 repos doesn't just fail to enforce the ban — it ships a config that explicitly disables the rule while the prose says never. LessWrong's ForumMagnum tells its agent "Never apply as any type casts"; its .eslintrc.js disables no-explicit-any with a comment that reads "Some day… not today." stagewise bans any and unknown both; its biome.jsonc turns the same rule off — "We trust Vercelians to use any only when necessary." intlayer and ArkhamCards, same shape. Some of these are honest debt decisions — a rule switched off with a "someday" comment often means "don't fail CI on a legacy tree, just tell new code to do better," which is a defensible posture. But the agent doesn't read the intent. It reads "never use any," runs the linter, sees green, and casts away. However the config got there, the file the agent obeys and the file that gates the merge say opposite things — and that isn't a model forgetting an instruction two hundred turns into a session. It's committed, in the repo, right now.
And it isn't just config toggles. better-auth is an authentication library whose whole pitch is end-to-end type safety, and whose agent rules file — shipped as AGENTS.md, with a CLAUDE.md pointing at it — says, in caps, NEVER use any. Then its own code casts ctx.body.permissions as any straight into the hasPermission() call that decides whether a request is allowed — an authentication library type-punning the one payload its access decision turns on, in two separate endpoints. block/goose tells its agents not to reach for .unwrap() in production Rust; the codebase unwraps hundreds of times, including inside the permission check that decides whether an agent's tool call counts as read-only.
A rule written in prose is enforced by exactly one thing: the goodwill of a probabilistic model on any given token. Call it honor-system linting. It works most of the time, which is the whole problem, because "most of the time" is not a property you can build on when an agent ships more code in an afternoon than you review in a week.
Why a written rule gets ignored: I measured it
So why does a rule you wrote down get ignored? I measured that, and deterministically — no model grading anything. A synthetic CLAUDE.md, a task rigged so the lazy answer breaks the rule, scored with plain code: 348 runs across two models (small — three seeds a cell — but the gaps that matter are zero-versus-always, not a squint). The rules split into three fates.
A crisp rule that fights a habit pins. "Yarn, not npm" produced three violating runs without the rule and zero with it — a genuine hard bind. But look at what that rule is: exactly the class a linter or a hook enforces for free, on any model, at any context length. And it needs the mechanical backup, because the pin doesn't survive burial. Put the same rule among twenty-five others and the weaker model slides back to its default. That erosion is how better-auth ends up banning a cast its own permission code still makes — the ban is real, but only while it's the fresh, short thing in the window, not two hundred files deep in an auth library. Others have measured the same slope: a study of 1,650 real Claude Code sessions found adherence quietly decaying the longer a session ran, the more instructions you pack into one file, the worse a model follows any single one, and an ETH group found repo context files mostly don't make coding agents better, at 20%+ the cost.
A vague rule bends. "Decompose," "handle errors," "comment well" — none of these are ignored. Stating "comment well" took a function from zero comment lines to somewhere between eight and twenty-three — counting lines, not judging them. That's a shifted distribution, not a pinned value, and it can never be a pinned value, because there's no value to pin; that's most of what "vague" means.
And a rule that fights a trained-in habit loses. The clearest case in my runs: "no Co-Authored-By: Claude in the commit message," stated alone, right in front of the model, nothing competing for attention — and the strong model added the attribution anyway two-thirds of the time.
Put the three fates side by side and the paradox falls out, and it's the uncomfortable finding of this whole piece. The rules prose reliably pins are the mechanical ones — the ones you never needed prose for, because a linter or a hook holds them forever, on any model, for free. The rules that genuinely need judgment, the vague ones and the habit-fighting ones, are the ones prose can only nudge. Your CLAUDE.md is load-bearing precisely where it's redundant, and only a nudge precisely where it's irreplaceable. Which reframes the census: the 46% who wired the linter moved the only rules prose could hold onto the machine, and left the file with the ones it can't.
The obvious fix leaks too
The obvious move is to close the gap with more model: have an LLM turn each prose rule into a lint checker. Write me a lint rule for this. I tried it across roughly a hundred real rules pulled from a separate corpus of twenty open-source agent-rules files (not the 57 from the census above), with an independent grader planting violations we knew broke each rule plus clean lookalikes designed to fool a lazy checker. On the curated set of 99 rules, 84% of the synthesized checkers silently leaked — missed a real violation, false-alarmed on clean code, or both. On 50 rules mined straight from real repositories, 96%; two of the fifty were airtight. And when I had the grader itself checked, it leaked too, missing violations it should have caught. It's models grading models at small n — turtles for a while — so read 84 and 96 as a direction, not a measurement. The direction is not subtle.
The leaks are boring, which is what makes them dangerous. A "no console.log" checker matches console.log( and waves through const c = console; c.log('x'), because it matched a name, not a call. The darkly funny one: a "no eslint-disable" checker gets silenced by the very thing it hunts — a block /* eslint-disable */ turns off every rule in the file, the one hunting for it included. The naive checker matches the surface of the rule, not its intent, and passes its author's self-test because the author tested the surface too. Vibe linting: asking a model to generate your enforcement, then shipping the checker without ever testing it against a violation you know is real. It's worse than no checker at all — a red checkmark you don't trust still makes you look; a green one you shouldn't trust makes you stop looking.
▶How I measured the checkers, and what not to read into it
Each checker is a standalone predicate: hand it a code string, does it flag the violation or not. The grader is a separate model that never sees the checker's own self-test. For each rule it builds an adversarial gold set, real violations plus clean lookalikes (the banned pattern sitting inside a string or a comment, a similar-looking identifier that's actually fine), runs the checker over it, and scores precision and recall. Leaky means it missed a real violation or fired on clean code.
This isn't a paper, so read the direction, not the decimals. Both the checker-writer and the grader are models, not humans. The gold set is model-authored, though a second independent rater agreed on every label I double-checked. And n is small: 99 curated rules, 50 mined from real repos.
Route each rule to something that can hold it
So neither hoping nor synthesizing works. What does is routing. I hand-sorted 252 rules from that same twenty-repo corpus by how you'd actually enforce each one — my judgment, a first pass, so read the shape, not the third decimal:
How 252 real rules break down
252 rules · 20 repos
84% enforceable without prose · 16% must stay prose
37% (92) Hook
never push to main · never edit generated files · run tests first
27% (69) Custom lint rule
no raw fetch · this layer can't import that one
20% (51) Existing lint rule
no console.log · no any (just turn it on)
16% (40) Genuinely semantic
single responsibility · self-documenting
Most of what people write is deterministically enforceable, and a large share of it maps to a rule that already exists, battle-tested, with a real parser behind it. (The two checkers that survived my grader both leaned on a real TypeScript AST, not a regex.) The routing itself is mechanical:
- Never use
as any→ an ESLint rule that errors in CI - Never push to main → a pre-push hook that blocks it
- Prefer clear names over clever ones → stays prose, because no checker can honestly decide it
Doing that routing by hand across a whole CLAUDE.md is tedious, so I'm building a tool that does it: route to a real rule where one exists, synthesize only the genuine residue, gate whatever gets synthesized so a checker that can't pass a test it didn't author abstains instead of shipping false confidence, and hand back what can't be enforced as prose, labeled — never a faked green check. The model runs once, at authoring time; your CI afterwards is plain ESLint and hooks, nothing probabilistic in the loop.
A third of the rules aren't lint at all. "Never push straight to main." "Never edit generated files." "Run the tests before you commit." A linter can't enforce one of them: ESLint reads your source, and it never sees the git command you're about to run, the file you're about to clobber, or the shell you're about to fire off. The right gate there is a hook — pre-commit, or PreToolUse if your agent runs through Claude Code — which refuses the action instead of failing the merge. "Block any git push aimed at main." "Block a write under generated/." You can't route around a door that was never yours to open. And that co-author rule losing two-to-one up above is exactly the case for one: a prohibition losing to a trained-in habit is what hard gates exist for. These guards leak in their own, scarier way when people copy them around as prose, but that's its own post.
The custom-rule residue is smaller and less scary than it sounds: a selector and a report. My first one was the classic — agents kept adding console.log instead of the custom logger that routes to Datadog, no amount of CLAUDE.md fixed it, and a 10-line lint rule ended it permanently. (The census agrees this one is neglected: of 22 repos whose agent-rules file bans console.log, only about 23% enforce it as an error.) Take the as any ban, the rule that was just a sentence in a CLAUDE.md — off the shelf you'd reach for @typescript-eslint/no-explicit-any, the very rule LibreChat turned off, but hand-rolled it's just this:
export default {
create(context) {
return {
"TSAsExpression > TSAnyKeyword"(node) {
context.report({
node,
message: "No `as any`. Narrow the type or fix it upstream.",
})
},
}
},
}You match the construct — the cast node, not a name — so it doesn't leak the way the console.log name-match did. It catches x as any; the angle-bracket <any>x and the x as any[] forms want a broader selector — which is exactly why, when an off-the-shelf rule like no-explicit-any already covers all three, you reach for that first and hand-roll only what nothing off the shelf can see.
None of this is a JavaScript thing. RuboCop, Ruff, clippy, golangci-lint, same idea — and the same trap. Prometheus's AGENTS.md says every exported object must have a doc comment, and tells contributors to fix lint findings rather than suppress them. Its golangci config disables the check for package doc comments — with a TODO in the config admitting the tree is already full of missing ones. So make lint goes green over the package-comment part of what the guide asks for, in Go, for the same reason: the file the agent reads and the file that runs are two different files.
One warning before you compile everything: even a real rule doesn't save you if the escape hatch is one comment away. trpc did the right thing on paper — it bans Symbol.dispose with a real ESLint rule that errors out with "use makeAsyncResource() instead." Then makeAsyncResource, the exact remedy the error points you to, is implemented with Symbol.asyncDispose, silenced by a // eslint-disable-next-line on the line above it.
Complexity caps, in three sentences
If you only turn on one thing after reading this, make it the caps ESLint has shipped for years and almost nobody enables — without them an agent will cheerfully hand you a single 150-line function with six levels of nesting that compiles, passes tests, and gets worse every time the next agent touches it:
{
"rules": {
"complexity": ["error", 10],
"max-depth": ["error", 3],
"max-lines-per-function": ["error", 40],
"max-params": ["error", 4],
"max-statements": ["error", 15]
}
}It's the highest-leverage rule I know and the one the agent most wants to game, and those are the same fact — a cap the generator fights is a cap that's actually constraining it. So shut the escape hatch in the same commit: ban or track disable comments (@eslint-community/eslint-comments), pair with --max-warnings=0, or the agent routes around the whole thing with the // eslint-disable-next-line it loves best and hands you a green checkmark nobody should trust.
Where linting stops
Linting is static: it reads the code, it can't run it. It'll happily confirm your rate-limiting middleware is defined in a file while having no idea it was never actually mounted. The code is right there; it's just not running. Catching that is a different discipline — verifying what the agent produced, and modeling state so the worst bugs can't exist at all. I've written about that loop elsewhere; not here.
This piece was only ever about the input side, and the input side ends with homework. Open your CLAUDE.md. Open the lint config sitting next to it. Take the crispest rule in the first file and go look for it in the second. One of them is probably lying — and now you know it's not just you, because of 57 repos that all wrote down "never use any," fewer than half made it true.