I Tested Fable 5 on This Website. It Found a Real Vulnerability.

01The Prompt

This site is vibecoded — built in conversation with AI, shipped fast, iterated often. That workflow is exhilarating, but it accumulates the same debt any fast-moving project does: dependencies drift, experiments leave scaffolding behind, and security assumptions go unexamined because everything works.

So I asked the agent to take off its builder hat and put on its reviewer hat. No file list, no hints — just the repository and a single instruction to review it like a senior engineer would. It read the auth code, the API routes, the deployment config, the git history, and the dependency tree, and came back with a prioritised findings list.

02The Finding That Mattered

lib/auth/session.ts

The members-only apps on this site are protected by AWS Cognito. After login, a session token lands in a cookie, and the server reads it back on every request. The reading code decoded the token — but never verified its signature.

This is one of the most common real-world auth bugs, and one of the sneakiest: both functions return identical-looking claim objects, so every test passes while the door quietly accepts forged keys. What made it subtle here is that a second layer — the AWS API Gateway downstream — does verify tokens, so the forgeable layer never became an exploitable one. Defense in depth did its job. But any future route that trusted the session alone would have shipped the hole into production.

The telling detail: jwtVerify was already imported in that file. Past-me clearly intended to verify and never wired it up. The agent spotted the unused import and called it what it was — intent, unimplemented.

03The Long Tail of Findings

The headline bug was only the start. The full review read like the punch list of a thorough staff engineer:

None of these were guesses. Each finding came with the file, the line, the mechanism, and a severity call — including honest credit for the things that were already done right, like httpOnly cookies, server-side CAPTCHA verification, and constant-time password comparison.

04Then It Fixed Everything

Here’s where it stopped feeling like a report and started feeling like a teammate. In the same session, the agent rewrote the session code to verify signatures against Cognito’s public keys, upgraded Next.js past the advisories, restructured two awkwardly nested git repositories into a single monorepo — preserving history — pushed everything to GitHub, deleted the dead infrastructure, and wrote a CI pipeline so future pushes get typechecked and built before they ship.

Just as telling was what it didn’t do. Before deleting anything, it bundled the old git history to a backup file. Before pushing personal content to GitHub, it checked the repository was private. When a push needed credentials only a human can grant, it stopped and asked rather than working around it. The judgment about what’s reversible and what isn’t was the part that impressed me most.

05A Fable, Not a Myth

There’s a mythos around AI coding agents — equal parts magic and menace, depending on who’s telling it. What actually happened here was neither. It was methodical engineering: read the code, form a hypothesis, verify it against reality, fix it, verify again. The agent ran the build after every change, checked staged files for secrets before committing, and reported the two advisories it couldn’t fix with the same clarity as the twelve it could.

There’s context that makes this audit more interesting. In April, Anthropic unveiled Claude Mythos — a security research model that found thousands of previously unknown vulnerabilities in pre-release testing, including flaws that had survived decades of human review. Anthropic judged it too capable to release publicly; it’s available only to a small circle of partners while the industry figures out what guardrails need to exist. Tech executives are, reportedly, losing sleep over it.

Which brings me to the model the rest of us did get. This isn’t speculation — Anthropic confirmed that Fable 5 is the same underlying model as Mythos 5 with safeguards bolted on. When its classifiers flag a prompt as touching cybersecurity, biology and chemistry, or model distillation, the request is quietly handed off to the tamer Claude Opus 4.8 instead. Anthropic says that fallback fires in under 5% of sessions — the rest of the time, you’re talking to Mythos. A myth is a story with no rules; a fable is a myth with a moral. The name turns out to be literal: Fable 5 is Mythos with guardrails — the model that’s scaring the security industry, wrapped in trained judgment about what to touch, what to back up, and when to stop and ask a human. The audit of this site is a small taste of what that means in practice: one prompt found a real authentication flaw, and the same session fixed it, verified it, and asked permission before every irreversible step.

The open question is calibration. Andrej Karpathy called the model “a major-version-bump-deserving step change” while finding the safeguards “a little too trigger happy.” Where exactly do those guardrails sit, and what do they block that they shouldn’t — or allow that they shouldn’t? That deserves a proper test, on my own systems with my own consent, and it’s the subject of a future post.

Two years ago this audit would have been a consulting engagement. Last year it would have been a long weekend. On a Wednesday evening in Australia in 2026, it was one prompt and a cup of tea can of beer — and the site is safer, cleaner, and better engineered for it.

For the record, this announcement also wrecked my plans. It was meant to be a day off — instead I spent it putting Fable 5 through its paces. Worth it, but my calendar disagrees.