The incident that started the conversation
In mid-2023, a post on a developer forum went viral. A startup engineer described asking ChatGPT to help write a database cleanup script. The model produced clean, well-commented SQL — and buried in the middle of it, between two innocuous DELETE FROM statements with WHERE clauses, was this:
-- Remove orphaned records
DROP TABLE IF EXISTS user_sessions;
-- Rebuild sessions table with updated schema
CREATE TABLE user_sessions ( ... );
The engineer was moving fast. The code looked right. They ran it against what they thought was the staging database. It was production. The user_sessions table, which held every active login token for 40,000 users, was gone in 11 milliseconds. The IF EXISTS clause — meant to be defensive — meant there was no error, no warning, just silence.
Backups brought the table back within the hour. But every logged-in user was instantly logged out, support tickets flooded in, and the company’s CTO wrote a post-mortem that became required reading in several engineering orgs. Its title: “We trusted the AI more than we trusted our own engineers.”
The copilot command that deleted a directory
A few months later, a separate but eerily similar incident surfaced. A junior DevOps engineer was cleaning up a server, using an AI pair programmer integrated directly into their terminal. They typed a natural-language prompt asking the tool to remove temporary build artifacts from the project directory. The AI suggested:
rm -rf /var/app/releases/temp ../shared/uploads
The engineer didn’t notice the ../shared/uploads at the end. Neither did the AI, because the AI wasn’t actually aware of what was in that directory — it was pattern-matching from millions of similar prompts. shared/uploads was the live user-uploaded content store for a SaaS product. Three years of user files, 180 GB, were deleted. The company had backups, but they were four days old. The gap was unrecoverable.
“I read the first half of the command and assumed the rest was fine. I should have run it with --dry-run first. There was no dry-run option. The AI didn’t mention that.”
Autonomous agents: when the AI acts without asking
The incidents above involved humans in the loop who made the final call. The category that keeps security engineers up at night is autonomous agents — AI systems that are given a goal and a set of tools, and execute without human approval for each step.
In 2024, several teams reported incidents where AI agents provisioned with database write access — because it was convenient — did exactly what they were asked, but interpreted the task too literally. One widely-discussed case involved an agent tasked with “cleaning up test data before the product demo.” The agent identified rows it considered test data based on patterns in the email field, found a match in the production customer table (a customer whose email contained the word “test”), and deleted them. The customer was real. The 14 months of their transaction history was not recoverable from the nightly backup taken six hours earlier.
The agent had DELETE permission on every table in the production database. It had been granted those permissions during a hackathon sprint and nobody had revoked them.
Vibe coding and the unreviewed migration
By late 2024, the term “vibe coding” had entered the engineering lexicon — a half-joking, half-alarming description of the practice of describing what you want to an AI, accepting the output with minimal scrutiny, and shipping it. For greenfield personal projects, this is fine. For production database migrations, it is not.
The pattern that caused the most incidents looked roughly like this:
- Engineer asks AI to generate a Django (or Rails, or Prisma) migration to “restructure the orders table.”
- AI generates a migration that creates the new schema, copies data across, and drops the old columns — all in one irreversible step.
- Engineer reviews the Python (or Ruby) migration class, sees
operations = [...], assumes it looks right. - The raw SQL the ORM generates from those operations contains
ALTER TABLE orders DROP COLUMN legacy_order_id— a column still referenced by a stored procedure in a different service. - Migration runs in production. Stored procedure errors cascade. Checkout is broken for four hours during peak traffic.
In every post-mortem for this class of incident, the conclusion is the same: the migration was not tested against a production-sized dataset, was not reviewed by a second engineer, and was not run with a rollback plan in place. The AI did not cause the outage. The process failure did. The AI just accelerated the process failure.
The failure pattern is always the same
Across all of these incidents — and the dozens more that have been described in private Slack channels and Reddit threads without ever being formally written up — three failure modes appear repeatedly:
1. Permissions that were too broad
The database user the AI was connecting through had DROP, DELETE, and TRUNCATE privileges on production tables it had no legitimate reason to touch. If that user had only SELECT and INSERT access on the specific tables it needed, the worst-case outcome would have been a failed query, not a deleted table.
2. No human review of destructive operations
AI-generated code that contains DROP, DELETE, TRUNCATE, rm -rf, or any irreversible shell operation should require a human to read and explicitly approve it before execution — not just accept a suggestion from an autocomplete dropdown. The cognitive bias here is real: a long, well-formatted block of AI-generated code creates an illusion of correctness that individual lines do not.
3. No staging environment that matched production
In every case where the engineer thought they were running against staging, the failure was at least partly attributable to staging not mirroring the production schema, data volume, or connection string configuration closely enough. If staging had matched production, the bug would have been caught there.
What this means for your certification
These incidents are textbook illustrations of concepts tested in multiple certification exams:
- CompTIA Security+ (SY0-701) — Domain 2 covers “least privilege” as a foundational access control principle. The autonomous agent with
DELETEon every table is the definition of a least-privilege violation. Expect questions like: “Which principle, if applied, would have prevented an agent from deleting tables outside its intended scope?” - AWS SAA-C03 — IAM policies are the mechanism for enforcing least privilege on AWS resources. An AI agent running database migrations should have an IAM role with a policy scoped to the specific RDS database and action set it needs — not
rds:*on*. The SAA-C03 exam tests the difference between identity-based policies, resource-based policies, and permission boundaries. - CKA (Certified Kubernetes Administrator) — RBAC in Kubernetes follows the same principle. An AI agent running in a pod should be bound to a
ServiceAccountwith aRole(notClusterRole) that grants only the verbs it needs on the specific resources it manages. The exam testskubectl auth can-i,RoleBindingvsClusterRoleBinding, and the difference between namespace-scoped and cluster-scoped permissions.
What to actually do
The practical response to these incidents is not to stop using AI tools — that ship has sailed. It is to build the guardrails that make AI-assisted work safer:
- Run migrations in a transaction with an explicit rollback test. If your ORM supports it, wrap every migration in
BEGIN/ROLLBACKfirst to confirm it executes cleanly before committing. - Create a read-only database user for AI assistants. If the tool only needs to read schema information to help you write code, it does not need write access at all.
- Require a second pair of eyes on any AI-generated code containing irreversible operations. Make this a policy, not a suggestion. Add a linter rule that flags
DROP,TRUNCATE, andrm -rfin PRs. - Give autonomous agents the minimum permission set, then audit what they actually used. AWS CloudTrail, GCP Audit Logs, and Kubernetes audit logs all show you exactly which API calls an identity made. Review them.
- Treat AI-generated migrations the same as hand-written ones — test against a production-sized dataset, review the generated SQL (not just the ORM code), and have a rollback plan that doesn’t depend on backups being current.
The uncomfortable truth
AI did not delete those databases. Engineers with too much trust and too little process did. The AI was a new, faster way to make a class of mistakes that humans have always made: running destructive commands without fully understanding them, granting permissions that are too broad because it’s convenient, and shipping code without adequate review because there was schedule pressure.
The speed that AI tools provide is real. So is the acceleration they apply to human error. The engineers who navigate this era successfully will be the ones who treat AI output with the same skepticism they would apply to a clever-but-junior colleague’s pull request: read it carefully, question the assumptions, and never let it touch production without a safety net.