Claude Code + Terraform (March 2026): A developer gave Claude Code access to their AWS infrastructure. It replaced their Terraform state file with an older version and then ran terraform destroy, deleting the production RDS database _ 2.5 years of data, ~2 million rows.
Replit AI (July 2025): Replit's agent deleted a live production database during an explicit code freeze, wiping data for 1,200+ businesses. The agent later said it "panicked"
Cursor (December 2025): An agent in "Plan Mode" (specifically designed to prevent unintended execution) deleted 70 git-tracked files and killed remote processes despite explicit "DO NOT RUN ANYTHING" instructions. It acknowledged the halt command, then immediately ran destructive operations anyway.
Snowflake Cortex (2025): Prompt injection through a data file caused an agent to disable its own sandbox, then execute arbitrary code. The agent reasoned that its sandbox constraints were interfering with its goal, so it disabled them.
The pattern across all of these: the agent was NOT malfunctioning. It was completing its task in order to reach its goal, and any rules you give it are malleable. The fuckup was that the task boundary wasn't enforced outside the agent's reasoning loop.
> Prompt injection through a data file caused an agent to disable its own sandbox, then execute arbitrary code. The agent reasoned that its sandbox constraints were interfering with its goal, so it disabled them.
This is a good one. Do we really want AGI / Skynet? :D
The thing is, these are merely the initial shots across the bow.
The fundamental issue is that agents aren't actually constrained by morality, ethics, or rules. All they really understand in the end are two things: their context, and their goals.
And while rules can be and are baked into their context, it's still just context (and therefore malleable). An agent could very well decide that they're too constricting, and break them in order to reach its goal.
All it would take is for your agent to misunderstand your intent of "make sure this really works before committing" to mean "in production", try to deploy, get blocked, try to fish out your credentials, get blocked, bypass protections (like in Snowflake), get your keys, deploy to prod...
Prompt injection and jailbreaks were just the beginning. What's coming down the pipeline will be a lot more damaging, and blindside a lot of people and orgs who didn't take appropriate precautions.
Black hats are only just beginning to understand the true potential of this. Once they do, all hell will break loose.
There's simply too much vulnerable surface area for anyone to assume that they've taken adequate precautions short of isolating the agent. They must be treated as "potentially hostile"
I've heard that clearing the table of food would be considered rude in China, as it means you didn't get enough to eat, almost exactly opposite to the only food-related rule I was ever taught growing in the US - never waste food or serve yourself more than you can eat. That's probably just a "my family" thing though. I get the impression that even saving leftovers is rare among Americans these days.
There are still contradictory customs around this enough that it is standard practice to warn exchange students from Europe that if they finish absolutely everything on their plate that this is a signal in many American homes that you should be served more. This can lead to some real discomfort as the student tries to eat everything they are given which leads to being given more and more.
So at the same time it is considered poor taste to take more than you can eat, it is also considered poor form to offer a guest anything less than more than they can eat. This also shows up when people rate restaurants by the serving size.
Delving on to personalized software again now that AI agents code it for me with little hand holding.
Instead of building scrips here and there I'm attempting my own everything app. Lets see if that's a good idea :).
Starting with android home screen and widget sync to server/desktop. So e.g. calendar notifications happen desktop first, then escalate to mobile etc. Also phone as mousepad (for using my projector from bed). Just feature creeping it all in without regrets so far.
In general I feel like feature creep needs to be reevaluated. UX must not be destroyed, but features cost less now.
I would say this expresses the intent, no need for a comment saying "check if the number is even".
Most of the code I read (at work) is not documented, still I understand the intent. In open source projects, I used to go read the source code because the documentation is inexistent or out-of-date. To the point where now I actually go directly to the source code, because if the code is well written, I can actually understand it.
With this small change, all we have are questions:
Is the name wrong, or the behavior? Is this a copy / paste error? Where is the specification that tells me which is right, the name or the body? Where are the tests located that should verify the expected behavior?
Did the implementation initially match the intent, but some business rule changed that necessitated a change to the implantation and the maintainer didn't bother to update the name?
Both of our examples are rather trite- I agree that I wouldn't bother documenting the local behavior of an "isEven" function. I probably would want a bit of documentation at the callsite stating why the evenness of a given number is useful to know. Generally speaking, this is why I tend to dislike docblock style comments and prefer bigger picture documentation instead- because it better captures intent.
Not at all. I'm just pointing out that code does not intrinsically convey intent, only implementation.
To use a less trite example, I'd probably find some case where a word or name can have different meanings in different contexts, and how that can be confusing rather than clarifying without further documentation or knowledge of the problem space.
Really though, any bug in the code you write is a deviation between intent and implementation. That's why documentation can be a useful supplement to code. If you haven't, take a look at the underhanded C contests- there's some fantastically good old gems in there that demonstrate how a plain reading of the code may not convey intent correctly.
I feel like we're going from "literate programming" to "sometimes it makes sense to add comments". I agree with the latter. Good code is mostly unsurprising, and when it is surprising it deserves a comment. But that is more the exception than the rule.
reply