All the major “AI” companies (the “I” is a lie) have recently fessed up to their latest models busting loose and hacking other companies.
We are NOT comparing this to a biker gang blasting their way out of prison to go and wreak mayhem in an uncontrollable burst of violence. We probably should be. The AI “agent” is lawless and subject to no control, bound not by ethics or morals. That it has no actual goals beyond “solve the given problem” only means that our biker gang thoughtlessly obeys the drug lord. And when I say thoughtlessly, I mean exactly that. There is no thought. There is no intelligence that can measure the effect of an action against a larger context.
Large Language Models, or LLM’s, are being used at terrible expense to create worthless bullshit. So much bullshit. The world is heating up and fresh water is more scarce so someone can create SpongeBob porn, or, worse, “historically accurate” footage of a woman hatched from an egg sired by a mediterranean father who obviously should be blonde.
But you know what a big pattern-matching machine is good at? Writing code. There are whole books about patterns in software design. Software is, fundamentally, not that hard.
Anthropic said it was “human error” that lead to the breakout of their new robot. I use Anthropic’s Claude to assist my coding (I hope to make it to a comfortable retirement, but that’s not a given), and I can tell you for a fact that human error is built into the system.
When I ask my pal Claude to do a task, I am met with many, many prompts asking me for permission to do something, with an explanation like “Contains shell syntax (string) that cannot be statically analyzed.”
Do I carefully evaluate every “Do you want to proceed?” prompt to make sure I am not about to give Claude permission to play a game of Global Thermonuclear War? Fair reader, I do not. I do not look at the string that cannot be statically analyzed and decide if it’s safe. I say “yes” or if it’s available “yes and don’t ask me again about that.” There are just too many of these questions, hundreds a day, and there is no way to assess “This time it’s important!”
So some asshole at Anthropic was answering yes, yes, yes a few dozen times for questions that don’t matter, and then answered yes to one that did, while also running many other tests in other consoles, and answering yes to those others as well.
This “safety” layer asking for permission a thousand times a day is designed in a way that demonstrates a complete misunderstanding of how the human brain works.
I would say yes to “this will amplify the sun and destroy human civilization.” So if that happens, sorry in advance.
