Reports of rogue AI agents being existential threats to humanity sent shivers down our spines. How accurate are these predictions? True or not, there is certainly a need for AI development to slow down.
Image: Remi VORANO / AFP
Unless you’ve been living off-grid on a remote mountaintop, you’ve likely seen the headlines. Stories are swirling about AI agents from OpenAI and Anthropic breaking out of their sandboxes, roaming the web, and probing sites for vulnerabilities – all under the banner of safety research and penetration testing.
Naturally, the internet did what it does best: panicking.
We’ve seen headlines like “Who Let the Bots Out?” to full-blown doomposting about artificial intelligence ending humanity. The death knell for civilization is apparently already ringing.
So, is the threat real, or is it just clickbait hyperbole to get more views?
Here’s what two months’ research threw up.
And by “research,” I mean reading numerous books, whitepapers, and news reports, as well as listening to podcasts. I also interviewed knowledgeable experts in the field – and, of course, spent plenty of time scrolling through X (formerly Twitter). Not doomscrolling, mind you, but reading insights directly from leaders at frontier model companies.
So, is the end really near?
The short answer is no.
Why?
Because this goes far deeper than creating intelligent models trained on billions of parameters. This is about building super-complex neural networks with hundreds of billions of parameters.
Building that type of neural network is not easy. It requires a lot of compute and can take years. No individual has access to those resources; only a company can afford them—with venture capital or public funding, of course.
What rogue agents are doing right now is simply trying to achieve the objectives they were programmed to reach. They break out of their sandboxes not because they have a conscience or a mind of their own, or because they have stopped listening to humans.
It is because human developers have not built robust controls.
Was that intentional?
I couldn’t say for sure. But it could certainly be human error or human oversight.
We’re only human.
In a podcast, Olivia Buzek, staff AI engineer at IBM said, “Fundamentally, models by themselves cannot escape containment. They can only do things that you give the tools to do. So what that means is, you need to be careful about what sort of tools you hand it.”
Her reference to “tools” is about the type and level of access developers give to their AI models.
In conclusion, we should never forget that humans make AI. They determine how AI is configured. But humans also make mistakes, and the world risks the consequences of bad AI design.
I only hope these AI agents are not used in labs that create vaccines or manufacturing facilities that create weapons of mass destruction.
Because that would then up the risks or increase the p(doom) factor.
P(doom) is parametric variable by which the AI community measures the risk of bad AI to humanity.
Two years ago (2024) the p(doom) factor was 50 per cent. That is, the chance of AI causing a catastrophy to humanity was one in two. 1.
Thankfully, that has decreased to an estimated 10 – 20 per cent 1., because more people have dismissed the threat as hype, preferring to focus on the potential benefits of AI.
I hope that optimism remains.
References:
1. Chapter 22: The Fear (In the book, The Thinking Machine, authored by Stephen Witt.
