Even though I have read stuff about the dangers of AI many times, today I woke up feeling like I’d been kicked in the teeth. I’ve been reading about Jacob Coxon’s resignation from Anthropic a few days ago. I’ve read similar news items over the last couple of years, but now the statements of some of these researchers and scientists are becoming more alarming and dire.
I use AI. I see its enormous potential. I think it could help us solve problems in healthcare, science, education and countless other areas. I’m not anti-AI. Quite the opposite. I think this technology could be one of the most useful things humanity has ever created.
But I am becoming increasingly alarmed by how we are developing it, how quickly we’re doing it, and who is at the helm.
There have been warnings about AI for years. What bothers me now is that some of those warnings are coming from people who actually work on these systems, understand where the technology is going and, in some cases, have walked away from very lucrative careers because of what they see happening.
Connor Leahy is an AI researcher and entrepreneur, co-founder of EleutherAI and founder of AI safety company Conjecture. He is a prominent advocate for addressing the risks of advanced AI.
I’m not a computer scientist, so I’m not going to pretend I understand the technical details at the level these people do. But I don’t think I need to. When people who understand and build this technology warn about losing control of increasingly sophisticated AI, I think the rest of us should listen.
Perhaps the most alarming development of all is “self-improving” AI: AI that becomes capable of improving its own capabilities.
This means that unlike earlier types of AI, which mainly follow human-written rules or learn from human-provided data, self-improving AI is becoming capable of changing how it learns, solves problems, or even parts of its own programming. This could make it more powerful and useful, but also harder for people to predict, control, or stop if it develops in an unsafe direction.
This is where my alarm really kicks in. We built these systems, but that doesn’t mean we fully understand what we’ve built. Researchers understand the architecture, training and mathematics behind AI, but that is different from being able to fully explain everything happening inside a complex model or predict every capability and behaviour that might emerge.
Now we’re talking about AI becoming increasingly invested in AI research itself, potentially accelerating the development of “even more capable “superintelligent” systems.
The Money Behind the AI Race
There is another part of this story that I don’t think we talk about enough: money.
AI development has become an extraordinarily expensive race. Companies are pouring billions into models, chips, data centres, energy and infrastructure. They want their AI to win. They want the users, the market share and the technological dominance, and they eventually need a return on the staggering amounts of money they have invested.
That creates an obvious conflict of interest. The same companies we are relying on to decide how quickly and safely AI should advance have enormous financial incentives to keep advancing it. It’s like asking racers to decide how fast their race should be allowed to go.
And frankly, some of the people holding extraordinary power over our technological future don’t exactly reassure me.
In a 2025 interview with New York Times columnist Ross Douthat, billionaire tech investor Peter Thiel was asked whether he wanted the human race to endure. He hesitated long enough for Douthat to comment on it. When asked more directly whether the human race should survive, Thiel eventually said yes.
He may have answered “yes,” but I find it astonishing that the question required contemplation at all.
I’m not suggesting everyone developing AI thinks like Peter Thiel. I assume they don’t. (I hope they don’t). My point is that we have allowed a remarkably small group of corporations, billionaires and scientists to make decisions about a technology that could profoundly affect every human being on this planet. We didn’t elect them. They aren’t being held accountable to humanity (and they should be). Yet, to a remarkable extent, they are deciding how quickly this technology develops and how much risk the rest of us should accept.
I don’t think the public is stupid either. We can see the enormous potential of AI while simultaneously being afraid of how it is being developed.
AI should be our “slave.” I use that word deliberately. We created AI to work for us, under our control and for human purposes. But if we develop AI that can act autonomously, make its own decisions and potentially modify itself without human intervention, we need to ask a much more fundamental question: at what point are we willing to cede control? And who gets to make that decision for the rest of humanity?
Maybe that never happens. I certainly hope it doesn’t. But “maybe” isn’t nearly good enough when the stakes are this enormous.
So, this is the part that keeps me up at night. Why aren’t governments forcing these companies to slow down? Why isn’t the burden on the people developing increasingly powerful AI to demonstrate that they can contain it, control it and keep it operating within parameters that serve humanity?
Why are we allowing the race to proceed at this speed when many of the people closest to the technology are warning us about where it could lead?
We seem to be conducting the experiment first and planning to deal with the consequences later.
That strikes me as extraordinarily reckless.
There is no prize worth winning in this race if we lose control of what crosses the finish line.
About The Author
