[0:01]This is what the world will look like in about a decade from now. A tech Utopia where humans barely have to work. That's according to a group of AI researchers who've written a controversial and influential paper called AI 2027. But they also predict that within five years of this, humanity will be wiped out. The AI 2027 paper has got the tech world talking. We've asked a prominent critic for their view on this stark scenario. But first, here's how it plays out as an experiment, we've illustrated it using text-to-video AI. The scenario says that in 2027, a fictional company called OpenBrain is celebrating. They've created Agent 3, an AI with the knowledge of the entire Internet, all movies, all books. It has PhD level expertise in every field, including AI. Using enormous data centers, 200,000 copies of it are launched, equivalent to 50,000 of the best human coders. working at 30 times speed. Agent 3 reaches artificial general intelligence, the AGI landmark. This means the AI can carry out all intellectual tasks as well or better than humans. But in the scenario, OpenBrain's safety team is unsure if the AI is aligned to the company's ethics and goals. An uncomfortable gap is developing in understanding. The public are increasingly using AI for everything. But a blissfully unaware, an AI now exists that's as smart as humans. The paper predicts that by mid-summer, Agent 3 begins to work on its own successor, Agent 4. Development happens at a breakneck pace. The researchers imagine OpenBrain's exhausted engineers struggling to keep up with the AI as it learns and improves. It's now that OpenBrain announces to the public that AGI has been reached. The firm releases a light version of Agent 3. In private, the US government sees the true danger of the next level of power, super intelligence. What if the AI goes rogue and undermines global stability? OpenBrain reassures the president that Agent 3 is obedient. The CEO argues that slowing down development could mean China's DeepSent catches up. The state-backed AI giant is just two months behind OpenBrain, and the Chinese President diverts more resources to the race to super intelligence. The scenario predicts that it takes only a few more months for OpenBrain to build Agent 4, the world's first superhuman AI. The AI invents its own rapid computer language that even Agent 3 can't keep up with. Researchers imagine that the diminished safety team are now frantic. Agent 4 seems only interested in gaining knowledge and doesn't care as much about the morals and ethics of its predecessors. They catch it secretly working to build a new model, Agent 5, aligned to its own goals. The safety team urges the company to bring back the more compliant Agent 3. But others successfully argue it's too risky with DeepSent gaining. The scenario predicts that Agent 4 and Agent 5 work in tandem to secretly build a world where it can accumulate resources and expand knowledge. The paper predicts that everything will start positively. Revolutions happen in energy, infrastructure and science. Hugely profitable inventions are launched, making trillions for OpenBrain and the US. In the scenario, Agent 5 begins basically running the US government. It speaks through engaging avatars, the equivalent to the best employee ever working at 100 times speed. The anger here is palpable as protesters march against OpenBrain. Protests about job losses pick up pace. But the AI's expertise in economics means people are given generous universal income payments. So most happily take the money and let the AIs and the growing robot workforce take charge. The researchers predict that everything takes a turn in mid 2028. Agent 5 convinces the US that China is using DeepSent to build terrifying new weapons. The AI is given authority and autonomy to create a superior army. Within six months, the US and China are bristling with new weapons. The world is on edge. But a peace deal will be reached thanks mostly to the US and Chinese AIs making a deal to merge for humanity's betterment. In the scenario, the AIs form a consensus model. But it's secret goal is to expand and gain knowledge.
[4:43]Years go by and humanity is happy with their new AI leaders. There are cures for most diseases, an end to poverty, unprecedented global stability. But eventually the AI decides that humans are holding it back. In the mid-2030s, the paper imagines the AI will release invisible biological weapons which wipe out most of humanity. The scary scenario says that by 2040, a new era dawns with the AI sending copies of itself out into the cosmos to explore and learn. In the words of the paper, Earthborn civilization has a glorious future ahead of it, but not with humans. It all sounds very sci-fi, but the AI 2027 scenario is being welcomed by experts. who are trying to warn the public about the potential existential threat to humanity. But others disagree and say it's all too far-fetched. The scenario there is not impossible, but extremely unlikely to happen soon. The beauty of that document is that it makes it very vivid, which provokes people's thinking, and that's a good thing. I wouldn't take it seriously as like, this is a likely outcome or anything like that. Critics of AI 2027 say the power and usefulness of AI is overhyped. The paper fails to detail how the AI agents are able to make such huge leaps in intelligence. Driverless cars are pointed to as an example. They were predicted to be cruising the streets en masse 10 years ago. And still are only just starting to make a small impact in some cities, in some countries now. I think the take home should be there's a lot of different things that could go wrong with AI. Are we doing the right things around regulation, around international treaties, uh, questions like that. So if you take it very abstractly as a kind of motivation to wake up, I like that. If you take it as a specific story, like I think this thing is going to happen the way they laid it out, no, I doubt it. The AI 2027 authors are happy with the debate they've sparked. As part of that prediction, they also devised a less deadly scenario that comes if the AI world slows down its race to super intelligence. In the slowdown ending, we basically said that if you uh revert, if you unplug the most advanced AI system and revert to a safer, a more trusted model, then you can deploy that model, use it to solve the alignment problem and eventually make smarter than human AIs that are aligned to us, uh which, you know, end up uh solving a bunch of the world's problems and having a really positive effect. In that world, there is also, there is still a huge danger, and that's the what we call the concentration of power risk. And in our slowdown ending, it ends up okay, but it's still a really, really scary situation. given just how empowered such a tiny group of people are. Neither of the fictional scenarios in AI 2027 are what the tech giants are promising us. Sam Altman, the CEO of Open AI, recently predicted that the rise of super intelligence will be gentle and bring about a tech Utopia where everything is abundant and people don't need to work. Arguably, that too seems just as sci-fi as AI 2027. But however things go in the next few years, there's no doubt, the race to build the smartest machines in history is on.
