'AI is the stuff of nightmares'

I LOOK out of my window and all appears as usual.

People walking and cycling by, looking cheerful and chatting, pushchairs and runners, children letting off steam. A postman going from door to door, sunlight flickering through the leaves of a tree across the road.

In Cardiff, Plaid Cymru, the Scottish National Party and Sinn Féin sign a memorandum of understanding expressing a shared commitment to independence from the UK for Wales and Scotland, and reunification for Northern Ireland with the Republic of Ireland.

Pictures show first ministers Rhun ap Iorwerth, Michelle O’Neill and John Swinney looking buoyant.

People in the street steeped in normality, and politicians pushing for radical change, share an assumption. Which is that, perhaps with adjustments, personal and national, life will go on. That there will be a future. That we will be able to go on enjoying this wonderful planet, that there will be a world for children and grandchildren to inherit and to bask in.

They will of course see worldwide problems continuing. But, for a majority, there will be a shared assumption of human survival.

Now, rather suddenly, artificial intelligence corporations in San Francisco come up with dire warnings of a different future. Of no future.

Three researchers with the industry giant Anthropic, one of whom has quit his job in protest, are warning that AI could kill us all off within 10 years if left unchecked.

That needs more than a moment or two to sink in.

In posts on social media, researcher Jacob Coxon says he resigned because Anthropic and his previous employer, OpenAI, were ignoring or, at best, mishandling their response to the threat.

“Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives,” Coxon wrote.

“The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible – but I hear the same people express fear privately. No other human activity poses this level of danger.”

Two other Anthropic employees endorsed Coxon’s predictions. One, Evan Hubinger, a lead in the company’s alignment division, which works on ensuring Anthropic’s AI models work in line with human goals, said his former colleague was “correct”, and that the industry was falling behind in attempts to deal with the apocalyptic potential.

He wrote: “We really do earnestly believe AI could kill all humans! I personally think (the chance of that happening) is more than 10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.”

Rival technology leaders Elon Musk and OpenAI head Sam Altman later backed a call from the head of Anthropic, Dario Amodei, to “slow the pace” of AI development, after he warned that an AI “swarm” could otherwise become capable of “taking over the entire internet” within a year.

Reflecting on the repercussions of that is the stuff of nightmares.

Amodei also called for safety coordination with China. Yet even if America and China worked cooperatively to make the wiliest AI safe, that would not ensure a global taming of the technology. We’re looking at fears for the survival of humanity. That being the case, Russia, and all other countries wedded to AI, should be involved in the drawing up of any safety code. The need is for a global rebellion against extinction, regardless of politics. And more fool any countries that don’t think that way.

Some people will doubt the seriousness or authenticity of the situation. They should be directed to a statement last week by Anthropic. This said the company had identified five cases in which attempts had been made to use its models to back the development of biological weapons.

Rogue-AI attacks on nuclear or other power systems, or AI-enabled pandemics, might also, presumably, be possibilities.

Nigel Shadbolt, an Oxford computer scientist, tells the BBC that laws and regulation are essential, that it’s no good simply telling an AI not to harm people. He believes a capable AI agent pursuing another objective might reinterpret that directive, satisfying it formally while concealing its conduct or finding a way around it.

But he also thinks a ‘Hippocratic oath’ for AI would fail in systems designed to identify, target or kill people, meaning that governments must not create a military exemption for AI. Some hope of agreement on that, you may think. Perhaps it all depends on the strength of the self-preservation instinct.

Certainly, sustained public pressure aimed at tight ‘alignment’ of AI must be a way forward. As must be the breaking of ranks exampled by Jacob Coxon and colleagues.

At the same time, we shouldn’t be swayed by counter-arguments apparently strongly supporting AI because it might find a cure for cancer, or sort out climate change. The priority must be to eradicate AI’s fatal Frankenstein tendency. Unless that’s done, there might be no-one left to benefit from such achievements.

Given a more cautious approach to an unpredictable technology, the anxiety now foisted on us was avoidable, and it’s hard not to feel badly towards the Californian operatives who have landed us in this mess.

They appear as technology’s equivalent of a closed monastic order totally absorbed in a single act of devotion. Yet, if the unthinkable happened, they would presumably perish along with the rest of us, their narrow pre-eminence no longer recognised or applauded, their consequent earnings unspent.

In the immediate term, and bearing in mind the spectre of a potential internet takeover by rogue AI, households in rural Wales lucky enough to still have landline telephones may think it a very good idea to hang onto them. And to resist any pressure from phone companies to accept substitute internet-dependent devices. Remembering, in any case, that such phones are automatically knocked out by internet failures, such as those caused by storms, rendering them useless in an emergency, as quite a few people have discovered.