The UK’s King Charles warns AI leaders of ‘existential dangers’

British monarch highlights AI’s potential darker capacities, urging safety and humanity-focused development.

The United Kingdom’s King Charles has warned Artificial Intelligence (AI) industry leaders of the “existential dangers” posed by the technology if it falls into the wrong hands.

The monarch’s comments on Thursday add to a global debate over whether, and how to regulate the industry.

Charles hosted AI giants, including the cofounders of Nvidia and Google DeepMind, along with leaders from OpenAI and Anthropic at a summit at his Dumfries House estate in Scotland, to address safety concerns before it was “too late” to rein it in.

“There seems urgency in adequately considering the existential dangers of such technologies falling into the wrong hands, and being used in potentially catastrophic ways,” Charles told guests at the opening of the conference.

“Those in our world who value our humanity and its vital moral component are anxiously seeking your reassurance that we will not lose control of our destiny,” he told the industry leaders.

“Those who have created these technologies are now increasingly warning that AI risks developing darker capacities, perhaps even to take life,” or be used in “potentially catastrophic ways”, the king also said.

In recent weeks, global figures have weighed in on whether the industry should regulate itself or allow governments to produce guidelines on how the rapidly evolving technology develops.

In May, Pope Leo XIV said ⁠”the world does not need an artificial intelligence that diminishes humanity” and must be “disarmed”.

Meanwhile, US President Donald Trump has dismissed concerns about the technology as a “hoax”.

Slow the pace of AI development

Last week, two Anthropic researchers warned that the rapidly advancing technology could lead to human extinction in the not-too-distant future.

Anthropic Chief Executive Dario Amodei has also called for companies to slow the pace of AI development.

However, that has prompted an ⁠international debate about whether some executives were trying to get ahead of a possible government backlash and set the rules to keep the lead of particular companies.

“The development of AI – its substance and its pace – are both intriguing and deeply concerning in equal measure,” the British monarch said.

He urged industry leaders to harness it “with safety at its heart” and “to ensure that it remains firmly in the service of humanity, community and the natural world”.

The summit in Scotland is not expected to produce any binding agreements.