SAN FRANCISCO — The tech industry is spending billions of dollars in a drive to upgrade today’s artificial intelligence tools into so-called superintelligence, a fuzzy term used by companies like Meta and OpenAI to invoke future software they hope will outperform humans in every way. On Wednesday, a loose coalition of Nobel laureates, policymakers, celebrities and British royalty said that work on the hypothetical technology must be halted.

“We call for a prohibition on the development of superintelligence,” said a public statement with more than 700 signatories, including prominent AI researchers, Prince Harry, former Trump adviser Steve Bannon and the Obama administration’s national security adviser Susan Rice. The ban should be lifted only after “strong public buy-in” and scientific consensus that the technology can be made safe, the statement said, without specifying how either could be measured.

Most AI experts in industry and academia do not think such technology is imminent, despite moves like Meta CEO Mark Zuckerberg spending about $15 billion to launch a new “superintelligence lab” this summer. Tech firms and national governments are competing to speed up AI development, making any ban unlikely.

But the statement, coordinated by the nonprofit Future of Life Institute, comes at a time of renewed debate in Washington, Silicon Valley and across the nation about whether the potentially disruptive effects of existing AI technology require new regulations.

Advertise with us

Half of Americans are more concerned about AI than they are excited about it, according to a spring survey by the Pew Research Center. A Gallup poll from around the same time found that 80 percent of Americans supported rules for AI even if they slowed down its technological development.

The Trump administration has aligned itself with tech figures who favor minimal regulation, rolling back AI rules introduced by President Joe Biden and supporting a proposed federal moratorium on state AI regulation.

But federal and state lawmakers have launched dozens of efforts to regulate AI technology. More than half of U.S. states have passed AI-related laws, creating a patchwork of regulations across the country.

In response, Meta and venture capital firm Andreessen Horowitz in recent months set up political action committees to support candidates that oppose strict AI regulation.

“The state of AI regulation is rapidly evolving. As public use of AI tools surges, policymakers are digging in,” said Katy Milner, a partner at law firm Hogan Lovells who works with technology companies. “Congress and the states have introduced hundreds of AI-related bills on issues such as data privacy and government use, with a few already enacted and many more bills likely to follow.”

Advertise with us

Few of those bills are likely to focus on the kind of speculative AI technology targeted in Wednesday’s statement released by the Future of Life Institute.

The nonprofit organized an open letter in 2023 in which some business leaders and AI researchers called for a six-month “pause” on the training of more capable AI, but that did not appear to slow down the tech industry. And the 2023 letter's most prominent signatory, Elon Musk, was at the same time quietly founding his own AI startup to compete with those he wanted to take a 6-month pause.

“Many people want powerful AI tools for science, medicine, productivity and other benefits. But the path AI corporations are taking, of racing toward smarter-than-human AI that is designed to replace people, is wildly out of step with what the public wants,” said Anthony Aguirre, co-founder and executive director of the Future of Life Institute.

Wednesday’s statement was also signed by actor Joseph Gordon-Levitt and several Nobel laureates in physics, including Geoffrey Hinton, an AI researcher widely seen as laying the groundwork for today’s AI tools. Hinton left Google in 2023 to speak more freely about his concerns about AI technology.

States like California have focused on regulations on potential harms of AI tools in daily use such as chatbots or requiring more transparency from companies developing AI.

Advertise with us

In September, California Gov. Gavin Newsom, a Democrat, signed a law that requires AI companies to publish safety reports detailing potential “catastrophic risks” of their technology — such as enabling hackers to break encryption software or terrorists to make new bioweapons.

This month, Newsom signed the nation’s first law regulating how AI chatbots can interact with minors, after three U.S. families alleged in separate lawsuits that an AI chatbot had contributed to a teen’s death by suicide.

In Washington, some Republican politicians continue to fight against AI regulation at both the federal and state level. Proponents of unfettered AI development nearly passed a 10-year moratorium on most state-level AI laws as part of Republicans’ sweeping tax and immigration bill earlier this year.

It ultimately failed in the Senate after opponents of the measure including labor and children’s safety advocates joined state lawmakers and attorneys general from both major parties in criticizing the moratorium. They alleged it was an infringement on states’ rights and a giveaway to Big Tech.

In a bid to resurrect the idea of a federal shield for AI firms, President Donald Trump in July released an “AI Action Plan” that called for blocking certain federal funding from states that pass AI laws deemed “burdensome.” Sen. Ted Cruz, R-Texas, has introduced a bill that would allow AI companies to apply for two-year exemptions from external regulations.

Advertise with us

Are worries about superintelligence also feeding AI hype?

The letter is likely to provoke ongoing debates within the AI research community about the likelihood of superhuman AI, the technical paths to reach it and how dangerous it could be.

“In the past, it’s mostly been the nerds versus the nerds,” said Max Tegmark, president of the Future of Life Institute and a professor at the Massachusetts Institute of Technology. “I feel what we’re really seeing here is how the criticism has gone very mainstream.”

Complicating the broader debates is that the same companies that are striving toward what some call superintelligence and others call artificial general intelligence, or AGI, are also sometimes inflating the capabilities of their products, which can make them more marketable and have contributed to concerns about an AI bubble. OpenAI was recently met with ridicule from mathematicians and AI scientists when its researcher claimed ChatGPT had figured out unsolved math problems — when what it really did was find and summarize what was already online.

“There’s a ton of stuff that’s overhyped and you need to be careful as an investor, but that doesn’t change the fact that — zooming out — AI has gone much faster in the last four years than most people predicted,” Tegmark said.

Advertise with us

Asked if he reached out to Musk again this time, Tegmark said he wrote to the CEOs of all major AI developers in the U.S. but didn’t expect them to sign.

“I really empathize for them, frankly, because they’re so stuck in this race to the bottom that they just feel an irresistible pressure to keep going and not get overtaken by the other guy,” Tegmark said. “I think that’s why it’s so important to stigmatize the race to superintelligence, to the point where the U.S. government just steps in.”

Google, Meta, OpenAI and Musk's xAI didn't immediately respond to requests for comment Wednesday.