What is hardest to appreciate in AI is the improvement curve. “The broader intellectual world seems to wildly overestimate how long it will take AI systems to go from ‘large impact on the world’ to ‘unrecognizably transformed world,’” Paul Christiano, a key member of OpenAI, wrote last year. “This is more likely to be years than decades, and there’s a real chance that it’s months.”

I find myself thinking back to the early days of COVID. There were weeks when it was clear that lockdowns were coming, that the world was tilting into crisis, and yet normalcy reigned, and you sounded like a loon telling your family to stock up on toilet paper. There was the difficulty of living in exponential time, the impossible task of speeding policy and social change to match the rate of viral replication.

I suspect that some of the political and social damage we still carry from the pandemic reflects that impossible acceleration. There is a natural pace to human deliberation. A lot breaks when we are denied the luxury of time.

But that is the kind of moment I believe we are in now. We do not have the luxury of moving this slowly in response, at least not if the technology is going to move this fast.

Advertise with us

Since moving to the Bay Area in 2018, I have tried to spend time regularly with the people working on AI. I don’t know that I can convey just how weird that culture is. I don’t mean that dismissively; I mean it descriptively. It is a community that is living with an altered sense of time and consequence. They are creating a power that they do not understand at a pace they often cannot believe.

In a 2022 survey, AI experts were asked, “What probability do you put on human inability to control future advanced AI systems causing human extinction or similarly permanent and severe disempowerment of the human species?” The median reply was 10%. Many put that probability even higher. Would you work on a technology you thought had a 10% chance of wiping out humanity?

As my colleague Ross Douthat wrote, this is an act of summoning. The coders casting these spells have no idea what will stumble through the portal. What is oddest, in my conversations with them, is that they speak of this freely. These are not naifs who believe their call can be heard only by angels. They believe they might summon demons. They are calling anyway.

I often ask them the same question: If you think calamity is so possible, why do this at all? Different people have different things to say, but after a few pushes, I find they often answer from something that sounds like the AI’s perspective. Many — not all, but enough that I feel comfortable in this characterization — feel that they have a responsibility to usher this new form of intelligence into the world.

Could these systems usher in a new era of scientific progress? In 2021, a system built by DeepMind managed to predict the 3D structure of tens of thousands of proteins, an advance so remarkable that the editors of the journal Science named it their breakthrough of the year.

Advertise with us

Will AI populate our world with nonhuman companions and personalities that become our friends and our enemies and our assistants and our gurus and perhaps even our lovers? “Within two months of downloading Replika, Denise Valenciano, a 30-year-old woman in San Diego, left her boyfriend and is now ‘happily retired from human relationships,’” New York Magazine reports.

Could AI put millions out of work? Automation already has, again and again. Could it help terrorists or antagonistic states develop lethal weapons and crippling cyberattacks? These systems will already offer guidance on building biological weapons if you ask them cleverly enough.

Could it end up controlling critical social processes or public infrastructure in ways we don’t understand and may not like? AI is already being used for predictive policing and judicial sentencing.

But I don’t think these laundry lists of the obvious do much to prepare us. We do not understand these systems, and it’s not clear we even can. “If you were to print out everything the networks do between input and output, it would amount to billions of arithmetic operations,” writes Meghan O’Gieblyn in her brilliant book, “God, Human, Animal, Machine,” “an ‘explanation’ that would be impossible to understand.”

That is perhaps the weirdest thing about what we are building: The “thinking,” for lack of a better word, is utterly inhuman, but we have trained it to present as deeply human. And the more inhuman the systems get — the more billions of connections they draw and layers and parameters and nodes and computing power they acquire — the more human they seem to us.

Advertise with us

The stakes here are material and they are social and they are metaphysical. O’Gieblyn observes that “as AI continues to blow past us in benchmark after benchmark of higher cognition, we quell our anxiety by insisting that what distinguishes true consciousness is emotions, perception, the ability to experience and feel: the qualities, in other words, that we share with animals.”

This is an inversion of centuries of thought, O’Gieblyn notes, in which humanity justified its own dominance by emphasizing our cognitive uniqueness. We may soon find ourselves taking metaphysical shelter in the subjective experience of consciousness: the qualities we share with animals but not, so far, with AI. “If there were gods, they would surely be laughing their heads off at the inconsistency of our logic,” she writes.

If we had eons to adjust, perhaps we could do so cleanly. But we do not. The major tech companies are in a race for AI dominance. The U.S. and China are in a race for AI dominance. Money is gushing toward companies with AI expertise. To suggest we go slower, or even stop entirely, has come to seem childish. If one company slows down, another will speed up. If one country hits pause, the others will push harder. Fatalism becomes the handmaiden of inevitability, and inevitability becomes the justification for acceleration.

Katja Grace, an AI safety researcher, summed up this illogic pithily. Slowing down “would involve coordinating numerous people — we may be arrogant enough to think that we might build a god-machine that can take over the world and remake it as a paradise, but we aren’t delusional.”

One of two things must happen. Humanity needs to accelerate its adaptation to these technologies or a collective, enforceable decision must be made to slow the development of these technologies. Even doing both may not be enough.

Advertise with us

What we cannot do is put these systems out of our mind, mistaking the feeling of normalcy for the fact of it. I recognize that entertaining these possibilities feels a little, yes, weird. It feels that way to me, too. Skepticism is more comfortable. But something Davis writes rings true to me: “In the court of the mind, skepticism makes a great grand vizier, but a lousy lord.”

Ezra Klein is a columnist for The New York Times. His previous article was “Bad news for the economy.” This article originally appeared in The New York Times.