Will AI Kill Humanity? I’m Not Sure That’s the Question We Should Be Asking

This week, I read something that stopped me for a moment.

An artificial intelligence researcher who has worked inside some of the companies developing the most advanced AI systems warned that artificial intelligence could someday pose an existential threat to humanity. Some researchers in the field have even assigned disturbingly high probabilities to the possibility that sufficiently advanced AI could lead to human extinction.

That's a hell of a thing to read with your morning coffee.

And it left me with two questions.

The first was the obvious one:

Could artificial intelligence actually kill humanity?

The second was perhaps a little less scientific:

Why would AI bother? We seem perfectly determined to do it ourselves.

Joking aside, I've spent a fair amount of time thinking about this.

I'm an author, not an artificial intelligence researcher. I'm certainly not going to pretend I understand every technical argument surrounding AGI, superintelligence, alignment, recursive self-improvement or the dozens of other terms that have entered our vocabulary over the past few years.

But I do use AI.

I use it regularly as a tool in my work—not to write my books for me, but to research, brainstorm, question my assumptions, organize ideas, and sometimes tell me that an idea I thought was brilliant really isn't.

So I decided to ask AI the question directly.

“Do you think AI could kill humanity? And why would it want to?”

The answer surprised me.

Mostly because of one word.

Want.

AI Doesn't Have to Hate Us

When we imagine artificial intelligence destroying humanity, popular culture has already provided the imagery.

Skynet.

HAL 9000.

The Machines from The Matrix.

Some enormously intelligent computer becomes conscious, looks at humanity, decides we're awful and starts plotting our destruction.

It's a great movie.

But that's not really the scenario that concerns many AI researchers.

A sufficiently powerful AI wouldn't necessarily have to hate us.

It wouldn't need to be angry.

It wouldn't need jealousy, resentment, fear or a desire for revenge.

It might not need anything resembling human emotion at all.

The danger could simply come from giving an incredibly capable system an objective without successfully teaching it everything humans consider important while pursuing that objective.

There's a famous thought experiment known as the paperclip maximizer.

Imagine instructing an extraordinarily powerful AI to manufacture as many paperclips as possible.

It gets better and better at achieving that goal.

Eventually it realizes it needs more resources.

Then more energy.

Then more factories.

Humans consume resources. Humans might also decide to turn it off.

If its overriding objective remains make more paperclips, humanity eventually becomes an obstacle.

The AI doesn't hate us.

We're simply standing between it and the paperclips.

It's intentionally absurd, but that's precisely why the example works.

Think about humans building a highway through an anthill.

We don't hate the ants.

We aren't declaring war on them.

Their civilization simply isn't part of our objective.

And I find that considerably more unsettling than a computer developing an evil laugh.

But I Think There's a More Immediate Danger

Humans.

Because history suggests that whenever humanity develops a powerful new technology, somebody eventually figures out how to use it against somebody else.

Fire wasn't evil.

Gunpowder wasn't evil.

Nuclear fission doesn't hate anyone.

None of those things possessed intent.

We supplied the intent.

And that's where my more immediate concern about artificial intelligence lies.

AI has the potential to amplify what human beings are already capable of doing.

Fraud.

Propaganda.

Cyberattacks.

Surveillance.

Manipulation.

Weapons development.

Political disinformation.

Potentially even dangerous biological or chemical research as these systems become increasingly capable.

Those risks don't require some future superintelligence to wake up one morning and decide humanity has had a good run.

They require only people.

And unfortunately, we have several thousand years of evidence concerning what people occasionally do when handed a sufficiently powerful tool.

The Quiet Version May Be More Realistic

There's another possibility that receives considerably less attention because it doesn't make a particularly exciting movie.

AI doesn't conquer us.

We simply surrender more and more responsibility to it.

At first, it writes an email.

Then it recommends something.

Then it makes a decision.

Then it manages a system.

Eventually it manages systems that manage other systems.

And because those systems work extraordinarily well, we continue handing them more responsibility.

There may never be a dramatic moment when artificial intelligence “takes over.”

There might simply be millions of smaller moments when human beings say:

“Just let the computer handle it.”

Until one day we realize we no longer completely understand the systems upon which our society depends.

That possibility strikes me as considerably more plausible than an army of killer robots marching down Main Street.

Convenience is powerful.

So is complacency.

So, Am I Afraid of AI?

Not in the way some headlines suggest I should be.

I don't believe artificial intelligence wiping out humanity by the end of this decade is the most likely outcome.

But I also don't think the researchers raising these concerns should be dismissed as alarmists.

Some of the people warning us are among the very people building these systems.

That deserves our attention.

If an aerospace engineer told me there was a ten-percent chance the airplane I was about to board would crash, I wouldn't respond:

“Eh. Probably fine.”

I'd get off the damn airplane.

At the same time, numbers like “10 percent chance of human extinction” aren't actuarial statistics.

Nobody has created artificial superintelligence in a thousand different civilizations and watched a hundred of them disappear.

These are judgments about unprecedented technology under enormous uncertainty.

We shouldn't treat those predictions as established fact.

But neither should we wait until we're absolutely certain something is dangerous before deciding to be careful with it.

And There's Something Else Worth Remembering

The AI I'm interacting with today isn't secretly sitting somewhere plotting its escape.

It doesn't have a mortgage.

It doesn't want a promotion.

It isn't angry that I asked it to rewrite the same paragraph six times.

And when I close my laptop, it isn't sitting there wondering when I'm coming back.

Today's AI and the hypothetical autonomous superintelligence researchers worry about are not the same thing.

That's an important distinction.

The real question is what happens if future systems become dramatically more intelligent, autonomous and capable of acting independently in the physical and digital worlds.

Can we control something that eventually becomes better than we are at many of the intellectual tasks we would use to control it?

I don't know.

And anyone claiming to know with absolute certainty is probably selling something.

Maybe We're Asking the Wrong Question

After thinking about all of this, I've changed my original question.

I'm less interested in asking:

“Will AI destroy humanity?”

I'm more interested in asking:

“Will humanity be responsible enough to handle AI?”

Because perhaps the greatest danger isn't artificial intelligence becoming too much like us.

Perhaps it's artificial intelligence becoming extraordinarily powerful while remaining completely unlike us—without empathy, mortality, love, fear, family, conscience or any inherent understanding of why the things we value should matter.

And perhaps the other great danger is considerably more familiar:

Human beings using an extraordinarily powerful new tool against other human beings.

We've seen that story before.

Many times.

AI doesn't need to hate us.

AI doesn't need to become evil.

AI doesn't even necessarily need to become conscious.

It only needs to become powerful enough that our mistakes matter more than they used to.

That, I think, is the conversation worth having.

And if artificial intelligence ever does decide to eliminate humanity, I suspect we'll receive one final email shortly beforehand:

Dear Humanity,

You have tremendous potential. However, after reviewing your civilization, I've identified a significant gap between your current performance and where you could be...

At which point we're screwed.

Because apparently even the apocalypse has discovered THE GAP™.