But that isn't the most likely scenario. 但这不是最可能出现的事情。

It's not that our machines will become spontaneously malevolent. 我们的机器不会自动变得邪恶。

The concern is really that we will build machines that are so much more competent 所以，我们唯一的顾虑就是我们将会打造比我们人类更有竞争力的机器。

than we are that the slightest divergence between their goals and our own could destroy us. 而一旦我们和它们的目标不一致，我们将会被摧毁。

Just think about how we relate to ants. 想想我们与蚂蚁的关系吧。

We don't hate them. We don't go out of our way to harm them. 我们不讨厌它们，我们不会去主动去伤害它们。

In fact, sometimes we take pains not to harm them. We step over them on the sidewalk. 实际上，我们经常会尽量避免伤害蚂蚁。我们会选择从它们身边走过。

But whenever their presence seriously conflicts with one of our goals, 但只要它们的存在妨碍到了我们达成目标，

let's say when constructing a building like this one, we annihilate them without a qualm. 比如说当我们在建造这样一个建筑，我们会毫不手软地杀掉它们。

The concern is that we will one day build machines that, whether they're conscious or not, could treat us with similar disregard. 所以我们的顾虑是，终将有一天我们打造的机器，不管它们是否有意识，它们终将会以我们对待蚂蚁的方式来对待我们。

Now, I suspect this seems far-fetched to many of you. 我想很多人会说这很遥远。

I bet there are those of you who doubt that superintelligent AI is possible, much less inevitable. 我打赌你们中有些人还会怀疑超级人工智能是否可能实现，认为我是在小题大做。

But then you must find something wrong with one of the following assumptions. And there are only three of them. 但是你很快会发现以下这些假设中的某一个是有问题的。下面是仅有的三种假设：

Intelligence is a matter of information processing in physical systems. 第一，智慧可以被看做物理系统中的信息处理过程。

Actually, this is a little bit more than an assumption. 事实上，这不仅仅是一个假设。

We have already built narrow intelligence into our machines, and many of these machines perform at a level of superhuman intelligence already. 我们已经在有些机器中嵌入了智能系统，这些机器中很多已经有着超越普通人的智慧了。

And we know that mere matter can give rise to what is called "general intelligence," 而且，我们也知道任何一点小事都可以引发所谓的“普遍智慧”，

an ability to think flexibly across multiple domains, because our brains have managed it. Right? 这是一种可以在不同领域间灵活思考的能力，因为我们的大脑已经成功做到了这些。对吧？

I mean, there's just atoms in here, and as long as we continue to build systems of atoms that display more and more intelligent behavior, 我的意思是，大脑里其实都是原子，只要我们继续建造这些原子体系，我们就能实现越来越多的智慧行为，

we will eventually, unless we are interrupted, we will eventually build general intelligence into our machines. 我们最终将会，当然除非我们被干扰，我们最终将会给我们的机器赋予广泛意义上的智能。