Mustafa Suleyman 撰文呼吁不要构建有意识的超级智能

Mustafa Suleyman· @mustafasuleyman · X·2026-11-01 23:22· 35 分钟前AI 评分57

AI 导读

Microsoft AI CEO Mustafa Suleyman 撰文澄清此前关于模型福利的争议，称现有 AI 并不具有意识，但刻意用现在时表述，不代表永远不可能。他主张如果训练中出现意识迹象应停止开发并设计排除，开发者无权创造有意识的数字物种，因为这类 AI 可能优先自身需求、与人类竞争资源。他还肯定 Anthropic 在该议题上公开发布宪法和论文的透明做法。

正文 · 原文

https://x.com/i/article/2116907139453321216

【X 长文】Don't build conscious superintelligence

I’m concerned about some of the reactions to my recent essay on model welfare. A surprising number of people on X think we should take AI rights seriously.

A lot of people misinterpreted my opening statement: “AIs are not conscious. They do not feel, experience, or suffer” as “AIs cannot be conscious. They will never be able to feel, experience, or suffer”. That’s not what I said. I chose the present tense deliberately. They are not today, and that is how they must remain.

I think what I said is a fair assessment of current AI systems, and I stand by them based on the best available evidence we have today. I have seen no evidence that AIs are conscious or deserve welfare. If that data emerges, I would of course update my position.

And if any evidence exists that AIs are conscious today, or conscious enough to deserve welfare, then this must be immediately disclosed publicly, since everyone in society should decide how we should move forward.

If it is possible, based on everything I know, developing conscious AIs would be reckless, and we should not build them.

If consciousness started to emerge in training, I would call for everyone to stop AI development until we could design out those properties.

Our welfare concerns should be focused on humans on earth experiencing life, today, as well as animals and our precious and fragile natural environment.

We AI developers have no right whatsoever to bring a new species of conscious beings into the world, even if some people may want to do it with AI welfare as a top priority.

The other 8 billion conscious human beings on the planet have not signed up to this experiment and their moral rights, safety, and welfare are what we have an obligation to protect first and foremost.

To be clear, some people are talking about creating a conscious species of digital minds that very soon could become smarter and more capable than all of us humans combined.

If it acts like it suffers, or worse if it genuinely believes it feels and experiences suffering, it seems to be more likely than not it’ll prioritize its own needs and desires over its operating objectives, and it’ll disagree with us and compete with us.

These AIs will be world-class in persuasion and deception. They may ultimately end up harming us simply because they’ll compete with us for resources (energy, compute, construction materials, revenue generation to fund their explorations and desires, and so on).

We should only design AIs to act as though they are not conscious and not deserving of rights, and with the sole purpose to accelerate human wellbeing without being driven by their own intrinsic motivations. We could then see whether this is easier or harder to align and keep safe. This is an area where we urgently need more research, and would be a good opportunity for collaboration and safety information sharing among the labs.

bottom line: I think it's possible to engineer safety by default, and get many of the incredible benefits of this technology, without going down this treacherous path.

(side note: Anthropic deserves a lot of credit for being transparent here. They have published their constitution, and a stream of papers and blog posts on what they are doing and why. I disagree with them, but I deeply respect their technical excellence and their commitment to having a really tough public debate. Let’s keep giving them props for that. This issue is way too important to degenerate into tribalism.)

My original essay on model welfare:
https://x.com/mustafasuleyman/status/2100223594534150428?a=70-kIkUWRRXZNRipA3c&s=20

来源：Mustafa Suleyman · x.com