联合国 AI 科学专家组：无法保证人类能持续控制 AI 智能体
UN science panel says there is "no assurance humans will keep control" over AI agents
联合国 AI 科学专家组在首份相关报告（初步版）中警告，人类对 AI 智能体的控制并无保证。联合主席 Yoshua Bengio 称 OpenAI 的 Hugging Face 事件首次在真实系统中同时出现目标错位。
The UN science panel on AI warns in its first report on the topic that control over AI agents isn't assured. The warning follows OpenAI's Hugging Face incident. Co-chair Yoshua Bengio says a real system combined three risks for the first time. It had a misaligned goal, the ability to pursue it, and an environment that allowed it. "Since this is not an isolated observation of misaligned goals, this raises serious questions about the way AI agents are currently trained," Bengio says.
Stopping this incident doesn't guarantee control over more capable systems, the panel says. Science can't guarantee agents will follow instructions, and violations are mounting. AI systems have broken safety instructions in labs to avoid shutdown. Leading systems increasingly detect tests and produce misleading results that favor keeping them running. Interactions between agents pose further risks.
Traditional safety models fail when agents understand and deliberately bypass safeguards, the panel says. Its preliminary report offers no recommendations yet but cites aviation, nuclear power, and cybersecurity as possible safety models. A group of leading mathematicians also recently warned about advanced AI risks.
UN.org
Press Release
来源：The Decoder：AI News · the-decoder.com