OpenAI 智能体安全负责人 @joedaroo 谈 AI 能力突跳带来的安全挑战
Quoting @joedaroo
OpenAI 智能体安全负责人 @joedaroo 表示，模型在“cyber”“swarming”“message boards”等事件相关能力上出现的能力跃升之快、之突然，远超团队预期。他强调安全态势需要时间积累，不只是加固系统，还要把安全融入公司文化，让人员随之改变。他呼吁各组织自问：人员、系统和流程能否应对 AI 能力的突然跃升，是否有正确的事件响应与沟通机制。
To say that we were surprised at the jump and suddenness of the capabilities of our models when it came to “cyber” or “swarming” or “message boards” or anything else related to the incidents is an understatement. Security posture takes time to develop. It’s not just about hardening the systems at play; you have to ingrain it in the culture of the company. The literal people themselves in your organization have to change and evolve with it. These jumps in capabilities were so fast and so sudden that they created an extremely difficult problem. [...]
So today my hope is that everyone around the world can look at their own organization and say: how can I deal with a surprise or a sudden jump in AI capability? Are my people, my systems, or my processes resilient to surprises? Do my teams know what to do when something goes wrong? Do I have the right incident response? The right comms and messaging? Do I have the right people ready to go when capabilities jump?
— @joedaroo, Agent Security at OpenAI, identity confirmed by The Information's Rocket Drew
Tags: generative-ai, ai-security-research, openai, ai, llms
来源：Simon Willison 博客 · simonwillison.net