跳到正文
Noam Brown· @polynoamial · X·· 2026-06-12AI 评分63
AI 导读

OpenAI 研究员 Noam Brown 表示,GPT-5.5 在 Agents' Last Exam(ALE)基准中排名第一,且按模型 token、成本或墙钟时间衡量同样表现最佳。ALE 由 @dawnsongtweets 团队创建,是一个滚动基准,包含超过 1500 个专家任务、覆盖 55 个职业,测试 AI 智能体能否执行实际经济价值工作。评估对象包括 GPT-5.5、Fable 5、Composer 2.5 等前沿系统。结果显示:当前智能体能解决部分专业任务,但在需要持续推理和深度专业知识的最难层级,所有被测前沿智能体(包括 Fable 5)成功率为 0%。

正文 · 原文

I'm happy GPT-5.5 tops this eval

I'm even happier it's still doing the best when measured vs tokens, cost, or wall-clock time!

引用Dawn Song@dawnsongtweets
Everyone says the latest AI agents will be "job-ready" soon, especially after the release of Fable 5 this week. But is that really the case? Over the past many months, my group and collaborators have been building Agents' Last Exam (ALE), a benchmark designed to test exactly that claim on real digital labor-market work. My group and collaborators previously have created many of the benchmarks the field runs on, including MMLU, MATH, CyberGym, and ExploitGym. Today, I'm excited to share Agents' Last Exam (ALE): a rolling benchmark that measures whether AI agents can actually perform economically valuable work across a broad range of real-world domains. With ALE, we evaluated Fable 5, GPT-5.5, Composer 2.5, and other frontier agent systems across more than 1,500 expert-sourced tasks spanning 55 occupations. The result is both impressive and sobering. Today's agents can solve a meaningful fraction of professional tasks. But when we look at the hardest tasks, the ones requiring sustained reasoning, deep domain expertise, and reliable execution over long horizons, they are still far from human-level performance. On ALE's hardest tier, every frontier agent we tested, including Fable 5, achieved a 0% success rate. The age of useful agents is here. The age of truly job-ready agents is not. We hope Agents' Last Exam (ALE) will serve as a new guidepost and north star for developing agents capable of reliably performing economically valuable work across a broad range of domains. 🧵
在 X 查看被引用的帖子

来源:Noam Brown · x.com