Noam Brown· @polynoamial · X·· 2026-04-09
AI 导读
作者吐槽业界仍习惯用单一数字评估推理模型，引用观点指出 MMLU/GSM8K 等基准早已过时却仍在被报告，认为 Intelligence/$（智能性价比）才是更优指标，并以 o1-mini 发布时的多维对比图表为例说明。
正文 · 原文
I'm surprised that, more than a year later, it's still the norm to compare reasoning models on evals by a single number.
LLM evals are slow to adapt. MMLU/GSM8K continued to be reported long after they were obsolete. I think the next thing to go away will be comparing models on evals by a single number. Intelligence/$ is a much better metric. I loved this plot from o1-mini's launch for example: https://t.co/U4ry4S3chR在 X 查看被引用的帖子
来源：Noam Brown · x.com