Not Diamond 发布模型路由评估方法论，称在多个主流基准上实现 Pareto 占优，以低 20–80% 的成本超过 Opus xhigh 质量。
Not Diamond just released the methodology behind their model routing which gets Opus xhigh quality while cutting agent costs 20–80%.
Model routing becomes much harder decision for long-running coding agents. So Not Diamond is benchmarking the router across simulated user turns, changing task complexity, and response delays that can expire the KV cache.
And harder than it seems, switch models too aggressively and you can throw away the KV cache. Pick one model from the opening prompt and you are assuming task complexity stays fixed for the rest of the session.
So Not Diamond is treating routing as a sequential decision problem instead.
At each step, its router predicts future reward and cost for a particular model and reasoning effort, using current and previous session state, message and token counts, task complexity, KV-cache state, and intermediate reward signals.
Read more detail on their technical report.
Today we’re releasing our methodology for evaluating model routing with interactive benchmarks, which represent agent cost accumulation better than static benchmarks do. Across leading benchmarks, we achieve Pareto-dominance, exceeding Opus xhigh quality at 20–80% lower cost.在 X 查看被引用的帖子
来源：Rohan Paul · x.com