So much brilliant work is coming on the routing layer of inference.

Not Diamond just announced Not Diamond Code, a routing layer that lets a coding agent switch between models at every step.

claims this cuts costs by 20% or more without reducing overall task quality.

The point is long-horizon coding makes model routing much harder than picking the cheapest model for each prompt.

A cheap model may handle the current turn, but switching to it can discard a warm KV cache or leave later steps with weaker context. Not Diamond Code instead predicts the future cost and reward of each model and reasoning-effort choice across the remaining agent trajectory.

e.g. the below graph is taken from their official blog, show that how different models are better at different parts of a coding task, and a router that can choose among them can outperform an Anthropic-only strategy while spending less.
In this benchmark, the mixed router improved performance by 3.6% and increased savings from 39% to 66%.