New Microsoft Paper on GitHub Copilot’s production traces show why coding agents should not be served like chat requests.

In 13.5M GitHub Copilot sessions, 87% of LLM calls came from the agent itself rather than a user.

A user prompt can fan out into an autonomous chain of model calls, tool actions, retries, and growing context, making the turn or session a more useful scheduling unit than an isolated request.

That structure is especially visible in the KV cache.

It shows that KV cache is not really a request-level resource; its value depends on where the agent is in the workflow.

Within a turn, average cache hit rate rises from about 45% on the first LLM call to 92–94% from the third call onward.

At a same-model turn boundary it falls to 55%, while a model switch pushes it down to 8%.

Median KV-cache idle time is 1.2 seconds within a turn versus 172 seconds across turns, while container idle time jumps from 5.8 to 243 seconds.

Using turn- and session-level features, the paper’s lightweight predictor captures 86–90% of total idle time, giving the serving stack a signal for cache offloading or container reclamation.

The implication is straightforward: coding-agent infrastructure should schedule workflow state across turns, because request-level policies discard some of the strongest signals in the workload.

– arxiv. org/abs/2608.00101

Title: "Agentic Coding in the Wild: Characterizing GitHub Copilot Traces at Production Scale"