Prime Intellect 发布 Verifiers v1：用于智能体强化学习训练与评估的可组合任务集、Harness 与运行时

Prime Intellect launched verifiers 0.2.0. It previews a rewritten core, shipped under the new verifiers.v1 namespace. Modern evaluations now run coding agents with tools, compaction, and subagents. Accordingly, v1 rebuilds environments to run these agentic workloads at scale.

What is verifiers v1?

First, consider what verifiers is: Prime Intellect’s environment stack for agentic reinforcement learning and evaluations. Previously, an environment bundled its data, agent logic, and infrastructure together. In contrast, v1 breaks that bundle into three composable pieces.

A taskset defines the work: the data, tools, and scoring. A harness solves the task and produces a rollout. That harness can be a ReAct loop, a CLI agent, or your own. The rollout then runs inside a runtime, either local or in a sandbox. Because the pieces decouple, any taskset runs under any compatible harness.

How the Architecture Works?

With those pieces defined, the next question is how they communicate. The central piece is the verifiers-managed interception server. It sits between the agent’s runtime and the inference server. Specifically, it proxies requests to, and responses from, inference. Meanwhile, it records the trace, sets sampling parameters, and can rewrite tool responses. That rewriting helps mitigate reward hacks during training.

For scale, each server multiplexes a constant number of rollouts, defaulting to 32. A pool then scales elastically with observed concurrency. The server also owns a client that relays those requests. During evaluation, an EvalClient acts as a blind HTTP proxy. During training, a TrainClient wraps renderers for faithful token-in RL training.

Because harnesses speak different dialects, verifiers supports three as of now. These are OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages. A dialect adapter normalizes each wire format into canonical vf.types. Consequently, your scoring logic stays independent of the agent tested.

v0 vs v1: A Quick Comparison

These changes separate v1 from v0.

Use Cases with Examples

With the architecture clear, consider how teams use it. For example, you can run Nemotron 3 Ultra on Terminal-Bench 2 under Codex.

Similarly, teams can reuse Harbor datasets without rewriting reward logic. Prime Intellect ported Terminal Bench 2 into v1 with only a small class. In its internal testing, verifiers matched Harbor’s performance on the same tasks. Harbor is the first fully-supported third-party format; NeMo Gym and OpenEnv have alpha support.

On the training side, the same environments plug into prime-rl directly. In a length-penalty ablation, GLM-4.5-Air trained on ScaleSWE across six H200 nodes. That run took two days and evaluated on SWE-Bench-Verified, showing stable agentic training.

A Minimal Taskset and Launch

Each run starts from a taskset that defines data and scoring, independent of any harness:

import verifiers.v1 as vf

class AdditionData(vf.TaskData):
    answer: int

class AdditionTask(vf.Task[AdditionData]):
    @vf.reward
    async def exact_match(self, trace: vf.Trace) -> float:
        return float(trace.last_reply == str(self.data.answer))

class AdditionTaskset(vf.Taskset[AdditionTask, vf.TasksetConfig]):
    def load(self) -> list[AdditionTask]:
        return [
            AdditionTask(
                AdditionData(idx=i, prompt=f"What is {i} + {i}?", answer=2 * i),
                self.config.task,
            )
            for i in range(100)
        ]

__all__ = ["AdditionTaskset"]

Any taskset then runs under a chosen harness via TOML and the CLI:

model = "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B"

[taskset]
id = "primeintellect/terminal-bench-2"

[harness]
id = "codex"
version = "0.116.0"

uv run eval @ path/to/config.toml

Key Takeaways

• verifiers v1 splits an environment into a taskset (what), a harness (how), and a runtime (where).
• A verifiers-managed interception server proxies harness–inference requests and records traces on the fly.
• A linear message-graph trace replaces v0’s quadratic prompt-completion pairs, enabling long-horizon training.
• It ships with full prime-rl training support; the legacy code path is now frozen.
• Harbor datasets and harnesses like Codex and Terminus 2 work out of the box.