Generalist AI 发布 GEN-1.5：一个从 3–12 秒演示中学会新任务的机器人基础模型

Generalist AI has released GEN-1.5, a robot foundation model that learns a new physical task from a single demonstration. Drop 3–12 seconds of sensorimotor data into its 30-second context window, and the robot performs the task. No gradient updates, no fine-tuning, no task-specific programming. Across 10 diverse manipulation tasks, this one-shot in-context prompting averaged 59% success (±10% std. dev.) straight from the pretrained model. Ten gradient steps on five minutes of data per task raised that to 83% (±9%). Generalist calls the mechanism physical prompting, and says it was never trained for: no architectural changes, no meta-learning loop, no auxiliary objectives. It emerged from over eight months of continuous pretraining on physical interaction data. The tasks are simple and short-horizon, and the company says so plainly. But this is the first model its team knows of where one-shot learning of physical skills has emerged at scale.

Is it deployable?

Not yet — this is a research release. There are no public weights, no API, no pricing page and no self-serve product. Generalist AI runs GEN-1.5 on its own fleet and data engine. Anyone who wants it today goes through a direct partnership.

What is GEN-1.5?

GEN-1.5 is a large multimodal model that takes video, sensor, language and proprioceptive inputs, holds 30 seconds of memory, and emits 100 Hz action trajectories. It has been pretraining continuously for over eight months on physical interaction data captured in homes, warehouses and factories.

The main mechanism is physical prompting. A sensorimotor example — sensor streams plus the action trajectory — is inserted into the 30-second context window through a drag-and-drop interface. The remainder of the window holds rolling observations. The model then performs the task immediately, with zero gradient steps and no fine-tuning.

Crucially, none of this was designed in. Generalist states there were no architectural changes to promote in-context learning, no meta-learning loop, and no auxiliary objectives encouraging improvisation. The capability emerged from pretraining scale, the same way one-shot prompting emerged in GPT-3.

The numbers

Across 10 diverse tasks, one-shot in-context prompting averaged 59% success (±10% std. dev.) from the pretrained model, with no training at all. Ten gradient steps on five minutes of data per task — roughly 50 demonstrations — raised that to 83% (±9%). In the extreme case, one gradient step on one minute of data reached 66.5% on a held-out task, with no adaptation-specific hyperparameter sweep.

The compute story is the interesting part. Adapting robot policies has typically taken tens of thousands of gradient steps. Ten steps here move the model weights on held-out tasks by less than 0.15%, which suggests fine-tuning is reconfiguring knowledge the model already has rather than building new representations. Generalist frames it as test-time training in an extremely low-data regime.

Three transfer results worth knowing

• Compositional generalization: Two independently recorded prompts placed in context get chained into one continuous behaviour. The model produces the bridging motions — repositioning, regrasping, error recovery — that appear in neither demonstration.
• Zero-shot sim-to-real: A demonstration recorded entirely in simulation works as a prompt for the real robot, despite pretraining containing no simulation data — neither rendered video nor simulated dynamics. For some tasks, demonstrations no longer need to be collected physically.
• Human-to-robot imitation: In some cases a person demonstrates with their own hands, in view of the robot’s cameras, and the model reproduces it with the robot’s hands.

Generalization also shows up after light fine-tuning. Trained on five minutes of brushing a block into a bowl, the model used a banana as a makeshift brush, and used a dustpan to lift and dump the block instead — a different contact sequence entirely. It also removed a sheet of paper covering the bowl, and worked ambidextrously when demonstrations used one hand.

Interactive explainer

Key Takeaways

• GEN-1.5 learns new manipulation tasks from a single 3–12 second demonstration dropped into its 30-second context window.
• One-shot in-context prompting hit 59% across 10 tasks; 10 gradient steps on 5 minutes of data hit 83%.
• One-shot, sim-to-real and human-to-robot transfer emerged from pretraining — none of it was explicitly trained for.
• Ten gradient steps change weights by under 0.15%, collapsing per-task adaptation compute by orders of magnitude.
• No weights, no API, no product: treat this as a research signal about scaling, not a deployable system.