Brilliant open-source robotics foundation-model release from @perceptroninc .

Isaac 0.5 cuts teleoperation needs by 210X by scaling action-free video.

The most expensive thing in robotics is still collecting robot data, and this model attacks that problem.

Isaac 0.5 is just a 36B param sparse model that can take in camera images/video, a human instruction, the robot's current state, and then produce things such as object locations, task progress, visual reasoning, or the robot's next actions.

So if you have a robot arm, you can take Isaac 0.5, fine-tune it on demonstrations from that arm, and use the resulting policy to control the robot. The same underlying model can also be used only for perception, feeding its visual understanding into another planner or controller.

Perceptron reports faster one-demonstration adaptation: one chess episode cut held-out action loss 7.0× to 10.5×, versus 2.3× to 3.1× for π0.5.