Robots have been good at seeing for a while, but this work and the super-massive 38.6TB dataset from TARS and other top Institutions, is about the part that Robots still routinely miss:

What contact actually feels like.

If you want to know why robots still find ordinary physical tasks hard, why providing additional tactile perception to robots actually make them perform worse? This is a good place to look: robotic manipulation has long lacked modeling of contact dynamics and effective utilization of tactile information.

OmniVTA is trying to solve a super important robotics problem: Understanding Contact.

They introduced OmniVTA and OmniViTac, a paired model-and-dataset effort aimed at one of robotics’ hardest problems: getting robots to handle contact-rich manipulation by not only seeing the world, but also predicting contact trends in advance (e.g., upcoming contact, changes in contact intensity, or slip risk) before the next mistake happens.

Robots do not struggle most when they are moving through empty space, they struggle when the world pushes back, and OmniVTA is designed for that moment.

The hard part in tasks like wiping, peeling, cutting, assembly, grasping, and in-hand adjustment is that vision misses the exact signals that decide success, because slip, friction changes, force buildup.

And tiny contact transitions often happen at the fingertip and unfold too fast or too subtly for cameras alone.

That is why this work starts with OmniViTac, a large synchronized visuo-tactile-action dataset with 21,879 trajectories across 86 tasks and 100+ objects, collected with multiple tactile sensors and two embodiments, a robot arm and a handheld system, so the data covers both robot-like motion and scalable human-centric data collection instead of staying trapped in one narrow setup.

What makes the dataset interesting is that it is organized around 6 physics-grounded interaction patterns rather than a random bag of demos, which means the data is grouped by how contact actually behaves.

And the paper shows this clearly through two recurring properties of touch in manipulation: spatial locality, where useful signals often live in small fingertip regions, and contact-driven dynamics, where the meaning of touch changes sharply once real physical interaction begins.