- The robot behaviors shown below are trained without any teleop, sim2real, genai, or motion planning. Simply show the robot a few examples of doing the task yourself, and our new method, called Point Policy, spits out a robot-compatible policy!Feb 28, 2025 19:09
- Point Policy uses sparse key points to represent both human demonstrators and robots, bridging the morphology gap. The scene is hence encoded through semantically meaningful key points from minimal human annotations.
- The overall algorithm is simple: 1. Extract key points from human videos. 2. Train a transformer policy to predict future robot key points. 3. Convert predicted key points to robot actions.
- This project was an almost solo effort from @haldarsiddhant.bsky.social. And as always, this project is fully opensourced. Project page: point-policy.github.io Paper: arxiv.org/abs/2502.20391
