Pre-training robot vision to learn from very few demonstrations

Learning robot manipulation policies from scarce expert videos is hampered by overreliance on task-irrelevant visual features when adapting frozen pre-trained vision models. GAP introduces a lightweight warm-up stage that pre-trains the spatial pooling adapter on a simulated task with object masks, encouraging it to extract stable geometric keypoints before downstream imitation learning. Evaluated on RoboMimic and ManiSkill under severe data scarcity (15–50 demonstrations), GAP achieves 62% success on Can with 15 demos (+16% over attention-based poolers), 63% on Tool Hang, and 61% on StackCube (+11% over end-to-end fine-tuning). The approach is practical and reusable across tasks without requiring additional labeling.