How robots learn from mistakes to avoid them next time

Robots operating in unpredictable real-world environments often encounter hazards specific to their design that can't be anticipated in advance. This work proposes a continual learning system where robots observe disturbances (collisions, slips, etc.), describe what happened using cameras, query a vision-language model to identify likely causes, and build spatial models of dangerous interactions. The approach combines kernel regression for quick anomaly characterization with semantic uncertainty estimation to enable better recovery. Tested in simulation and on hardware across different robot types and failure modes.