← Back to Computer Vision
cs.CV

Can AI learn to rate surgery skills by watching hand movements?

Roi Papo, Idan Smoller, Shlomi Laufer

May 22, 2026

Surgical training depends on expert feedback, which doesn't scale. ExpOS watches hand-tool dynamics from video to predict skill level and explain *why*—identifying which movements matter most. Trained on 221 student surgery videos, it combines hand pose tracking with temporal attention networks to pinpoint informative moments and behaviors. The strongest results came on fascial closure (r=0.778), suggesting the approach could enable autonomous practice and scalable feedback without constant instructor oversight.
Published as ExpOS: Explainable Open-Surgery Skills Assessment Using 3D Hand Reconstruction arXiv:2605.23653
Read the original paper →