Why GUI agents fail at dragging and how to fix it

GUI agents can click but struggle with drag operations (highlighting, resizing, slider manipulation)—a gap in training data of an order of magnitude. DragOn provides 3.5M drag tasks across four interaction types to address this. Fine-tuning Qwen on the dataset improves performance on complex drag interactions, suggesting the bottleneck was simply missing examples rather than fundamental model limits.