← Back to Computation and Language
cs.CL

Do LLM agent skills actually work? Testing 30 open-source tools

Jiahao Ying, Boxian Ai, Wei Tang, Siyuan Liu, Yixin Cao

May 22, 2026

LLM agents are increasingly augmented with "skills"—structured workflows for tasks like web design and report generation. OpenSkillEval tests whether these 30 community skills actually help by generating realistic tasks across five application domains. Finding: skill availability doesn't guarantee effective usage; benefits depend heavily on the underlying model and agent framework, and many popular skills underperform agents without any skills at all.
Published as OpenSkillEval: Automatically Auditing the Open Skill Ecosystem for LLM Agents arXiv:2605.23657
Read the original paper →