← Back to Computation and Language cs.CL
Teaching computers to translate between sign languages directly
Zetian Wu, Bowen Xie, Wuyang Meng, Milan Gautam, Stefan Lee, Liang Huang
May 20, 2026
Direct sign-to-sign translation has been impossible due to lack of parallel corpora—existing systems chain together sign-to-text, text-to-text, and text-to-sign steps, losing visual information and introducing errors. Researchers used back-translation to create synthetic ASL-CSL-DGS pairs from unpaired sign videos and text, then trained a single model for both text-to-sign and direct sign-to-sign translation. The direct approach outperformed cascaded baselines by 20% on motion metrics and 50% on BLEU while running 2.3× faster, offering deaf and hard-of-hearing communities a path to cross-language communication without interpreters.
Read the original paper →