Can swapping English words for translations speed up multilingual learning?

Building effective language models for low-resource languages usually demands expensive parallel data or translation systems. LINK replaces random English words with their translations during pretraining—nothing fancy, just bilingual dictionary lookups. Tested across eight languages and five model sizes, the method cuts training time in half to reach the same downstream task performance, with gains in reasoning and world knowledge tasks.