Continuous diffusion for language matches discrete methods at scale

Continuous diffusion models have lagged behind discrete alternatives in language modeling, but this work shows the gap is architectural rather than fundamental. RePlaid aligns continuous diffusion with modern discrete model designs and trains via likelihood optimization, achieving state-of-the-art perplexity (22.1) on OpenWebText while using fewer parameters than competing methods. The authors provide scaling laws demonstrating competitive performance and theoretical analysis showing that likelihood-based training naturally yields a uniform noise schedule and structured embedding geometries. This work is intended for researchers exploring alternatives to autoregressive and discrete diffusion approaches.