Why do language models favor Latin script internally?

Language models handle multiple writing systems (Arabic, Cyrillic, Latin) for the same language, but how? This work reveals they maintain a shared internal representation and systematically favor Latin script: a small set of attention heads causally control which script the model outputs, these heads transfer across unrelated languages, and linear steering can flip scripts while preserving meaning. Crucially, the model uses compact mechanisms to generate non-Latin text but relies on diffuse network-wide contributions for Latin—suggesting an architectural bias baked into modern LLMs.