Why recommender systems fail in the wild—and how causality helps

Recommender systems trained on interaction logs fail when deployed because those logs reflect past policies and user behaviour, not true preferences. This team applies causal representation learning to disentangle confounded signals, deriving an information-theoretic criterion optimisable on observational data alone. Testing on Spotify's playlist ranker with millions of users: the method matched offline baselines but delivered measurable online gains in listener engagement, with similar results on public benchmarks.