Teaching language models which tokens matter for better reasoning

Current reinforcement learning methods for language models treat all tokens equally when learning from right and wrong answers—so formatting tokens dominate, drowning out the reasoning-critical updates. DelTA identifies which tokens genuinely distinguish high-reward from low-reward responses, then amplifies their gradient signal while suppressing noise. On math benchmarks, this yields 3.26-point gains over comparable models, with consistent improvements on code and out-of-distribution tasks.