Removing backgrounds from video without reshooting

Virtual production uses LED walls to render backgrounds in real time, but changing them later requires expensive reshoots. CineMatte solves this by accurately extracting foreground elements from video so backgrounds can be swapped afterward. The method uses two parallel vision transformers to separately analyze the frame and captured background, then compares them with cross-attention to predict the foreground—avoiding the boundary artifacts that plague earlier approaches. The authors also release CineMatte-4K, the first professional 4K HDR dataset for this task, shot on an actual LED stage with tracked camera motion for correct parallax.