Fine-tuning language models together while detecting bad actors

Federated learning lets multiple computers train models together while keeping data local, but it breaks when some participants are malicious or have corrupted data. CLAIR detects these bad actors while recovering a shared low-rank subspace for LLM fine-tuning, even when clients use different data and model variants. On text tasks, it identifies contaminated clients accurately and improves performance over isolated local training.