Training neural networks with thousands of hard constraints

Training neural networks with constraints—for fairness, physics laws, or business rules—lacks a general solution for the messy non-convex setting that real deep learning creates. This work introduces SPBM, which combines penalty and barrier methods with exponential dual averaging and the Moreau envelope to handle constraints and non-smooth objectives simultaneously. Experiments show it matches or beats existing constrained optimization methods while adding just linear overhead, scaling to 10,000+ constraints.