Neural network weights as a generative resource to build new models

This position paper proposes treating trained neural network checkpoints as a generative data modality. The core insight is that high-performing models cluster in structured regions of weight space shaped by symmetry, modularity, and shared subspaces, making it possible to synthesize weights directly rather than optimize them. The authors organize existing methods into a five-stage pipeline and survey practical applications ranging from adapter-scale generation to conditional checkpoint synthesis. Current limits remain at frontier-scale unrestricted synthesis. The proposal shifts the paradigm from per-task optimization to sampling models from learned weight distributions, enabling AI systems to generate or improve other AI systems directly.