How to Build a Diffusion Language Model

The Kuleshov Group at Cornell published a step-by-step guide to building diffusion language models from scratch. The post covers the masked diffusion process for discrete tokens, the forward corruption schedule, and practical training details. It accompanies their recent paper on improved masked diffusion models and includes working code.