Diffusion Language Models

Diffusion language models (DLMs) have recently emerged as a non-autoregressive approach to language generation, enabling parallel, iterative, and refinement-based decoding beyond the conventional left-to-right paradigm.

This tutorial offers the NLP community a technical map of this emerging field: starting from diffusion formulations for text, we clarify their connections to autoregressive language modeling, masked language modeling, and earlier iterative generation methods. We then survey recent progress in large-scale DLMs, efficient inference, training objectives, and post-training, including reinforcement learning for reasoning and alignment.

Throughout the tutorial, we treat DLMs not as a solved replacement for autoregressive language models, but as a rapidly developing paradigm whose efficiency claims, controllability, practical assessment, and deployment implications require careful examination. We close by discussing representative open directions, from broader formulations such as flow-based language modeling to reliability, safety, and social bias as key requirements for the social acceptability of DLMs.

Questions about the tutorial are welcome. Please get in touch with Daisuke Oba or Danushka Bollegala.