Diff-course Lecture 02 - Variational Perspective From VAEs to DDPMs

In this lecture, we introduce the diffusion models from the variational perspective. We will present the ideas of autoencoders (AEs), variational autoencoders (VAEs), denoising diffusion probabilistic models (DDPMs), discrete variational autoencoders (DVAEs), and discrete denoising diffusion probabilistic models (D3PMs).

Published

14 December 2025

This blog explains the main content of Chapter 2 in The Principles of Diffusion Models. We will explain the diffusion model from the variational perspective. We first introduce the original architecture of autoencoder. We then explain the development of variational autoencoder. The limitation of variational autoencoder leads to the development of diffusion model, i.e., denoising diffusion probabilistic model (DDPM). From the discrete data modeling perspective, we then see how the discrete variational autoencoder discrete VAE and the discrete denoising diffusion probabilistic models.

1. Autoencoder

The autoencoder model was originally developed by Hinton and Salakhutdinov (Hinton & Salakhutdinov, 2006)

  1. Hinton, G. E., & Salakhutdinov, R. R. (2006). Reducing the dimensionality of data with neural networks. Science, 313(5786), 504–507.