Blog Link: https://s-sahoo.com/mdlm/
Discrete Diffusion
Text you can’t naturally “add little Gaussian noise” because a word is a discrete choice instead of a continuous value
Discrete diffusion takes the distribution of categories and gradually corrupts them towards some simple distribution