Skip to content

KL Divergence

Typical:

\text{KL}\left[ q(\mathbf x) || \mathcal{P}(\mathbf x)\right] = \int_\mathcal{X} q(\mathbf x) \log \frac{\mathcal{P}(\mathbf x)}{q(\mathbf x)} d\mathbf x

VI:

\text{KL}\left[ q(\mathbf x) || \mathcal{P}(\mathbf x)\right] = - \int_\mathcal{X} q(\mathbf x) \log \frac{\mathcal{P}(\mathbf x)}{q(\mathbf x)} d\mathbf x

Positive and Reverse KL