ML4EOContent License: Creative Commons Attribution 4.0 International (CC-BY-4.0)Credit must be given to the creatorDownloadsDownloadGeo-ModelingJ. Emmanuel JohnsonCSIC UCM IGEO Unconditional Density Estimation¶Data¶D={yn},yn∈RDy\begin{aligned} \mathcal{D} &=\left\{y_n \right\}, && && y_n\in\mathbb{R}^{D_y} \end{aligned}D={yn},yn∈RDy(1)Parametric¶Joint Distribution:p(y,θ)=p(y∣θ)p(θ)\begin{aligned} \text{Joint Distribution}: && && p(y,\theta) &= p(y|\theta)p(\theta) \end{aligned}Joint Distribution:p(y,θ)=p(y∣θ)p(θ)(2)Generative¶Joint Distribution:p(y,z,θ)=p(y∣z,θ)p(z∣θ)p(θ)\begin{aligned} \text{Joint Distribution}: && && p(y,z,\theta) &= p(y|z,\theta)p(z|\theta)p(\theta) \end{aligned}Joint Distribution:p(y,z,θ)=p(y∣z,θ)p(z∣θ)p(θ)(3)Conditional Density Estimation¶Data¶D={yn,xn},yn∈RDyxn∈RDx\begin{aligned} \mathcal{D} &=\left\{y_n, x_n \right\}, && && y_n\in\mathbb{R}^{D_y} && x_n\in\mathbb{R}^{D_x} \end{aligned}D={yn,xn},yn∈RDyxn∈RDx(4)Parametric¶Joint Distribution:p(y,x,θ)=p(y∣x,θ)p(θ)\begin{aligned} \text{Joint Distribution}: && && p(y,x,\theta) &= p(y|x,\theta)p(\theta) \end{aligned}Joint Distribution:p(y,x,θ)=p(y∣x,θ)p(θ)(5)Generative¶Joint Distribution I:p(y,x,z,θ)=p(y∣z,x,θ)p(z∣θ)p(θ)Joint Distribution II:p(y,x,z,θ)=p(y∣z,θ)p(z∣x,θ)p(θ)Joint Distribution III:p(y,x,z,θ)=p(y∣x,z,θ)p(z∣x,θ)p(θ)\begin{aligned} \text{Joint Distribution I}: && && p(y,x,z,\theta) &= p(y|z,x,\theta)p(z|\theta)p(\theta) \\ \text{Joint Distribution II}: && && p(y,x,z,\theta) &= p(y|z,\theta)p(z|x,\theta)p(\theta) \\ \text{Joint Distribution III}: && && p(y,x,z,\theta) &= p(y|x,z,\theta)p(z|x,\theta)p(\theta) \\ \end{aligned}Joint Distribution I:Joint Distribution II:Joint Distribution III:p(y,x,z,θ)p(y,x,z,θ)p(y,x,z,θ)=p(y∣z,x,θ)p(z∣θ)p(θ)=p(y∣z,θ)p(z∣x,θ)p(θ)=p(y∣x,z,θ)p(z∣x,θ)p(θ)(6)Dynamical Models¶Observations¶D={yt}t=1T,yt∈RDy\begin{aligned} \mathcal{D} &=\left\{y_t \right\}_{t=1}^T, && && y_t\in\mathbb{R}^{D_y} \end{aligned}D={yt}t=1T,yt∈RDy(7)Parametric (Global, IID)¶p(y1:T,θ)=p(θ)∏t=1Tp(yt∣θ)\begin{aligned} p(y_{1:T},\theta) &= p(\theta)\prod_{t=1}^T p(y_t|\theta) \end{aligned}p(y1:T,θ)=p(θ)t=1∏Tp(yt∣θ)(8)Parametric (Local)¶p(y1:T,θ0:T)=p(θ0)∏t=1Tp(yt∣θt)\begin{aligned} p(y_{1:T},\theta_{0:T}) &= p(\theta_0)\prod_{t=1}^T p(y_t|\theta_t) \end{aligned}p(y1:T,θ0:T)=p(θ0)t=1∏Tp(yt∣θt)(9)Generative¶p(y1:T,z1:T,θ)=p(θ)p(z0)∏t=1Tp(yt∣zt,θ)p(zt∣zt−1,θ)\begin{aligned} p(y_{1:T},z_{1:T},\theta) &= p(\theta)p(z_0)\prod_{t=1}^T p(y_t|z_t,\theta)p(z_t|z_{t-1},\theta) \end{aligned}p(y1:T,z1:T,θ)=p(θ)p(z0)t=1∏Tp(yt∣zt,θ)p(zt∣zt−1,θ)(10)Conditional Generative¶p(y1:T,x1:T,z1:T,θ)=p(θ)p(z0)∏t=1Tp(yt∣zt,θ)p(zt∣zt−1,xt,θ)\begin{aligned} p(y_{1:T},x_{1:T},z_{1:T},\theta) &= p(\theta)p(z_0)\prod_{t=1}^T p(y_t|z_t,\theta)p(z_t|z_{t-1}, x_{t},\theta) \end{aligned}p(y1:T,x1:T,z1:T,θ)=p(θ)p(z0)t=1∏Tp(yt∣zt,θ)p(zt∣zt−1,xt,θ)(11)Dynamical¶p(y1:T,u0:T,θ)=p(θ)p(u0)∏t=1Tp(yt∣ut,θ)p(ut∣ut−1,θ)\begin{aligned} p(y_{1:T},u_{0:T},\theta) &= p(\theta)p(u_0)\prod_{t=1}^T p(y_t|u_t,\theta)p(u_t|u_{t-1},\theta) \end{aligned}p(y1:T,u0:T,θ)=p(θ)p(u0)t=1∏Tp(yt∣ut,θ)p(ut∣ut−1,θ)(12)Conditional Dynamical¶p(y1:T,x1:T,u0:T,θ)=p(θ)p(u0)∏t=1Tp(yt∣ut,xt,θ)p(ut∣ut−1,xt,θ)\begin{aligned} p(y_{1:T},x_{1:T}, u_{0:T},\theta) &= p(\theta)p(u_0)\prod_{t=1}^T p(y_t|u_t,x_t,\theta)p(u_t|u_{t-1}, x_{t},\theta) \end{aligned}p(y1:T,x1:T,u0:T,θ)=p(θ)p(u0)t=1∏Tp(yt∣ut,xt,θ)p(ut∣ut−1,xt,θ)(13)Generative Dynamical¶p(y1:T,u1:T,z0:T,θ)=p(θ)p(z0)∏t=1Tp(yt∣ut,θ)p(ut∣zt,θ)p(zt∣zt−1,θ)\begin{aligned} p(y_{1:T},u_{1:T}, z_{0:T},\theta) &= p(\theta)p(z_0)\prod_{t=1}^T p(y_t|u_t,\theta)p(u_t|z_{t}, \theta)p(z_t|z_{t-1},\theta) \end{aligned}p(y1:T,u1:T,z0:T,θ)=p(θ)p(z0)t=1∏Tp(yt∣ut,θ)p(ut∣zt,θ)p(zt∣zt−1,θ)(14)Conditional Generative Dynamical¶p(y1:T,u1:T,x1:T,z0:T,θ)=p(θ)p(z0)∏t=1Tp(yt∣ut,xt,θ)p(ut∣zt,xt,θ)p(zt∣zt−1,θ)\begin{aligned} p(y_{1:T},u_{1:T}, x_{1:T},z_{0:T},\theta) &= p(\theta)p(z_0)\prod_{t=1}^T p(y_t|u_t,x_t,\theta)p(u_t|z_{t}, x_{t},\theta)p(z_t|z_{t-1},\theta) \end{aligned}p(y1:T,u1:T,x1:T,z0:T,θ)=p(θ)p(z0)t=1∏Tp(yt∣ut,xt,θ)p(ut∣zt,xt,θ)p(zt∣zt−1,θ)(15)