Skip to article frontmatterSkip to article content

Overview

  • Independent Realization
  • Time
  • Space
  • Variable

Unconditional Density Estimation

IID Data

D={yn}n=1N\mathcal{D} = \{ \mathbf{y}_n\}_{n=1}^N
(1)

A more convenient way to represent this is to show the stacked matrices.

Measurements:Y∈RN×Dyyn∈R\begin{aligned} \text{Measurements}: && && \mathbf{Y} &\in\mathbb{R}^{N\times D_y} && && \mathbf{y}_n \in\mathbb{R} \end{aligned}
(2)

Examples:

  • Ensembles
  • Batches
  • Patches

Fully Pooled Model (Temperature, Precipitation)

p(Y,θ)=p(θ)∏n=1Np(yn∣θ)p(\mathbf{Y},\boldsymbol{\theta}) = p(\boldsymbol{\theta})\prod_{n=1}^N p(\mathbf{y}_n|\boldsymbol{\theta})
(3)

Non-Pooled Model

p(Y,θ)=∏n=1Np(yn∣θn)p(\mathbf{Y},\boldsymbol{\theta}) = \prod_{n=1}^N p(\mathbf{y}_n|\boldsymbol{\theta}_n)
(4)

Partially Pooled Model

p(Y,Z,θ)=p(θ)∏n=1Np(yn∣zn)p(zn∣θ)p(\mathbf{Y},\mathbf{Z},\boldsymbol{\theta}) = p(\boldsymbol{\theta})\prod_{n=1}^N p(\mathbf{y}_n|\mathbf{z}_n)p(\mathbf{z}_n|\boldsymbol{\theta})
(5)

Time Series Data

D={tn,yn}n=1NT,yn∈RDytn∈R+\begin{aligned} \mathcal{D} &= \left\{ t_n, \mathbf{y}_n \right\}_{n=1}^{N_T}, && && \mathbf{y}_n \in\mathbb{R}^{D_y} && && t_n\in\mathbb{R}^+ \end{aligned}
(6)

A more convenient way to represent this is to show the stacked matrices.

Measurements:Y∈RNT×Dyyn∈RTime Stamps:t∈RNTtn∈R+\begin{aligned} \text{Measurements}: && && \mathbf{Y} &\in\mathbb{R}^{N_T\times D_y} && && \mathbf{y}_n \in\mathbb{R}\\ \text{Time Stamps}: && && \mathbf{t} &\in\mathbb{R}^{N_T} && && t_n \in\mathbb{R}^+ \end{aligned}
(7)

Temporally Conditioned Model

p(Y,t,Z,θ)=p(θ)∏n=1NTp(yn∣zn)p(zn∣tn,θ)p(\mathbf{Y},\mathbf{t},\mathbf{Z},\boldsymbol{\theta}) = p(\boldsymbol{\theta}) \prod_{n=1}^{N_T} p(\mathbf{y}_n|\mathbf{z}_n)p(\mathbf{z}_n|t_n,\boldsymbol{\theta})
(8)

Dynamical Model

p(Y,Z,θ)=p(θ)p(z0∣θ)∏t=1Tp(yt∣zt)p(zt∣zt−1,θ)p(\mathbf{Y},\mathbf{Z},\boldsymbol{\theta}) = p(\boldsymbol{\theta}) p(\mathbf{z}_0|\boldsymbol{\theta}) \prod_{t=1}^T p(\mathbf{y}_t|\mathbf{z}_t)p(\mathbf{z}_t|\mathbf{z}_{t-1},\boldsymbol{\theta})
(9)

Spatial Field Data

D={sm,ym}m=1NΩ\mathcal{D} = \{\mathbf{s}_m,\mathbf{y}_m\}_{m=1}^{N_\Omega}
(10)

A more convenient way to represent this is to show the stacked matrices.

Measurements:Y∈RNΩ×Dyyn∈RDySpatial Coordinates:S∈RNΩ×Dssn∈RDs\begin{aligned} \text{Measurements}: && && \mathbf{Y} &\in\mathbb{R}^{N_\Omega\times D_y} && && \mathbf{y}_n \in\mathbb{R}^{D_y}\\ \text{Spatial Coordinates}: && && \mathbf{S} &\in\mathbb{R}^{N_\Omega\times D_s} && && \mathbf{s}_n \in\mathbb{R}^{D_s} \end{aligned}
(11)

Spatially Conditioned Model

p(Y,S,Z,θ)=p(θ)∏n=1NTp(yn∣zn)p(zn∣sn,θ)p(\mathbf{Y},\mathbf{S},\mathbf{Z},\boldsymbol{\theta}) = p(\boldsymbol{\theta})\prod_{n=1}^{N_T} p(\mathbf{y}_n|\mathbf{z}_n)p(\mathbf{z}_n|\mathbf{s}_n,\boldsymbol{\theta})
(12)

Spatio-Temporal Data

D={tn,sm,ynm}n=1,m=1NT,NΩ\mathcal{D} = \{t_n, \mathbf{s}_m,\mathbf{y}_{nm}\}_{n=1,m=1}^{N_T,N_\Omega}
(13)

A more convenient way to represent this is to show the stacked matrices.

Measurements:Y∈RNT×Dyyn∈RTime Stamps:t∈RNTtn∈R+Spatial Coordinates:S∈RNΩ×Dssn∈RDs\begin{aligned} \text{Measurements}: && && \mathbf{Y} &\in\mathbb{R}^{N_T\times D_y} && && \mathbf{y}_n \in\mathbb{R}\\ \text{Time Stamps}: && && \mathbf{t} &\in\mathbb{R}^{N_T} && && t_n \in\mathbb{R}^+ \\ \text{Spatial Coordinates}: && && \mathbf{S} &\in\mathbb{R}^{N_\Omega\times D_s} && && \mathbf{s}_n \in\mathbb{R}^{D_s} \end{aligned}
(14)

Spatiotemporal Conditioned Model

p(Y,t,S,Z,θ)=p(θ)∏n=1NT∏m=1NΩp(ynm∣znm)p(znm∣tn,sm,θ)p(\mathbf{Y},\mathbf{t},\mathbf{S},\mathbf{Z},\boldsymbol{\theta}) = p(\boldsymbol{\theta}) \prod_{n=1}^{N_T} \prod_{m=1}^{N_\Omega} p(\mathbf{y}_{nm}|\mathbf{z}_{nm})p(\mathbf{z}_{nm}|t_n, \mathbf{s}_{m},\boldsymbol{\theta})
(15)

Dynamical Model

p(Y,Z,θ)=p(θ)p(z0∣θ)∏t=1Tp(yt∣zt)p(zt∣zt−1,θ)p(\mathbf{Y},\mathbf{Z},\boldsymbol{\theta}) = p(\boldsymbol{\theta}) p(\mathbf{z}_0|\boldsymbol{\theta}) \prod_{t=1}^T p(\mathbf{y}_t|\mathbf{z}_t)p(\mathbf{z}_t|\mathbf{z}_{t-1},\boldsymbol{\theta})
(16)

Conditional Density Estimation

IID Data

D={xn,yn}n=1N\mathcal{D} = \{ \mathbf{x}_n,\mathbf{y}_n\}_{n=1}^N
(17)

A more convenient way to represent this is to show the stacked matrices.

Measurements:Y∈RNT×Dyyn∈RDyCovariates:X∈RNT×Dxxn∈RDx\begin{aligned} \text{Measurements}: && && \mathbf{Y} &\in\mathbb{R}^{N_T\times D_y} && && \mathbf{y}_n \in\mathbb{R}^{D_y}\\ \text{Covariates}: && && \mathbf{X} &\in\mathbb{R}^{N_T \times D_x} && && \mathbf{x}_n \in\mathbb{R}^{D_x} \end{aligned}
(18)

Non-Pooled Model

p(Y,X,Z,θ)=∏n=1Np(yn∣zn)p(zn∣xn,θn)p(\mathbf{Y},\mathbf{X},\mathbf{Z},\boldsymbol{\theta}) = \prod_{n=1}^N p(\mathbf{y}_n|\mathbf{z}_n)p(\mathbf{z}_n|\mathbf{x}_n,\boldsymbol{\theta}_n)
(19)

Partially Pooled Model

p(Y,X,Z,θ)=p(θ)∏n=1Np(yn∣zn)p(zn∣xn,θ)p(\mathbf{Y},\mathbf{X},\mathbf{Z},\boldsymbol{\theta}) = p(\boldsymbol{\theta})\prod_{n=1}^N p(\mathbf{y}_n|\mathbf{z}_n)p(\mathbf{z}_n|\mathbf{x}_n,\boldsymbol{\theta})
(20)

Time Series Data

D={tn,xn,yn}n=1N\mathcal{D} = \{ t_n, \mathbf{x}_n, \mathbf{y}_n\}_{n=1}^N
(21)

A more convenient way to represent this is to show the stacked matrices.

Measurements:Y∈RNT×Dyyn∈RDyCovariates:X∈RNT×Dxxn∈RDxTime Stamps:t∈RNTtn∈R+\begin{aligned} \text{Measurements}: && && \mathbf{Y} &\in\mathbb{R}^{N_T\times D_y} && && \mathbf{y}_n \in\mathbb{R}^{D_y}\\ \text{Covariates}: && && \mathbf{X} &\in\mathbb{R}^{N_T \times D_x} && && \mathbf{x}_n \in\mathbb{R}^{D_x} \\ \text{Time Stamps}: && && \mathbf{t} &\in\mathbb{R}^{N_T} && && t_n \in\mathbb{R}^+ \end{aligned}
(22)

Temporally Conditioned Model

p(Y,t,X,Z,θ)=p(θ)∏n=1NTp(yn∣zn)p(zn∣tn,xn,θ)p(\mathbf{Y},\mathbf{t},\mathbf{X},\mathbf{Z},\boldsymbol{\theta}) = p(\boldsymbol{\theta}) \prod_{n=1}^{N_T} p(\mathbf{y}_{n}|\mathbf{z}_n)p(\mathbf{z}_n|t_n,\mathbf{x}_n,\boldsymbol{\theta})
(23)

Dynamical Model

p(Y,X,Z,θ)=p(θ)p(z0∣θ)∏t=1Tp(yt∣zt)p(zt∣zt−1,xt,θ)p(\mathbf{Y},\mathbf{X},\mathbf{Z},\boldsymbol{\theta}) = p(\boldsymbol{\theta}) p(\mathbf{z}_0|\boldsymbol{\theta}) \prod_{t=1}^T p(\mathbf{y}_t|\mathbf{z}_t)p(\mathbf{z}_t|\mathbf{z}_{t-1},\mathbf{x}_t,\boldsymbol{\theta})
(24)

Spatial Field Data

D={sn,xn,yn}n=1N\mathcal{D} = \{\mathbf{s}_n,\mathbf{x}_n,\mathbf{y}_n\}_{n=1}^N
(25)

A more convenient way to represent this is to show the stacked matrices.

Measurements:Y∈RNΩ×Dyyn∈RDyCovariates:X∈RNΩ×Dxxn∈RDxSpatial Coordinates:S∈RNΩ×Dssn∈RDs\begin{aligned} \text{Measurements}: && && \mathbf{Y} &\in\mathbb{R}^{N_\Omega\times D_y} && && \mathbf{y}_n \in\mathbb{R}^{D_y}\\ \text{Covariates}: && && \mathbf{X} &\in\mathbb{R}^{N_\Omega \times D_x} && && \mathbf{x}_n \in\mathbb{R}^{D_x} \\ \text{Spatial Coordinates}: && && \mathbf{S} &\in\mathbb{R}^{N_\Omega\times D_s} && && \mathbf{s}_n \in\mathbb{R}^{D_s} \end{aligned}
(26)

Spatially Conditioned Model

p(Y,S,X,Z,θ)=p(θ)∏m=1NΩp(ym∣zm)p(zm∣sm,xm,θ)p(\mathbf{Y},\mathbf{S},\mathbf{X},\mathbf{Z},\boldsymbol{\theta}) = p(\boldsymbol{\theta}) \prod_{m=1}^{N_\Omega} p(\mathbf{y}_m|\mathbf{z}_m)p(\mathbf{z}_m|\mathbf{s}_m,\mathbf{x}_m,\boldsymbol{\theta})
(27)

Spatio-Temporal Data

D={tn,sm,xnm,ynm}n=1,m=1NT,NΩ,N=NTNΩ\begin{aligned} \mathcal{D} &= \{t_n, \mathbf{s}_m, \mathbf{x}_{nm}, \mathbf{y}_{nm}\}_{n=1,m=1}^{N_T,N_\Omega}, && && N = N_TN_\Omega \end{aligned}
(28)

A more convenient way to represent this is to show the stacked matrices.

Measurements:Y∈RNT×NΩ×Dyyn∈RDyCovariates:X∈RNΩ×Dxxn∈RDxTime Stamps:t∈RNTtn∈R+Spatial Coordinates:S∈RNΩ×Dssn∈RDs\begin{aligned} \text{Measurements}: && && \mathbf{Y} &\in\mathbb{R}^{N_T\times N_\Omega \times D_y} && && \mathbf{y}_n \in\mathbb{R}^{D_y}\\ \text{Covariates}: && && \mathbf{X} &\in\mathbb{R}^{N_\Omega \times D_x} && && \mathbf{x}_n \in\mathbb{R}^{D_x} \\ \text{Time Stamps}: && && \mathbf{t} &\in\mathbb{R}^{N_T} && && t_n \in\mathbb{R}^+ \\ \text{Spatial Coordinates}: && && \mathbf{S} &\in\mathbb{R}^{N_\Omega\times D_s} && && \mathbf{s}_n \in\mathbb{R}^{D_s} \end{aligned}
(29)

Spatiotemporal Conditioned Model

p(Y,X,t,S,Z,θ)=p(θ)∏n=1NT∏m=1NΩp(ynm∣znm)p(znm∣sm,tn,xnm,θ)p(\mathbf{Y},\mathbf{X},\mathbf{t},\mathbf{S},\mathbf{Z},\boldsymbol{\theta}) = p(\boldsymbol{\theta}) \prod_{n=1}^{N_T} \prod_{m=1}^{N_\Omega} p(\mathbf{y}_{nm}|\mathbf{z}_{nm}) p(\mathbf{z}_{nm}|\mathbf{s}_m,t_n,\mathbf{x}_{nm},\boldsymbol{\theta})
(30)

Dynamical Model

p(Y,X,Z,θ)=p(θ)p(z0∣θ)∏t=1Tp(yt∣zt)p(zt∣zt−1,xt,θ)p(\mathbf{Y},\mathbf{X},\mathbf{Z},\boldsymbol{\theta}) = p(\boldsymbol{\theta}) p(\mathbf{z}_0|\boldsymbol{\theta}) \prod_{t=1}^T p(\mathbf{y}_t|\mathbf{z}_t)p(\mathbf{z}_t|\mathbf{z}_{t-1},\mathbf{x}_t,\boldsymbol{\theta})
(31)
Parameterization
Overview
Parameterization
Dynamical System