Skip to article frontmatterSkip to article content
ML4EO

Concept - Game of Dependencies

CSIC
UCM
IGEO
  • Univariate Time Series
  • Multiple Univariate Time Series
  • Univariate Spatiotemporal Series
  • Multiple Univariate Spatiotemporal Series

Univariate Time Series

In this case, we have a univariate time series. For this, we will investigate how we can model a time series.

Tutorials:

  • Global Mean Surface Temperature Anomaly
  • Single Weather Station for Spain

Unconditional Density Estimation

D={yn}n=1N,N=NTyn∈RDy\begin{aligned} \mathcal{D} &= \left\{ y_n \right\}_{n=1}^N, && && N = N_T && && y_n \in\mathbb{R}^{D_y} \end{aligned}
(1)

We have a few types of models we can use when we are faced with this situation.

Fully Pooled:p(y,z,θ)=p(θ)p(z∣θ)∏n=1Np(yn∣z)Non-Pooled:p(y,z,θ)=∏n=1Np(yn∣zn)p(zn∣θn)p(θn)Partially-Pooled:p(y,z,θ)=p(θ)∏n=1Np(yn∣zn)p(zn∣θ)\begin{aligned} \text{Fully Pooled}: && && p(\mathbf{y},\mathbf{z},\boldsymbol{\theta}) &= p(\boldsymbol{\theta}) p(\mathbf{z}|\boldsymbol{\theta}) \prod_{n=1}^N p(\mathbf{y}_n|\mathbf{z}) \\ \text{Non-Pooled}: && && p(\mathbf{y},\mathbf{z},\boldsymbol{\theta}) &= \prod_{n=1}^N p(y_n|\mathbf{z}_n) p(\mathbf{z}_n|\boldsymbol{\theta}_n) p(\boldsymbol{\theta}_n) \\ \text{Partially-Pooled}: && && p(\mathbf{y},\mathbf{z},\boldsymbol{\theta}) &= p(\boldsymbol{\theta})\prod_{n=1}^N p(y_n|\mathbf{z}_n)p(\mathbf{z}_n|\boldsymbol{\theta}) \\ \end{aligned}
(2)

The conclusion of this demonstration is that it’s almost always favourable to use a partially-pooled model as it effectively gives us options for modeling the dynamics of the parameters.

Temporally Conditioned Density Estimation

We can use conditional density estimation but we only condition on the time component. In this case, we have some pairwise entries of measurements, yny_n, at some associated time stamp, tnt_n.

D={tn,yn}n=1Nyn∈RDytn∈R+ \begin{aligned} \mathcal{D} &= \left\{ t_n, y_n \right\}_{n=1}^N && && y_n \in\mathbb{R}^{D_y} && && t_n\in\mathbb{R}^+ \end{aligned}
(3)

where N=NTN=N_T.

Measurements:y∈RNTyn∈RTime Stamps:t∈RNTtn∈R+Latent Variables:Z∈RNT×Dzzn∈RDzParameters:θ∈RDθ\begin{aligned} \text{Measurements}: && && \mathbf{y} &\in\mathbb{R}^{N_T} && && y_n \in\mathbb{R}\\ \text{Time Stamps}: && && \mathbf{t} &\in\mathbb{R}^{N_T} && && t_n \in\mathbb{R}^+\\ \text{Latent Variables}: && && \mathbf{Z} &\in\mathbb{R}^{N_T\times D_z} && && \mathbf{z}_n \in\mathbb{R}^{D_z}\\ \text{Parameters}: && && \boldsymbol{\theta} &\in\mathbb{R}^{D_\theta} \\ \end{aligned}
(4)

Here, we need to use a conditional

p(y,t,Z,θ)=p(θ)∏t=1NTp(yn∣zn)p(zn∣tn,θ)p(\mathbf{y},\mathbf{t},\mathbf{Z},\boldsymbol{\theta}) = p(\boldsymbol{\theta}) \prod_{t=1}^{N_T} p(y_n|\mathbf{z}_n) p(\mathbf{z}_n|t_n,\boldsymbol{\theta})
(5)

Unconditional Dynamic Model

D={tn,yt}n=1Nyt∈RDyt∈R+\begin{aligned} \mathcal{D} &= \left\{ t_n, y_t \right\}_{n=1}^N && && y_t \in\mathbb{R}^{D_y} && && t\in\mathbb{R}^+ \end{aligned}
(6)

where N=NTN=N_T.

Measurements:y∈RNTyt∈RLatent Variables:Z∈RNT×Dzzt∈RDzParameters:θ∈RDθ\begin{aligned} \text{Measurements}: && && \mathbf{y} &\in\mathbb{R}^{N_T} && && y_t \in\mathbb{R}\\ \text{Latent Variables}: && && \mathbf{Z} &\in\mathbb{R}^{N_T\times D_z} && && \mathbf{z}_t \in\mathbb{R}^{D_z}\\ \text{Parameters}: && && \boldsymbol{\theta} &\in\mathbb{R}^{D_\theta} \\ \end{aligned}
(7)

Finally, we can write the joint distribution

Strong-Constrained:p(y,Z,θ)=p(θ)p(z0∣θ)∏t=1Tp(yt∣zt)p(zt∣z0)Weak-Constrained:p(y,z,θ)=p(θ)p(z0∣θ)∏t=1Tp(yt∣zt)p(zt∣zt−1,θ)\begin{aligned} \text{Strong-Constrained}: && && p(\mathbf{y},\mathbf{Z},\boldsymbol{\theta}) &= p(\boldsymbol{\theta}) p(\mathbf{z}_0|\boldsymbol{\theta}) \prod_{t=1}^{T} p(y_t|\mathbf{z}_t)p(\mathbf{z}_t|\mathbf{z}_0) \\ \text{Weak-Constrained}: && && p(\mathbf{y},\mathbf{z},\boldsymbol{\theta}) &= p(\boldsymbol{\theta}) p(\mathbf{z}_0|\boldsymbol{\theta}) \prod_{t=1}^{T} p(y_t|\mathbf{z}_t)p(\mathbf{z}_t|\mathbf{z}_{t-1},\boldsymbol{\theta}) \\ \end{aligned}
(8)

Multivariate Time Series

In this case, we have a univariate time series. For this, we will investigate how we can model a time series.

Tutorials:

  • Single Weather Station for Spain + Multiple Variables

Unconditional Density Estimation

D={yn}n=1N,N=NTyn∈RDy\begin{aligned} \mathcal{D} &= \left\{ \mathbf{y}_n \right\}_{n=1}^N, && && N = N_T && && \mathbf{y}_n \in\mathbb{R}^{D_y} \end{aligned}
(9)

We have a few types of models we can use when we are faced with this situation.

Partially-Pooled:p(Y,Z,θ)=p(θ)∏n=1Np(yn∣zn)p(zn∣θ)\begin{aligned} \text{Partially-Pooled}: && && p(\mathbf{Y},\mathbf{Z},\boldsymbol{\theta}) &= p(\boldsymbol{\theta})\prod_{n=1}^N p(\mathbf{y}_n|\mathbf{z}_n)p(\mathbf{z}_n|\boldsymbol{\theta}) \\ \end{aligned}
(10)

The conclusion of this demonstration is that it’s almost always favourable to use a partially-pooled model as it effectively gives us options for modeling the dynamics of the parameters.

Temporally Conditioned Density Estimation

We can use conditional density estimation but we only condition on the time component. In this case, we have some pairwise entries of measurements, yny_n, at some associated time stamp, tnt_n.

D={tn,yn}n=1Nyn∈RDytn∈R+ \begin{aligned} \mathcal{D} &= \left\{ t_n, \mathbf{y}_n \right\}_{n=1}^N && && \mathbf{y}_n \in\mathbb{R}^{D_y} && && t_n\in\mathbb{R}^+ \end{aligned}
(11)

where N=NTN=N_T.

Measurements:Y∈RNT×Dyyn∈RTime Stamps:t∈RNTtn∈R+Latent Variables:Z∈RNT×Dzzn∈RDzParameters:θ∈RDθ\begin{aligned} \text{Measurements}: && && \mathbf{Y} &\in\mathbb{R}^{N_T\times D_y} && && \mathbf{y}_n \in\mathbb{R}\\ \text{Time Stamps}: && && \mathbf{t} &\in\mathbb{R}^{N_T} && && t_n \in\mathbb{R}^+\\ \text{Latent Variables}: && && \mathbf{Z} &\in\mathbb{R}^{N_T\times D_z} && && \mathbf{z}_n \in\mathbb{R}^{D_z}\\ \text{Parameters}: && && \boldsymbol{\theta} &\in\mathbb{R}^{D_\theta} \\ \end{aligned}
(12)

Here, we need to use a conditional

p(Y,t,Z,θ)=p(θ)∏t=1NTp(yn∣zn)p(zn∣tn,θ)p(\mathbf{Y},\mathbf{t},\mathbf{Z},\boldsymbol{\theta}) = p(\boldsymbol{\theta}) \prod_{t=1}^{N_T} p(\mathbf{y}_n|\mathbf{z}_n) p(\mathbf{z}_n|t_n,\boldsymbol{\theta})
(13)

Unconditional Dynamic Model

D={tn,yt}n=1Nyt∈RDyt∈R+\begin{aligned} \mathcal{D} &= \left\{ t_n, \mathbf{y}_t \right\}_{n=1}^N && && \mathbf{y}_t \in\mathbb{R}^{D_y} && && t\in\mathbb{R}^+ \end{aligned}
(14)

where N=NTN=N_T.

Measurements:Y∈RNT×Dyyt∈RDyLatent Variables:Z∈RNT×Dzzt∈RDzParameters:θ∈RDθ\begin{aligned} \text{Measurements}: && && \mathbf{Y} &\in\mathbb{R}^{N_T\times D_y} && && \mathbf{y}_t \in\mathbb{R}^{D_y}\\ \text{Latent Variables}: && && \mathbf{Z} &\in\mathbb{R}^{N_T\times D_z} && && \mathbf{z}_t \in\mathbb{R}^{D_z}\\ \text{Parameters}: && && \boldsymbol{\theta} &\in\mathbb{R}^{D_\theta} \\ \end{aligned}
(15)

Finally, we can write the joint distribution

Strong-Constrained:p(Y,Z,θ)=p(θ)p(z0∣θ)∏t=1Tp(yt∣zt)p(zt∣z0)Weak-Constrained:p(Y,Z,θ)=p(θ)p(z0∣θ)∏t=1Tp(yt∣zt)p(zt∣zt−1,θ)\begin{aligned} \text{Strong-Constrained}: && && p(\mathbf{Y},\mathbf{Z},\boldsymbol{\theta}) &= p(\boldsymbol{\theta}) p(\mathbf{z}_0|\boldsymbol{\theta}) \prod_{t=1}^{T} p(\mathbf{y}_t|\mathbf{z}_t)p(\mathbf{z}_t|\mathbf{z}_0) \\ \text{Weak-Constrained}: && && p(\mathbf{Y},\mathbf{Z},\boldsymbol{\theta}) &= p(\boldsymbol{\theta}) p(\mathbf{z}_0|\boldsymbol{\theta}) \prod_{t=1}^{T} p(\mathbf{y}_t|\mathbf{z}_t)p(\mathbf{z}_t|\mathbf{z}_{t-1},\boldsymbol{\theta}) \\ \end{aligned}
(16)

Multivariate Spatiotemporal Series

In this case, we have a univariate time series. For this, we will investigate how we can model a time series.

Tutorials:

  • Multiple Weather Station for Spain + Multiple Variables

Unconditional Density Estimation

D={yn}n=1N,N=NTD=DyDΩyn∈RD\begin{aligned} \mathcal{D} &= \left\{ \mathbf{y}_n \right\}_{n=1}^N, && && N = N_T && && D = D_y D_\Omega && && \mathbf{y}_n \in\mathbb{R}^{D} \end{aligned}
(17)

We have a few types of models we can use when we are faced with this situation.

Partially-Pooled:p(Y,Z,θ)=p(θ)∏n=1Np(yn∣zn)p(zn∣θ)\begin{aligned} \text{Partially-Pooled}: && && p(\mathbf{Y},\mathbf{Z},\boldsymbol{\theta}) &= p(\boldsymbol{\theta})\prod_{n=1}^N p(\mathbf{y}_n|\mathbf{z}_n)p(\mathbf{z}_n|\boldsymbol{\theta}) \\ \end{aligned}
(18)

The conclusion of this demonstration is that it’s almost always favourable to use a partially-pooled model as it effectively gives us options for modeling the dynamics of the parameters.

Temporally Conditioned Density Estimation

Coordinate-Based

We can use conditional density estimation but we only condition on the time component. In this case, we have some pairwise entries of measurements, yny_n, at some associated time stamp, tnt_n.

D={(tn,sn),yn}n=1Nyn∈RDytn∈R+ \begin{aligned} \mathcal{D} &= \left\{ (t_n, \mathbf{s}_n), \mathbf{y}_n \right\}_{n=1}^N && && \mathbf{y}_n \in\mathbb{R}^{D_y} && && t_n\in\mathbb{R}^+ \end{aligned}
(19)

where N=NTN=N_T and D=DyD=D_y.

Measurements:Y∈RNT×Dyyn∈RDyTime Stamps:t∈RNTtn∈R+Spatial Coordinates:S∈RNT×Dssn∈RDsLatent Variables:Z∈RNT×Dzzn∈RDzParameters:θ∈RDθ\begin{aligned} \text{Measurements}: && && \mathbf{Y} &\in\mathbb{R}^{N_T\times D_y} && && \mathbf{y}_n \in\mathbb{R}^{D_y}\\ \text{Time Stamps}: && && \mathbf{t} &\in\mathbb{R}^{N_T} && && t_n \in\mathbb{R}^+\\ \text{Spatial Coordinates}: && && \mathbf{S} &\in\mathbb{R}^{N_T \times D_s} && && \mathbf{s}_n \in\mathbb{R}^{D_s}\\ \text{Latent Variables}: && && \mathbf{Z} &\in\mathbb{R}^{N_T\times D_z} && && \mathbf{z}_n \in\mathbb{R}^{D_z}\\ \text{Parameters}: && && \boldsymbol{\theta} &\in\mathbb{R}^{D_\theta} \\ \end{aligned}
(20)

Here, we need to use a conditional

p(Y,t,S,Z,θ)=p(θ)∏t=1NTp(yn∣zn)p(zn∣tn,sn,θ)p(\mathbf{Y},\mathbf{t},\mathbf{S},\mathbf{Z},\boldsymbol{\theta}) = p(\boldsymbol{\theta}) \prod_{t=1}^{N_T} p(\mathbf{y}_n|\mathbf{z}_n) p(\mathbf{z}_n|t_n,\mathbf{s}_n,\boldsymbol{\theta})
(21)

Field-Based

We can use conditional density estimation but we only condition on the time component. In this case, we have some pairwise entries of measurements, yny_n, at some associated time stamp, tnt_n.

D={tn,yn}n=1Nyn∈RDtn∈R+ \begin{aligned} \mathcal{D} &= \left\{ t_n, \mathbf{y}_n \right\}_{n=1}^N && && \mathbf{y}_n \in\mathbb{R}^{D} && && t_n\in\mathbb{R}^+ \end{aligned}
(22)

where N=NTN=N_T and D=DΩDyD=D_\Omega D_y.

Measurements:Y∈RNT×Dyn∈RDTime Stamps:t∈RNTtn∈R+Latent Variables:Z∈RNT×Dzzn∈RDzParameters:θ∈RDθ\begin{aligned} \text{Measurements}: && && \mathbf{Y} &\in\mathbb{R}^{N_T\times D} && && \mathbf{y}_n \in\mathbb{R}^D\\ \text{Time Stamps}: && && \mathbf{t} &\in\mathbb{R}^{N_T} && && t_n \in\mathbb{R}^+\\ \text{Latent Variables}: && && \mathbf{Z} &\in\mathbb{R}^{N_T\times D_z} && && \mathbf{z}_n \in\mathbb{R}^{D_z}\\ \text{Parameters}: && && \boldsymbol{\theta} &\in\mathbb{R}^{D_\theta} \\ \end{aligned}
(23)

Here, we need to use a conditional

p(Y,t,Z,θ)=p(θ)∏t=1NTp(yn∣zn)p(zn∣tn,θ)p(\mathbf{Y},\mathbf{t},\mathbf{Z},\boldsymbol{\theta}) = p(\boldsymbol{\theta}) \prod_{t=1}^{N_T} p(\mathbf{y}_n|\mathbf{z}_n) p(\mathbf{z}_n|t_n,\boldsymbol{\theta})
(24)

Unconditional Dynamic Model

D={tn,yt}n=1Nyt∈RDt∈R+\begin{aligned} \mathcal{D} &= \left\{ t_n, \mathbf{y}_t \right\}_{n=1}^N && && \mathbf{y}_t \in\mathbb{R}^{D} && && t\in\mathbb{R}^+ \end{aligned}
(25)

where N=NTN=N_T and D=DΩDyD=D_\Omega D_y.

Measurements:Y∈RNT×Dyt∈RDLatent Variables:Z∈RNT×Dzzt∈RDzParameters:θ∈RDθ\begin{aligned} \text{Measurements}: && && \mathbf{Y} &\in\mathbb{R}^{N_T\times D} && && \mathbf{y}_t \in\mathbb{R}^{D}\\ \text{Latent Variables}: && && \mathbf{Z} &\in\mathbb{R}^{N_T\times D_z} && && \mathbf{z}_t \in\mathbb{R}^{D_z}\\ \text{Parameters}: && && \boldsymbol{\theta} &\in\mathbb{R}^{D_\theta} \\ \end{aligned}
(26)

Finally, we can write the joint distribution

Strong-Constrained:p(Y,Z,θ)=p(θ)p(z0∣θ)∏t=1Tp(yt∣zt)p(zt∣z0)Weak-Constrained:p(Y,Z,θ)=p(θ)p(z0∣θ)∏t=1Tp(yt∣zt)p(zt∣zt−1,θ)\begin{aligned} \text{Strong-Constrained}: && && p(\mathbf{Y},\mathbf{Z},\boldsymbol{\theta}) &= p(\boldsymbol{\theta}) p(\mathbf{z}_0|\boldsymbol{\theta}) \prod_{t=1}^{T} p(\mathbf{y}_t|\mathbf{z}_t)p(\mathbf{z}_t|\mathbf{z}_0) \\ \text{Weak-Constrained}: && && p(\mathbf{Y},\mathbf{Z},\boldsymbol{\theta}) &= p(\boldsymbol{\theta}) p(\mathbf{z}_0|\boldsymbol{\theta}) \prod_{t=1}^{T} p(\mathbf{y}_t|\mathbf{z}_t)p(\mathbf{z}_t|\mathbf{z}_{t-1},\boldsymbol{\theta}) \\ \end{aligned}
(27)

Coupled Multivariate Spatiotemporal Series

In this case, we have a univariate time series. For this, we will investigate how we can model a time series.

Tutorials:

  • Multiple Weather Station for Spain + Multiple Variables

Conditional Density Estimation

D={xn,yn}n=1N,N=NTD=DyDΩyn∈RDxn∈RDx\begin{aligned} \mathcal{D} &= \left\{ \mathbf{x}_n, \mathbf{y}_n \right\}_{n=1}^N, && && N = N_T && && D = D_y D_\Omega && && \mathbf{y}_n \in\mathbb{R}^{D} && && \mathbf{x}_n \in\mathbb{R}^{D_x} \end{aligned}
(28)

We have a few types of models we can use when we are faced with this situation.

Partially-Pooled:p(Y,X,Z,θ)=p(θ)∏n=1Np(yn∣zn)p(zn∣xn,θ)\begin{aligned} \text{Partially-Pooled}: && && p(\mathbf{Y},\mathbf{X},\mathbf{Z},\boldsymbol{\theta}) &= p(\boldsymbol{\theta})\prod_{n=1}^N p(\mathbf{y}_n|\mathbf{z}_n)p(\mathbf{z}_n|\mathbf{x}_n,\boldsymbol{\theta}) \\ \end{aligned}
(29)

The conclusion of this demonstration is that it’s almost always favourable to use a partially-pooled model as it effectively gives us options for modeling the dynamics of the parameters.

Temporal Conditional Density Estimation

Coordinate-Based

We can use conditional density estimation but we only condition on the time component. In this case, we have some pairwise entries of measurements, yny_n, at some associated time stamp, tnt_n.

D={(tn,sn),xn,yn}n=1Nyn∈RDyxn∈RDxtn∈R+ \begin{aligned} \mathcal{D} &= \left\{ (t_n, \mathbf{s}_n), \mathbf{x}_n, \mathbf{y}_n \right\}_{n=1}^N && && \mathbf{y}_n \in\mathbb{R}^{D_y} && && \mathbf{x}_n \in\mathbb{R}^{D_x} && && t_n\in\mathbb{R}^+ \end{aligned}
(30)

where N=NTN=N_T and D=DyD=D_y.

Measurements:Y∈RNT×Dyyn∈RDyCovariates:X∈RNT×Dxxn∈RDxTime Stamps:t∈RNTtn∈R+Spatial Coordinates:S∈RNT×Dssn∈RDsLatent Variables:Z∈RNT×Dzzn∈RDzParameters:θ∈RDθ\begin{aligned} \text{Measurements}: && && \mathbf{Y} &\in\mathbb{R}^{N_T\times D_y} && && \mathbf{y}_n \in\mathbb{R}^{D_y}\\ \text{Covariates}: && && \mathbf{X} &\in\mathbb{R}^{N_T\times D_x} && && \mathbf{x}_n \in\mathbb{R}^{D_x}\\ \text{Time Stamps}: && && \mathbf{t} &\in\mathbb{R}^{N_T} && && t_n \in\mathbb{R}^+\\ \text{Spatial Coordinates}: && && \mathbf{S} &\in\mathbb{R}^{N_T \times D_s} && && \mathbf{s}_n \in\mathbb{R}^{D_s}\\ \text{Latent Variables}: && && \mathbf{Z} &\in\mathbb{R}^{N_T\times D_z} && && \mathbf{z}_n \in\mathbb{R}^{D_z}\\ \text{Parameters}: && && \boldsymbol{\theta} &\in\mathbb{R}^{D_\theta} \\ \end{aligned}
(31)

Here, we need to use a conditional

p(Y,X,t,S,Z,θ)=p(θ)∏t=1NTp(yn∣zn)p(zn∣tn,sn,xn,θ)p\left(\mathbf{Y},\mathbf{X},\mathbf{t},\mathbf{S},\mathbf{Z},\boldsymbol{\theta}\right) = p(\boldsymbol{\theta}) \prod_{t=1}^{N_T} p(\mathbf{y}_n|\mathbf{z}_n) p(\mathbf{z}_n|t_n,\mathbf{s}_n,\mathbf{x}_n, \boldsymbol{\theta})
(32)

Field-Based

We can use conditional density estimation but we only condition on the time component. In this case, we have some pairwise entries of measurements, yny_n, at some associated time stamp, tnt_n.

D={tn,xn,yn}n=1Nyn∈RDyxn∈RDxtn∈R+ \begin{aligned} \mathcal{D} &= \left\{ t_n, \mathbf{x}_n, \mathbf{y}_n \right\}_{n=1}^N && && \mathbf{y}_n \in\mathbb{R}^{D_y} && && \mathbf{x}_n \in\mathbb{R}^{D_x} && && t_n\in\mathbb{R}^+ \end{aligned}
(33)

where N=NTN=N_T and Dy=DΩDyD_y=D_\Omega D_y, Dx=DΩxD_x=D_{\Omega_x}.

Measurements:Y∈RNT×Dyn∈RDCovariates:X∈RNT×Dxxn∈RDxTime Stamps:t∈RNTtn∈R+Latent Variables:Z∈RNT×Dzzn∈RDzParameters:θ∈RDθ\begin{aligned} \text{Measurements}: && && \mathbf{Y} &\in\mathbb{R}^{N_T\times D} && && \mathbf{y}_n \in\mathbb{R}^D\\ \text{Covariates}: && && \mathbf{X} &\in\mathbb{R}^{N_T\times D_x} && && \mathbf{x}_n \in\mathbb{R}^{D_x}\\ \text{Time Stamps}: && && \mathbf{t} &\in\mathbb{R}^{N_T} && && t_n \in\mathbb{R}^+\\ \text{Latent Variables}: && && \mathbf{Z} &\in\mathbb{R}^{N_T\times D_z} && && \mathbf{z}_n \in\mathbb{R}^{D_z}\\ \text{Parameters}: && && \boldsymbol{\theta} &\in\mathbb{R}^{D_\theta} \\ \end{aligned}
(34)

Here, we need to use a conditional

p(Y,X,t,Z,θ)=p(θ)∏t=1NTp(yn∣zn)p(zn∣tn,xn,θ)p(\mathbf{Y},\mathbf{X},\mathbf{t},\mathbf{Z},\boldsymbol{\theta}) = p(\boldsymbol{\theta}) \prod_{t=1}^{N_T} p(\mathbf{y}_n|\mathbf{z}_n) p(\mathbf{z}_n|t_n,\mathbf{x}_n,\boldsymbol{\theta})
(35)

Conditional Dynamic Model

D={tn,xt,yt}n=1Nyt∈RDyxt∈RDxt∈R+\begin{aligned} \mathcal{D} &= \left\{ t_n, \mathbf{x}_t,\mathbf{y}_t \right\}_{n=1}^N && && \mathbf{y}_t \in\mathbb{R}^{D_y} && && \mathbf{x}_t \in\mathbb{R}^{D_x} && && t\in\mathbb{R}^+ \end{aligned}
(36)

where N=NTN=N_T and D=DΩDyD=D_\Omega D_y.

Measurements:Y∈RNT×Dyyt∈RDyCovariates:X∈RNT×Dxxt∈RDxLatent Variables:Z∈RNT×Dzzt∈RDzParameters:θ∈RDθ\begin{aligned} \text{Measurements}: && && \mathbf{Y} &\in\mathbb{R}^{N_T\times D_y} && && \mathbf{y}_t \in\mathbb{R}^{D_y}\\ \text{Covariates}: && && \mathbf{X} &\in\mathbb{R}^{N_T\times D_x} && && \mathbf{x}_t \in\mathbb{R}^{D_x}\\ \text{Latent Variables}: && && \mathbf{Z} &\in\mathbb{R}^{N_T\times D_z} && && \mathbf{z}_t \in\mathbb{R}^{D_z}\\ \text{Parameters}: && && \boldsymbol{\theta} &\in\mathbb{R}^{D_\theta} \\ \end{aligned}
(37)

Finally, we can write the joint distribution

Strong-Constrained:p(Y,X,Z,θ)=p(θ)p(z0∣θ)∏t=1Tp(yt∣zt)p(zt∣xt,z0)Weak-Constrained:p(Y,X,Z,θ)=p(θ)p(z0∣θ)∏t=1Tp(yt∣zt)p(zt∣xt,zt−1,θ)\begin{aligned} \text{Strong-Constrained}: && && p(\mathbf{Y},\mathbf{X},\mathbf{Z},\boldsymbol{\theta}) &= p(\boldsymbol{\theta}) p(\mathbf{z}_0|\boldsymbol{\theta}) \prod_{t=1}^{T} p(\mathbf{y}_t|\mathbf{z}_t)p(\mathbf{z}_t|\mathbf{x}_t,\mathbf{z}_0) \\ \text{Weak-Constrained}: && && p(\mathbf{Y},\mathbf{X},\mathbf{Z},\boldsymbol{\theta}) &= p(\boldsymbol{\theta}) p(\mathbf{z}_0|\boldsymbol{\theta}) \prod_{t=1}^{T} p(\mathbf{y}_t|\mathbf{z}_t)p(\mathbf{z}_t|\mathbf{x}_t,\mathbf{z}_{t-1},\boldsymbol{\theta}) \\ \end{aligned}
(38)
Parameterizations
Parameterization - TOC
Parameterizations
Anatomy of PDEs