Concept - Game of Dependencies
Univariate Time Series
Multiple Univariate Time Series
Univariate Spatiotemporal Series
Multiple Univariate Spatiotemporal Series
Univariate Time Series ¶ In this case, we have a univariate time series.
For this, we will investigate how we can model a time series.
Tutorials :
Unconditional Density Estimation ¶ D = { y n } n = 1 N , N = N T y n ∈ R D y \begin{aligned}
\mathcal{D} &= \left\{ y_n \right\}_{n=1}^N, && &&
N = N_T && &&
y_n \in\mathbb{R}^{D_y}
\end{aligned} D = { y n } n = 1 N , N = N T y n ∈ R D y We have a few types of models we can use when we are faced with this situation.
Fully Pooled : p ( y , z , θ ) = p ( θ ) p ( z ∣ θ ) ∏ n = 1 N p ( y n ∣ z ) Non-Pooled : p ( y , z , θ ) = ∏ n = 1 N p ( y n ∣ z n ) p ( z n ∣ θ n ) p ( θ n ) Partially-Pooled : p ( y , z , θ ) = p ( θ ) ∏ n = 1 N p ( y n ∣ z n ) p ( z n ∣ θ ) \begin{aligned}
\text{Fully Pooled}: && &&
p(\mathbf{y},\mathbf{z},\boldsymbol{\theta})
&=
p(\boldsymbol{\theta})
p(\mathbf{z}|\boldsymbol{\theta})
\prod_{n=1}^N
p(\mathbf{y}_n|\mathbf{z}) \\
\text{Non-Pooled}: && &&
p(\mathbf{y},\mathbf{z},\boldsymbol{\theta})
&=
\prod_{n=1}^N
p(y_n|\mathbf{z}_n)
p(\mathbf{z}_n|\boldsymbol{\theta}_n)
p(\boldsymbol{\theta}_n) \\
\text{Partially-Pooled}: && &&
p(\mathbf{y},\mathbf{z},\boldsymbol{\theta})
&=
p(\boldsymbol{\theta})\prod_{n=1}^N
p(y_n|\mathbf{z}_n)p(\mathbf{z}_n|\boldsymbol{\theta}) \\
\end{aligned} Fully Pooled : Non-Pooled : Partially-Pooled : p ( y , z , θ ) p ( y , z , θ ) p ( y , z , θ ) = p ( θ ) p ( z ∣ θ ) n = 1 ∏ N p ( y n ∣ z ) = n = 1 ∏ N p ( y n ∣ z n ) p ( z n ∣ θ n ) p ( θ n ) = p ( θ ) n = 1 ∏ N p ( y n ∣ z n ) p ( z n ∣ θ ) The conclusion of this demonstration is that it’s almost always favourable to use a partially-pooled model as it effectively gives us options for modeling the dynamics of the parameters.
Temporally Conditioned Density Estimation ¶ We can use conditional density estimation but we only condition on the time component.
In this case, we have some pairwise entries of measurements, y n y_n y n t n t_n t n
D = { t n , y n } n = 1 N y n ∈ R D y t n ∈ R + \begin{aligned}
\mathcal{D} &= \left\{ t_n, y_n \right\}_{n=1}^N
&& &&
y_n \in\mathbb{R}^{D_y}
&& &&
t_n\in\mathbb{R}^+
\end{aligned} D = { t n , y n } n = 1 N y n ∈ R D y t n ∈ R + where N = N T N=N_T N = N T
Measurements : y ∈ R N T y n ∈ R Time Stamps : t ∈ R N T t n ∈ R + Latent Variables : Z ∈ R N T × D z z n ∈ R D z Parameters : θ ∈ R D θ \begin{aligned}
\text{Measurements}: && &&
\mathbf{y} &\in\mathbb{R}^{N_T}
&& &&
y_n \in\mathbb{R}\\
\text{Time Stamps}: && &&
\mathbf{t} &\in\mathbb{R}^{N_T} && &&
t_n \in\mathbb{R}^+\\
\text{Latent Variables}: && &&
\mathbf{Z} &\in\mathbb{R}^{N_T\times D_z} && &&
\mathbf{z}_n \in\mathbb{R}^{D_z}\\
\text{Parameters}: && &&
\boldsymbol{\theta} &\in\mathbb{R}^{D_\theta} \\
\end{aligned} Measurements : Time Stamps : Latent Variables : Parameters : y t Z θ ∈ R N T ∈ R N T ∈ R N T × D z ∈ R D θ y n ∈ R t n ∈ R + z n ∈ R D z Here, we need to use a conditional
p ( y , t , Z , θ ) = p ( θ ) ∏ t = 1 N T p ( y n ∣ z n ) p ( z n ∣ t n , θ ) p(\mathbf{y},\mathbf{t},\mathbf{Z},\boldsymbol{\theta}) =
p(\boldsymbol{\theta})
\prod_{t=1}^{N_T}
p(y_n|\mathbf{z}_n)
p(\mathbf{z}_n|t_n,\boldsymbol{\theta}) p ( y , t , Z , θ ) = p ( θ ) t = 1 ∏ N T p ( y n ∣ z n ) p ( z n ∣ t n , θ ) Unconditional Dynamic Model ¶ D = { t n , y t } n = 1 N y t ∈ R D y t ∈ R + \begin{aligned}
\mathcal{D} &= \left\{ t_n, y_t \right\}_{n=1}^N
&& &&
y_t \in\mathbb{R}^{D_y}
&& &&
t\in\mathbb{R}^+
\end{aligned} D = { t n , y t } n = 1 N y t ∈ R D y t ∈ R + where N = N T N=N_T N = N T
Measurements : y ∈ R N T y t ∈ R Latent Variables : Z ∈ R N T × D z z t ∈ R D z Parameters : θ ∈ R D θ \begin{aligned}
\text{Measurements}: && &&
\mathbf{y} &\in\mathbb{R}^{N_T}
&& &&
y_t \in\mathbb{R}\\
\text{Latent Variables}: && &&
\mathbf{Z} &\in\mathbb{R}^{N_T\times D_z} && &&
\mathbf{z}_t \in\mathbb{R}^{D_z}\\
\text{Parameters}: && &&
\boldsymbol{\theta} &\in\mathbb{R}^{D_\theta} \\
\end{aligned} Measurements : Latent Variables : Parameters : y Z θ ∈ R N T ∈ R N T × D z ∈ R D θ y t ∈ R z t ∈ R D z Finally, we can write the joint distribution
Strong-Constrained : p ( y , Z , θ ) = p ( θ ) p ( z 0 ∣ θ ) ∏ t = 1 T p ( y t ∣ z t ) p ( z t ∣ z 0 ) Weak-Constrained : p ( y , z , θ ) = p ( θ ) p ( z 0 ∣ θ ) ∏ t = 1 T p ( y t ∣ z t ) p ( z t ∣ z t − 1 , θ ) \begin{aligned}
\text{Strong-Constrained}: && &&
p(\mathbf{y},\mathbf{Z},\boldsymbol{\theta})
&=
p(\boldsymbol{\theta})
p(\mathbf{z}_0|\boldsymbol{\theta})
\prod_{t=1}^{T}
p(y_t|\mathbf{z}_t)p(\mathbf{z}_t|\mathbf{z}_0) \\
\text{Weak-Constrained}: && &&
p(\mathbf{y},\mathbf{z},\boldsymbol{\theta})
&=
p(\boldsymbol{\theta})
p(\mathbf{z}_0|\boldsymbol{\theta})
\prod_{t=1}^{T}
p(y_t|\mathbf{z}_t)p(\mathbf{z}_t|\mathbf{z}_{t-1},\boldsymbol{\theta}) \\
\end{aligned} Strong-Constrained : Weak-Constrained : p ( y , Z , θ ) p ( y , z , θ ) = p ( θ ) p ( z 0 ∣ θ ) t = 1 ∏ T p ( y t ∣ z t ) p ( z t ∣ z 0 ) = p ( θ ) p ( z 0 ∣ θ ) t = 1 ∏ T p ( y t ∣ z t ) p ( z t ∣ z t − 1 , θ ) Multivariate Time Series ¶ In this case, we have a univariate time series.
For this, we will investigate how we can model a time series.
Tutorials :
Unconditional Density Estimation ¶ D = { y n } n = 1 N , N = N T y n ∈ R D y \begin{aligned}
\mathcal{D} &= \left\{ \mathbf{y}_n \right\}_{n=1}^N, && &&
N = N_T && &&
\mathbf{y}_n \in\mathbb{R}^{D_y}
\end{aligned} D = { y n } n = 1 N , N = N T y n ∈ R D y We have a few types of models we can use when we are faced with this situation.
Partially-Pooled : p ( Y , Z , θ ) = p ( θ ) ∏ n = 1 N p ( y n ∣ z n ) p ( z n ∣ θ ) \begin{aligned}
\text{Partially-Pooled}: && &&
p(\mathbf{Y},\mathbf{Z},\boldsymbol{\theta})
&=
p(\boldsymbol{\theta})\prod_{n=1}^N
p(\mathbf{y}_n|\mathbf{z}_n)p(\mathbf{z}_n|\boldsymbol{\theta}) \\
\end{aligned} Partially-Pooled : p ( Y , Z , θ ) = p ( θ ) n = 1 ∏ N p ( y n ∣ z n ) p ( z n ∣ θ ) The conclusion of this demonstration is that it’s almost always favourable to use a partially-pooled model as it effectively gives us options for modeling the dynamics of the parameters.
Temporally Conditioned Density Estimation ¶ We can use conditional density estimation but we only condition on the time component.
In this case, we have some pairwise entries of measurements, y n y_n y n t n t_n t n
D = { t n , y n } n = 1 N y n ∈ R D y t n ∈ R + \begin{aligned}
\mathcal{D} &= \left\{ t_n, \mathbf{y}_n \right\}_{n=1}^N
&& &&
\mathbf{y}_n \in\mathbb{R}^{D_y}
&& &&
t_n\in\mathbb{R}^+
\end{aligned} D = { t n , y n } n = 1 N y n ∈ R D y t n ∈ R + where N = N T N=N_T N = N T
Measurements : Y ∈ R N T × D y y n ∈ R Time Stamps : t ∈ R N T t n ∈ R + Latent Variables : Z ∈ R N T × D z z n ∈ R D z Parameters : θ ∈ R D θ \begin{aligned}
\text{Measurements}: && &&
\mathbf{Y} &\in\mathbb{R}^{N_T\times D_y}
&& &&
\mathbf{y}_n \in\mathbb{R}\\
\text{Time Stamps}: && &&
\mathbf{t} &\in\mathbb{R}^{N_T} && &&
t_n \in\mathbb{R}^+\\
\text{Latent Variables}: && &&
\mathbf{Z} &\in\mathbb{R}^{N_T\times D_z} && &&
\mathbf{z}_n \in\mathbb{R}^{D_z}\\
\text{Parameters}: && &&
\boldsymbol{\theta} &\in\mathbb{R}^{D_\theta} \\
\end{aligned} Measurements : Time Stamps : Latent Variables : Parameters : Y t Z θ ∈ R N T × D y ∈ R N T ∈ R N T × D z ∈ R D θ y n ∈ R t n ∈ R + z n ∈ R D z Here, we need to use a conditional
p ( Y , t , Z , θ ) = p ( θ ) ∏ t = 1 N T p ( y n ∣ z n ) p ( z n ∣ t n , θ ) p(\mathbf{Y},\mathbf{t},\mathbf{Z},\boldsymbol{\theta}) =
p(\boldsymbol{\theta})
\prod_{t=1}^{N_T}
p(\mathbf{y}_n|\mathbf{z}_n)
p(\mathbf{z}_n|t_n,\boldsymbol{\theta}) p ( Y , t , Z , θ ) = p ( θ ) t = 1 ∏ N T p ( y n ∣ z n ) p ( z n ∣ t n , θ ) Unconditional Dynamic Model ¶ D = { t n , y t } n = 1 N y t ∈ R D y t ∈ R + \begin{aligned}
\mathcal{D} &= \left\{ t_n, \mathbf{y}_t \right\}_{n=1}^N
&& &&
\mathbf{y}_t \in\mathbb{R}^{D_y}
&& &&
t\in\mathbb{R}^+
\end{aligned} D = { t n , y t } n = 1 N y t ∈ R D y t ∈ R + where N = N T N=N_T N = N T
Measurements : Y ∈ R N T × D y y t ∈ R D y Latent Variables : Z ∈ R N T × D z z t ∈ R D z Parameters : θ ∈ R D θ \begin{aligned}
\text{Measurements}: && &&
\mathbf{Y} &\in\mathbb{R}^{N_T\times D_y}
&& &&
\mathbf{y}_t \in\mathbb{R}^{D_y}\\
\text{Latent Variables}: && &&
\mathbf{Z} &\in\mathbb{R}^{N_T\times D_z} && &&
\mathbf{z}_t \in\mathbb{R}^{D_z}\\
\text{Parameters}: && &&
\boldsymbol{\theta} &\in\mathbb{R}^{D_\theta} \\
\end{aligned} Measurements : Latent Variables : Parameters : Y Z θ ∈ R N T × D y ∈ R N T × D z ∈ R D θ y t ∈ R D y z t ∈ R D z Finally, we can write the joint distribution
Strong-Constrained : p ( Y , Z , θ ) = p ( θ ) p ( z 0 ∣ θ ) ∏ t = 1 T p ( y t ∣ z t ) p ( z t ∣ z 0 ) Weak-Constrained : p ( Y , Z , θ ) = p ( θ ) p ( z 0 ∣ θ ) ∏ t = 1 T p ( y t ∣ z t ) p ( z t ∣ z t − 1 , θ ) \begin{aligned}
\text{Strong-Constrained}: && &&
p(\mathbf{Y},\mathbf{Z},\boldsymbol{\theta})
&=
p(\boldsymbol{\theta})
p(\mathbf{z}_0|\boldsymbol{\theta})
\prod_{t=1}^{T}
p(\mathbf{y}_t|\mathbf{z}_t)p(\mathbf{z}_t|\mathbf{z}_0) \\
\text{Weak-Constrained}: && &&
p(\mathbf{Y},\mathbf{Z},\boldsymbol{\theta})
&=
p(\boldsymbol{\theta})
p(\mathbf{z}_0|\boldsymbol{\theta})
\prod_{t=1}^{T}
p(\mathbf{y}_t|\mathbf{z}_t)p(\mathbf{z}_t|\mathbf{z}_{t-1},\boldsymbol{\theta}) \\
\end{aligned} Strong-Constrained : Weak-Constrained : p ( Y , Z , θ ) p ( Y , Z , θ ) = p ( θ ) p ( z 0 ∣ θ ) t = 1 ∏ T p ( y t ∣ z t ) p ( z t ∣ z 0 ) = p ( θ ) p ( z 0 ∣ θ ) t = 1 ∏ T p ( y t ∣ z t ) p ( z t ∣ z t − 1 , θ ) Multivariate Spatiotemporal Series ¶ In this case, we have a univariate time series.
For this, we will investigate how we can model a time series.
Tutorials :
Unconditional Density Estimation ¶ D = { y n } n = 1 N , N = N T D = D y D Ω y n ∈ R D \begin{aligned}
\mathcal{D} &= \left\{ \mathbf{y}_n \right\}_{n=1}^N, && &&
N = N_T
&& &&
D = D_y D_\Omega
&& &&
\mathbf{y}_n \in\mathbb{R}^{D}
\end{aligned} D = { y n } n = 1 N , N = N T D = D y D Ω y n ∈ R D We have a few types of models we can use when we are faced with this situation.
Partially-Pooled : p ( Y , Z , θ ) = p ( θ ) ∏ n = 1 N p ( y n ∣ z n ) p ( z n ∣ θ ) \begin{aligned}
\text{Partially-Pooled}: && &&
p(\mathbf{Y},\mathbf{Z},\boldsymbol{\theta})
&=
p(\boldsymbol{\theta})\prod_{n=1}^N
p(\mathbf{y}_n|\mathbf{z}_n)p(\mathbf{z}_n|\boldsymbol{\theta}) \\
\end{aligned} Partially-Pooled : p ( Y , Z , θ ) = p ( θ ) n = 1 ∏ N p ( y n ∣ z n ) p ( z n ∣ θ ) The conclusion of this demonstration is that it’s almost always favourable to use a partially-pooled model as it effectively gives us options for modeling the dynamics of the parameters.
Temporally Conditioned Density Estimation ¶ Coordinate-Based ¶ We can use conditional density estimation but we only condition on the time component.
In this case, we have some pairwise entries of measurements, y n y_n y n t n t_n t n
D = { ( t n , s n ) , y n } n = 1 N y n ∈ R D y t n ∈ R + \begin{aligned}
\mathcal{D} &= \left\{ (t_n, \mathbf{s}_n), \mathbf{y}_n \right\}_{n=1}^N
&& &&
\mathbf{y}_n \in\mathbb{R}^{D_y}
&& &&
t_n\in\mathbb{R}^+
\end{aligned} D = { ( t n , s n ) , y n } n = 1 N y n ∈ R D y t n ∈ R + where N = N T N=N_T N = N T D = D y D=D_y D = D y
Measurements : Y ∈ R N T × D y y n ∈ R D y Time Stamps : t ∈ R N T t n ∈ R + Spatial Coordinates : S ∈ R N T × D s s n ∈ R D s Latent Variables : Z ∈ R N T × D z z n ∈ R D z Parameters : θ ∈ R D θ \begin{aligned}
\text{Measurements}: && &&
\mathbf{Y} &\in\mathbb{R}^{N_T\times D_y}
&& &&
\mathbf{y}_n \in\mathbb{R}^{D_y}\\
\text{Time Stamps}: && &&
\mathbf{t} &\in\mathbb{R}^{N_T} && &&
t_n \in\mathbb{R}^+\\
\text{Spatial Coordinates}: && &&
\mathbf{S} &\in\mathbb{R}^{N_T \times D_s} && &&
\mathbf{s}_n \in\mathbb{R}^{D_s}\\
\text{Latent Variables}: && &&
\mathbf{Z} &\in\mathbb{R}^{N_T\times D_z} && &&
\mathbf{z}_n \in\mathbb{R}^{D_z}\\
\text{Parameters}: && &&
\boldsymbol{\theta} &\in\mathbb{R}^{D_\theta} \\
\end{aligned} Measurements : Time Stamps : Spatial Coordinates : Latent Variables : Parameters : Y t S Z θ ∈ R N T × D y ∈ R N T ∈ R N T × D s ∈ R N T × D z ∈ R D θ y n ∈ R D y t n ∈ R + s n ∈ R D s z n ∈ R D z Here, we need to use a conditional
p ( Y , t , S , Z , θ ) = p ( θ ) ∏ t = 1 N T p ( y n ∣ z n ) p ( z n ∣ t n , s n , θ ) p(\mathbf{Y},\mathbf{t},\mathbf{S},\mathbf{Z},\boldsymbol{\theta}) =
p(\boldsymbol{\theta})
\prod_{t=1}^{N_T}
p(\mathbf{y}_n|\mathbf{z}_n)
p(\mathbf{z}_n|t_n,\mathbf{s}_n,\boldsymbol{\theta}) p ( Y , t , S , Z , θ ) = p ( θ ) t = 1 ∏ N T p ( y n ∣ z n ) p ( z n ∣ t n , s n , θ ) Field-Based ¶ We can use conditional density estimation but we only condition on the time component.
In this case, we have some pairwise entries of measurements, y n y_n y n t n t_n t n
D = { t n , y n } n = 1 N y n ∈ R D t n ∈ R + \begin{aligned}
\mathcal{D} &= \left\{ t_n, \mathbf{y}_n \right\}_{n=1}^N
&& &&
\mathbf{y}_n \in\mathbb{R}^{D}
&& &&
t_n\in\mathbb{R}^+
\end{aligned} D = { t n , y n } n = 1 N y n ∈ R D t n ∈ R + where N = N T N=N_T N = N T D = D Ω D y D=D_\Omega D_y D = D Ω D y
Measurements : Y ∈ R N T × D y n ∈ R D Time Stamps : t ∈ R N T t n ∈ R + Latent Variables : Z ∈ R N T × D z z n ∈ R D z Parameters : θ ∈ R D θ \begin{aligned}
\text{Measurements}: && &&
\mathbf{Y} &\in\mathbb{R}^{N_T\times D}
&& &&
\mathbf{y}_n \in\mathbb{R}^D\\
\text{Time Stamps}: && &&
\mathbf{t} &\in\mathbb{R}^{N_T} && &&
t_n \in\mathbb{R}^+\\
\text{Latent Variables}: && &&
\mathbf{Z} &\in\mathbb{R}^{N_T\times D_z} && &&
\mathbf{z}_n \in\mathbb{R}^{D_z}\\
\text{Parameters}: && &&
\boldsymbol{\theta} &\in\mathbb{R}^{D_\theta} \\
\end{aligned} Measurements : Time Stamps : Latent Variables : Parameters : Y t Z θ ∈ R N T × D ∈ R N T ∈ R N T × D z ∈ R D θ y n ∈ R D t n ∈ R + z n ∈ R D z Here, we need to use a conditional
p ( Y , t , Z , θ ) = p ( θ ) ∏ t = 1 N T p ( y n ∣ z n ) p ( z n ∣ t n , θ ) p(\mathbf{Y},\mathbf{t},\mathbf{Z},\boldsymbol{\theta}) =
p(\boldsymbol{\theta})
\prod_{t=1}^{N_T}
p(\mathbf{y}_n|\mathbf{z}_n)
p(\mathbf{z}_n|t_n,\boldsymbol{\theta}) p ( Y , t , Z , θ ) = p ( θ ) t = 1 ∏ N T p ( y n ∣ z n ) p ( z n ∣ t n , θ ) Unconditional Dynamic Model ¶ D = { t n , y t } n = 1 N y t ∈ R D t ∈ R + \begin{aligned}
\mathcal{D} &= \left\{ t_n, \mathbf{y}_t \right\}_{n=1}^N
&& &&
\mathbf{y}_t \in\mathbb{R}^{D}
&& &&
t\in\mathbb{R}^+
\end{aligned} D = { t n , y t } n = 1 N y t ∈ R D t ∈ R + where N = N T N=N_T N = N T D = D Ω D y D=D_\Omega D_y D = D Ω D y
Measurements : Y ∈ R N T × D y t ∈ R D Latent Variables : Z ∈ R N T × D z z t ∈ R D z Parameters : θ ∈ R D θ \begin{aligned}
\text{Measurements}: && &&
\mathbf{Y} &\in\mathbb{R}^{N_T\times D}
&& &&
\mathbf{y}_t \in\mathbb{R}^{D}\\
\text{Latent Variables}: && &&
\mathbf{Z} &\in\mathbb{R}^{N_T\times D_z} && &&
\mathbf{z}_t \in\mathbb{R}^{D_z}\\
\text{Parameters}: && &&
\boldsymbol{\theta} &\in\mathbb{R}^{D_\theta} \\
\end{aligned} Measurements : Latent Variables : Parameters : Y Z θ ∈ R N T × D ∈ R N T × D z ∈ R D θ y t ∈ R D z t ∈ R D z Finally, we can write the joint distribution
Strong-Constrained : p ( Y , Z , θ ) = p ( θ ) p ( z 0 ∣ θ ) ∏ t = 1 T p ( y t ∣ z t ) p ( z t ∣ z 0 ) Weak-Constrained : p ( Y , Z , θ ) = p ( θ ) p ( z 0 ∣ θ ) ∏ t = 1 T p ( y t ∣ z t ) p ( z t ∣ z t − 1 , θ ) \begin{aligned}
\text{Strong-Constrained}: && &&
p(\mathbf{Y},\mathbf{Z},\boldsymbol{\theta})
&=
p(\boldsymbol{\theta})
p(\mathbf{z}_0|\boldsymbol{\theta})
\prod_{t=1}^{T}
p(\mathbf{y}_t|\mathbf{z}_t)p(\mathbf{z}_t|\mathbf{z}_0) \\
\text{Weak-Constrained}: && &&
p(\mathbf{Y},\mathbf{Z},\boldsymbol{\theta})
&=
p(\boldsymbol{\theta})
p(\mathbf{z}_0|\boldsymbol{\theta})
\prod_{t=1}^{T}
p(\mathbf{y}_t|\mathbf{z}_t)p(\mathbf{z}_t|\mathbf{z}_{t-1},\boldsymbol{\theta}) \\
\end{aligned} Strong-Constrained : Weak-Constrained : p ( Y , Z , θ ) p ( Y , Z , θ ) = p ( θ ) p ( z 0 ∣ θ ) t = 1 ∏ T p ( y t ∣ z t ) p ( z t ∣ z 0 ) = p ( θ ) p ( z 0 ∣ θ ) t = 1 ∏ T p ( y t ∣ z t ) p ( z t ∣ z t − 1 , θ ) Coupled Multivariate Spatiotemporal Series ¶ In this case, we have a univariate time series.
For this, we will investigate how we can model a time series.
Tutorials :
Conditional Density Estimation ¶ D = { x n , y n } n = 1 N , N = N T D = D y D Ω y n ∈ R D x n ∈ R D x \begin{aligned}
\mathcal{D} &= \left\{ \mathbf{x}_n, \mathbf{y}_n \right\}_{n=1}^N, && &&
N = N_T
&& &&
D = D_y D_\Omega
&& &&
\mathbf{y}_n \in\mathbb{R}^{D}
&& &&
\mathbf{x}_n \in\mathbb{R}^{D_x}
\end{aligned} D = { x n , y n } n = 1 N , N = N T D = D y D Ω y n ∈ R D x n ∈ R D x We have a few types of models we can use when we are faced with this situation.
Partially-Pooled : p ( Y , X , Z , θ ) = p ( θ ) ∏ n = 1 N p ( y n ∣ z n ) p ( z n ∣ x n , θ ) \begin{aligned}
\text{Partially-Pooled}: && &&
p(\mathbf{Y},\mathbf{X},\mathbf{Z},\boldsymbol{\theta})
&=
p(\boldsymbol{\theta})\prod_{n=1}^N
p(\mathbf{y}_n|\mathbf{z}_n)p(\mathbf{z}_n|\mathbf{x}_n,\boldsymbol{\theta}) \\
\end{aligned} Partially-Pooled : p ( Y , X , Z , θ ) = p ( θ ) n = 1 ∏ N p ( y n ∣ z n ) p ( z n ∣ x n , θ ) The conclusion of this demonstration is that it’s almost always favourable to use a partially-pooled model as it effectively gives us options for modeling the dynamics of the parameters.
Temporal Conditional Density Estimation ¶ Coordinate-Based ¶ We can use conditional density estimation but we only condition on the time component.
In this case, we have some pairwise entries of measurements, y n y_n y n t n t_n t n
D = { ( t n , s n ) , x n , y n } n = 1 N y n ∈ R D y x n ∈ R D x t n ∈ R + \begin{aligned}
\mathcal{D} &= \left\{ (t_n, \mathbf{s}_n), \mathbf{x}_n, \mathbf{y}_n \right\}_{n=1}^N
&& &&
\mathbf{y}_n \in\mathbb{R}^{D_y}
&& &&
\mathbf{x}_n \in\mathbb{R}^{D_x}
&& &&
t_n\in\mathbb{R}^+
\end{aligned} D = { ( t n , s n ) , x n , y n } n = 1 N y n ∈ R D y x n ∈ R D x t n ∈ R + where N = N T N=N_T N = N T D = D y D=D_y D = D y
Measurements : Y ∈ R N T × D y y n ∈ R D y Covariates : X ∈ R N T × D x x n ∈ R D x Time Stamps : t ∈ R N T t n ∈ R + Spatial Coordinates : S ∈ R N T × D s s n ∈ R D s Latent Variables : Z ∈ R N T × D z z n ∈ R D z Parameters : θ ∈ R D θ \begin{aligned}
\text{Measurements}: && &&
\mathbf{Y} &\in\mathbb{R}^{N_T\times D_y}
&& &&
\mathbf{y}_n \in\mathbb{R}^{D_y}\\
\text{Covariates}: && &&
\mathbf{X} &\in\mathbb{R}^{N_T\times D_x}
&& &&
\mathbf{x}_n \in\mathbb{R}^{D_x}\\
\text{Time Stamps}: && &&
\mathbf{t} &\in\mathbb{R}^{N_T} && &&
t_n \in\mathbb{R}^+\\
\text{Spatial Coordinates}: && &&
\mathbf{S} &\in\mathbb{R}^{N_T \times D_s} && &&
\mathbf{s}_n \in\mathbb{R}^{D_s}\\
\text{Latent Variables}: && &&
\mathbf{Z} &\in\mathbb{R}^{N_T\times D_z} && &&
\mathbf{z}_n \in\mathbb{R}^{D_z}\\
\text{Parameters}: && &&
\boldsymbol{\theta} &\in\mathbb{R}^{D_\theta} \\
\end{aligned} Measurements : Covariates : Time Stamps : Spatial Coordinates : Latent Variables : Parameters : Y X t S Z θ ∈ R N T × D y ∈ R N T × D x ∈ R N T ∈ R N T × D s ∈ R N T × D z ∈ R D θ y n ∈ R D y x n ∈ R D x t n ∈ R + s n ∈ R D s z n ∈ R D z Here, we need to use a conditional
p ( Y , X , t , S , Z , θ ) = p ( θ ) ∏ t = 1 N T p ( y n ∣ z n ) p ( z n ∣ t n , s n , x n , θ ) p\left(\mathbf{Y},\mathbf{X},\mathbf{t},\mathbf{S},\mathbf{Z},\boldsymbol{\theta}\right) =
p(\boldsymbol{\theta})
\prod_{t=1}^{N_T}
p(\mathbf{y}_n|\mathbf{z}_n)
p(\mathbf{z}_n|t_n,\mathbf{s}_n,\mathbf{x}_n, \boldsymbol{\theta}) p ( Y , X , t , S , Z , θ ) = p ( θ ) t = 1 ∏ N T p ( y n ∣ z n ) p ( z n ∣ t n , s n , x n , θ ) Field-Based ¶ We can use conditional density estimation but we only condition on the time component.
In this case, we have some pairwise entries of measurements, y n y_n y n t n t_n t n
D = { t n , x n , y n } n = 1 N y n ∈ R D y x n ∈ R D x t n ∈ R + \begin{aligned}
\mathcal{D} &= \left\{ t_n, \mathbf{x}_n, \mathbf{y}_n \right\}_{n=1}^N
&& &&
\mathbf{y}_n \in\mathbb{R}^{D_y}
&& &&
\mathbf{x}_n \in\mathbb{R}^{D_x}
&& &&
t_n\in\mathbb{R}^+
\end{aligned} D = { t n , x n , y n } n = 1 N y n ∈ R D y x n ∈ R D x t n ∈ R + where N = N T N=N_T N = N T D y = D Ω D y D_y=D_\Omega D_y D y = D Ω D y D x = D Ω x D_x=D_{\Omega_x} D x = D Ω x
Measurements : Y ∈ R N T × D y n ∈ R D Covariates : X ∈ R N T × D x x n ∈ R D x Time Stamps : t ∈ R N T t n ∈ R + Latent Variables : Z ∈ R N T × D z z n ∈ R D z Parameters : θ ∈ R D θ \begin{aligned}
\text{Measurements}: && &&
\mathbf{Y} &\in\mathbb{R}^{N_T\times D}
&& &&
\mathbf{y}_n \in\mathbb{R}^D\\
\text{Covariates}: && &&
\mathbf{X} &\in\mathbb{R}^{N_T\times D_x}
&& &&
\mathbf{x}_n \in\mathbb{R}^{D_x}\\
\text{Time Stamps}: && &&
\mathbf{t} &\in\mathbb{R}^{N_T} && &&
t_n \in\mathbb{R}^+\\
\text{Latent Variables}: && &&
\mathbf{Z} &\in\mathbb{R}^{N_T\times D_z} && &&
\mathbf{z}_n \in\mathbb{R}^{D_z}\\
\text{Parameters}: && &&
\boldsymbol{\theta} &\in\mathbb{R}^{D_\theta} \\
\end{aligned} Measurements : Covariates : Time Stamps : Latent Variables : Parameters : Y X t Z θ ∈ R N T × D ∈ R N T × D x ∈ R N T ∈ R N T × D z ∈ R D θ y n ∈ R D x n ∈ R D x t n ∈ R + z n ∈ R D z Here, we need to use a conditional
p ( Y , X , t , Z , θ ) = p ( θ ) ∏ t = 1 N T p ( y n ∣ z n ) p ( z n ∣ t n , x n , θ ) p(\mathbf{Y},\mathbf{X},\mathbf{t},\mathbf{Z},\boldsymbol{\theta}) =
p(\boldsymbol{\theta})
\prod_{t=1}^{N_T}
p(\mathbf{y}_n|\mathbf{z}_n)
p(\mathbf{z}_n|t_n,\mathbf{x}_n,\boldsymbol{\theta}) p ( Y , X , t , Z , θ ) = p ( θ ) t = 1 ∏ N T p ( y n ∣ z n ) p ( z n ∣ t n , x n , θ ) Conditional Dynamic Model ¶ D = { t n , x t , y t } n = 1 N y t ∈ R D y x t ∈ R D x t ∈ R + \begin{aligned}
\mathcal{D} &= \left\{ t_n, \mathbf{x}_t,\mathbf{y}_t \right\}_{n=1}^N
&& &&
\mathbf{y}_t \in\mathbb{R}^{D_y}
&& &&
\mathbf{x}_t \in\mathbb{R}^{D_x}
&& &&
t\in\mathbb{R}^+
\end{aligned} D = { t n , x t , y t } n = 1 N y t ∈ R D y x t ∈ R D x t ∈ R + where N = N T N=N_T N = N T D = D Ω D y D=D_\Omega D_y D = D Ω D y
Measurements : Y ∈ R N T × D y y t ∈ R D y Covariates : X ∈ R N T × D x x t ∈ R D x Latent Variables : Z ∈ R N T × D z z t ∈ R D z Parameters : θ ∈ R D θ \begin{aligned}
\text{Measurements}: && &&
\mathbf{Y} &\in\mathbb{R}^{N_T\times D_y}
&& &&
\mathbf{y}_t \in\mathbb{R}^{D_y}\\
\text{Covariates}: && &&
\mathbf{X} &\in\mathbb{R}^{N_T\times D_x}
&& &&
\mathbf{x}_t \in\mathbb{R}^{D_x}\\
\text{Latent Variables}: && &&
\mathbf{Z} &\in\mathbb{R}^{N_T\times D_z} && &&
\mathbf{z}_t \in\mathbb{R}^{D_z}\\
\text{Parameters}: && &&
\boldsymbol{\theta} &\in\mathbb{R}^{D_\theta} \\
\end{aligned} Measurements : Covariates : Latent Variables : Parameters : Y X Z θ ∈ R N T × D y ∈ R N T × D x ∈ R N T × D z ∈ R D θ y t ∈ R D y x t ∈ R D x z t ∈ R D z Finally, we can write the joint distribution
Strong-Constrained : p ( Y , X , Z , θ ) = p ( θ ) p ( z 0 ∣ θ ) ∏ t = 1 T p ( y t ∣ z t ) p ( z t ∣ x t , z 0 ) Weak-Constrained : p ( Y , X , Z , θ ) = p ( θ ) p ( z 0 ∣ θ ) ∏ t = 1 T p ( y t ∣ z t ) p ( z t ∣ x t , z t − 1 , θ ) \begin{aligned}
\text{Strong-Constrained}: && &&
p(\mathbf{Y},\mathbf{X},\mathbf{Z},\boldsymbol{\theta})
&=
p(\boldsymbol{\theta})
p(\mathbf{z}_0|\boldsymbol{\theta})
\prod_{t=1}^{T}
p(\mathbf{y}_t|\mathbf{z}_t)p(\mathbf{z}_t|\mathbf{x}_t,\mathbf{z}_0) \\
\text{Weak-Constrained}: && &&
p(\mathbf{Y},\mathbf{X},\mathbf{Z},\boldsymbol{\theta})
&=
p(\boldsymbol{\theta})
p(\mathbf{z}_0|\boldsymbol{\theta})
\prod_{t=1}^{T}
p(\mathbf{y}_t|\mathbf{z}_t)p(\mathbf{z}_t|\mathbf{x}_t,\mathbf{z}_{t-1},\boldsymbol{\theta}) \\
\end{aligned} Strong-Constrained : Weak-Constrained : p ( Y , X , Z , θ ) p ( Y , X , Z , θ ) = p ( θ ) p ( z 0 ∣ θ ) t = 1 ∏ T p ( y t ∣ z t ) p ( z t ∣ x t , z 0 ) = p ( θ ) p ( z 0 ∣ θ ) t = 1 ∏ T p ( y t ∣ z t ) p ( z t ∣ x t , z t − 1 , θ )