Skip to article frontmatterSkip to article content

Now, we’re going back to basics.

Assume we have a physical model

∂tu=f(ut−1,t;θ)\partial_t \boldsymbol{u} = \boldsymbol{f}(\boldsymbol{u}_{t-1},t;\boldsymbol{\theta})
(1)

We can solve this using classical PDEs

ut=ODESolve(f,u0,t,θ)\boldsymbol{u}_t = \text{ODESolve}(f, u_0, t, \theta)
(2)

Data

D={yt}t=1Tyt∈RDy\mathcal{D} = \{ y_t \}_{t=1}^T \hspace{10mm} y_t\in\mathbb{R}^{D_y}
(3)

Model

Initial:u0∼N(u0∣ub,Σb)Dynamical Model:ut=ODESolve(f,ut−1,t−1,θ)Observation Model:yt=h(ut,t;θ)+εy\begin{aligned} \text{Initial}: && && \boldsymbol{u}_0 &\sim \mathcal{N}(\boldsymbol{u}_0|\boldsymbol{u}_b, \boldsymbol{\Sigma}_b)\\ \text{Dynamical Model}: && && \boldsymbol{u}_t &= \text{ODESolve}(f, u_{t-1}, t-1, \theta) \\ \text{Observation Model}: && && \boldsymbol{y}_t &= \boldsymbol{h}(\boldsymbol{u}_{t},t;\boldsymbol{\theta}) + \boldsymbol{\varepsilon_y} \\ \end{aligned}
(4)

Criteria

L(θ)=∑t=1T∣∣yt−h(ut,t;θ)∣∣Σy−12+∣∣u0−ub∣∣Σb−12\mathcal{L}(\boldsymbol{\theta}) = \sum_{t=1}^T||y_t - h(u_t, t;\boldsymbol{\theta})||_{\Sigma_y^{-1}}^2 + ||u_0 - \boldsymbol{u}_b||_{\boldsymbol{\Sigma}_b^{-1}}^2
(5)

Here, the parameters are

θ={θ,ub,Σb}\boldsymbol{\theta} = \{ \boldsymbol{\theta}, \boldsymbol{u}_b, \boldsymbol{\Sigma}_b \}
(6)

Latent Dynamical Model

Model

Initial Latent Dist.:z0∼p(z0∣θ)Latent Dynamical Model:zt=f(zt−1,t;θ)+εzObservation Model:yt=h(zt,t;θ)+εy\begin{aligned} \text{Initial Latent Dist.}: && && \boldsymbol{z}_0 &\sim p(\boldsymbol{z}_{0}|\boldsymbol{\theta})\\ \text{Latent Dynamical Model}: && && \boldsymbol{z}_t &= \boldsymbol{f}(\boldsymbol{z}_{t-1},t;\boldsymbol{\theta}) + \boldsymbol{\varepsilon_z} \\ \text{Observation Model}: && && \boldsymbol{y}_t &= \boldsymbol{h}(\boldsymbol{z}_{t},t;\boldsymbol{\theta}) + \boldsymbol{\varepsilon_y} \\ \end{aligned}
(7)

Criteria

L(θ,ϕ;D)=∑t=1TEqϕ(zt−1)[log⁡pθ(zt∣zt−1)+log⁡pθ(yt∣zt)−log⁡qϕ(zt∣zt−1)]\mathcal{L}(\boldsymbol{\theta}, \boldsymbol{\phi};\mathcal{D}) = \sum_{t=1}^T\mathbb{E}_{q_\phi(\boldsymbol{z}_{t-1})}\left[\log p_\theta(\boldsymbol{z}_t|\boldsymbol{z}_{t-1}) + \log p_\theta(y_t|z_t) - \log q_\phi(z_t|z_{t-1}) \right]
(8)

where

Prior:p(zt∣zt−1;θ)=N(zt∣fθe(zt−1);)\begin{aligned} \text{Prior}: && && p(\boldsymbol{z}_t|\boldsymbol{z}_{t-1};\boldsymbol{\theta}) &= \mathcal{N}(\boldsymbol{z}_t|\boldsymbol{f_{\theta_e}}(\boldsymbol{z}_{t-1});) \end{aligned}
(9)
DA with Obs
Deep Equilibrium Models
DA with Obs
Cascading Approaches