Modern 4DVar

Space-Time Decomposition

CNRS

MEOM

PDE Formulation/Inspiration

Dynamical System¶

Spatial Operators¶

Local & Global, Gradient & Integral, Numerical & ML

Whirlwind Tour¶

Criteria¶

Uniform Grid. This determines if the grid structure is uniform or not.

Mesh Invariant. This determines if one can use arbitrary resolutions or not.

Fixed Parameters. The parameters of the transformation are fixed or not, e.g. convolutional kernel, kernel function hyper-parameters, neural network function parameters.

Local vs Global. This says if the transformation is a local transformation, e.g. gradient, or a global transformation, e.g. integral.

Fixed Structure Transformation. This is whether the transformation can be applied to any new arbitrary grid structure. This is especially useful for methods which

Table 1:Hybrid Spatiotemporal Operators

Class	Name	Uniform Grid	Mesh Invariant	Fixed Params	Local / Global	Fixed
Coordinate-Based	Kernels Methods	No	Yes	No	Both*	No
Coordinate-Based	Neural Fields	No	Yes	No	Both*	No
Numerical Operators	Finite Difference	No	No	No	Local	Yes
Numerical Operators	Finite Volume	No	No	No	Local	Yes
Numerical Operators	Finite Element	No	No	No	Local	Yes
Numerical Operators	Spectral Methods	No	No	No	Global	Yes
Neural Operators	Deep ONets [Lu et al., 2021]	No	Yes	No	Global	Yes
Neural Operators	FNO [Kovachki et al., 2021]	Yes	Yes	No	Global	Yes

Time Steppers¶

Integrals, Numerical & ML

Fundamental Theorem of Calculus

u(x, t) = u_0 + \int_0^t F(u, u_0, \tau,)d\tau

(1)

Auto-Regressive

u(x, \Delta t) = u_0 + \int_0^{\Delta t} F(u, u_0, \Delta t,)d\tau

(2)

Criteria

Integral Approx - MC, IS, Quadrature, Taylor Expansion
Implicit vs Explicit
Fixed vs Variable Params
Local (AR) vs Global (FC)
Structured vs Unstructured - weird time steps

Hybrid¶

\partial_t u = \alpha \boldsymbol{F}_\text{dyn}(u, t;\boldsymbol{\theta}) + (1 - \alpha) \boldsymbol{F}_\text{param}(u, t;\boldsymbol{\theta})

(3)

Dynamical Model - In this example, $\alpha=1$ and we assume that the system dynamics are known we can write our problem as a PDE.

Surrogate Model - In this case, $\alpha=0$ and we assume that the system dynamics are unknown and we cannot formulate our problem as a PDE.

Hybrid Model - In this case, $0 < \alpha < 1$ ad we assume that the system dynamics are partially-known and we can formulate portions of our problem (spatially, temporally, or both) as a PDE and the other portion as a parameterized function.

Table 1:Hybrid Spatiotemporal Operators

Name	Spatial	Temporal	Example
Numerical PDE	Finite Derivatives (Diff,Vol,Elem)	Time Stepper	QG + RK4 Scheme
Numerical PDE	Spectral Derivatives	Time Stepper	QG + RK4 Scheme
Surrogate	Spatial NN Operator	Temporal NN Operator	Neural Flows [Biloš et al., 2021], PDE-Refiner [Lippe et al., 2023], Message Passing PDE [Brandstetter et al., 2022]
Surrogate	Spatiotemporal NN Operator	" "	NerFs, ConvLSTM [Shi et al., 2015], FNO [Kovachki et al., 2021], CORAL [Serrano et al., 2023]
Hybrid	Spatial NN Operator	TimeStepper	Neural ODE [Kidger, 2022]
Hybrid	Finite Derivatives	Neural Network	PDE Solver [Brandstetter et al., 2022]
Hybrid	Finite Der. + NN	TimeStepper	Universal Differential Equations [Rackauckas et al., 2020]
Hybrid	Finite Der. + NN	TimeStepper + NN	FouRKS [Chattopadhyay & Hassanzadeh, 2023]

References

Lu, L., Jin, P., Pang, G., Zhang, Z., & Karniadakis, G. E. (2021). Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nature Machine Intelligence, 3(3), 218–229. 10.1038/s42256-021-00302-5
Kovachki, N., Li, Z., Liu, B., Azizzadenesheli, K., Bhattacharya, K., Stuart, A., & Anandkumar, A. (2021). Neural Operator: Learning Maps Between Function Spaces. arXiv. 10.48550/ARXIV.2108.08481
Biloš, M., Sommer, J., Rangapuram, S. S., Januschowski, T., & Günnemann, S. (2021). Neural Flows: Efficient Alternative to Neural ODEs. arXiv. 10.48550/ARXIV.2110.13040
Lippe, P., Veeling, B. S., Perdikaris, P., Turner, R. E., & Brandstetter, J. (2023). PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers. arXiv. 10.48550/ARXIV.2308.05732
Brandstetter, J., Worrall, D., & Welling, M. (2022). Message Passing Neural PDE Solvers. arXiv. 10.48550/ARXIV.2202.03376

Model Representations

Operators

Model Representations

Data Assimilation