Application Track ¶ Overview ¶ Extreme Values Modeling
The first application track we have are looking at extreme values.
This allows us to slowly introduce some core concepts that will be useful in the later application tracks.
D s = [ Altitude, Longitude, Latitude ] Ω = [ Unstructured Weather Stations ] Δ t = { Daily, Hourly } D y = [ Temperature, Precipitation, Wind Speed ] \begin{aligned}
D_s &= \left[ \text{Altitude, Longitude, Latitude}\right] \\
\Omega &= \left[ \text{Unstructured Weather Stations}\right] \\
\Delta t &= \left\{ \text{Daily, Hourly} \right\} \\
D_y &= \left[ \text{Temperature, Precipitation, Wind Speed}\right]
\end{aligned} D s Ω Δ t D y = [ Altitude, Longitude, Latitude ] = [ Unstructured Weather Stations ] = { Daily, Hourly } = [ Temperature, Precipitation, Wind Speed ] Sea Surface Mapping
The second application track is working with sea surface mapping of variables like temperature, height, and ocean colour.
In this setting, we will deal with how to transform between different geometries like the alongtrack satellite observations and the regular gridded applications.
These measurements feature a lot of missing values, so we will learn how to deal with this.
The bulk of this application will involve learning-to-learn, whereby we find various ways to learn prior models
D s = [ Depth, Longitude, Latitude ] Ω = [ Irregular Along Track, Regular Gridded ] Δ t = { Daily, Hourly } D y = [ Temperature, Height, Salinity, Colour ] \begin{aligned}
D_s &= \left[ \text{Depth, Longitude, Latitude}\right] \\
\Omega &= \left[ \text{Irregular Along Track, Regular Gridded}\right] \\
\Delta t &= \left\{ \text{Daily, Hourly} \right\} \\
D_y &= \left[ \text{Temperature, Height, Salinity, Colour}\right]
\end{aligned} D s Ω Δ t D y = [ Depth, Longitude, Latitude ] = [ Irregular Along Track, Regular Gridded ] = { Daily, Hourly } = [ Temperature, Height, Salinity, Colour ] Instrument-2-Instrument Translation
In this third application track, we will investigate how we can apply a method to translate one type of satellite to another type of satellite.
Here, we have to do all tasks including interpolation, X-casting, and variable transformation.
The interpolation component is due to the non-consistent geometry amongst the satellites.
The X-casting component comes from the
We also have to deal with a
D s = [ Height, Longitude, Latitude ] Ω = [ Irregular Polar-Orbiting Satellite, Regular Geostationary Satellite ] Δ t = { Days, 15 Minutes } D y = [ Spectral Signature ] \begin{aligned}
D_s &= \left[ \text{Height, Longitude, Latitude}\right] \\
\Omega &= \left[ \text{Irregular Polar-Orbiting Satellite, Regular Geostationary Satellite}\right] \\
\Delta t &= \left\{ \text{Days, 15 Minutes} \right\} \\
D_y &= \left[ \text{Spectral Signature}\right]
\end{aligned} D s Ω Δ t D y = [ Height, Longitude, Latitude ] = [ Irregular Polar-Orbiting Satellite, Regular Geostationary Satellite ] = { Days, 15 Minutes } = [ Spectral Signature ] Break Down ¶ Here, we have a breakdown of how we will structure the application section.
We will analyse the data which will be used to learn.
Then, we will do whatever interpolation or discretization necessary to transform it to the form we want.
Then, we will do some parameter estimation to learn a representation of the system.
Lastly, we will do state estimation using whichever form is
Data Acquisition - GPs, Simulations, L1 Data, L2 Data, L3 Data
Interpolation & Discretization - Histograms, Nearest Neighbors, Gaussian Processes
Learning Priors - Gaussian Processes, L2 Data, L3 Data, Simulations
Parameter Estimation - Representation Learning, Dynamical Systems
State Estimation - 3DVar, Strong-Constrained 4D Var, Weak-Constrained 4DVar, Deep Equillibrium
Walk-Through ¶ For each of these sections
Download EO Data ¶ Examples: Sea Surface Height/Temperature/Salinity, Satellite Data, Weather Stations
LS Data : [ GP Priors , Simulations ] L1 Data : [ Weather Stations , ARGO Floats , Satellite ] L2 Data : [ Interpolated Data ] L3 Data : [ Reanalysis ] L4 Data : [ Reanalysis ] \begin{aligned}
\text{LS Data}: && &&
&\left[\text{GP Priors}, \text{Simulations} \right] \\
\text{L1 Data}: && &&
&\left[\text{Weather Stations}, \text{ARGO Floats}, \text{Satellite}\right] \\
\text{L2 Data}: && &&
&\left[\text{Interpolated Data} \right] \\
\text{L3 Data}: && &&
&\left[\text{Reanalysis} \right] \\
\text{L4 Data}: && &&
&\left[\text{Reanalysis} \right] \\
\end{aligned} LS Data : L1 Data : L2 Data : L3 Data : L4 Data : [ GP Priors , Simulations ] [ Weather Stations , ARGO Floats , Satellite ] [ Interpolated Data ] [ Reanalysis ] [ Reanalysis ] Discretizations ¶ Examples: Histogram, Nearest Neighbours, Radius Neighbours, Kernel Density Estimation
p ( y 1 : T , θ ) = ∏ n = 1 N T p ( y t ∣ θ ) p ( θ ∣ s n , t n ) p(y_{1:T},\boldsymbol{\theta}) = \prod_{n=1}^{N_T}p(y_t|\theta)p(\theta|s_n, t_n) p ( y 1 : T , θ ) = n = 1 ∏ N T p ( y t ∣ θ ) p ( θ ∣ s n , t n ) Functional Regression ¶ Examples: Nearest Neighbors, Gaussian Processes, Neural Fields
p ( y 1 : N , s 1 : N , t 1 : N , θ ) = p ( θ ) ∏ n = 1 N T p ( y n ∣ s n , t n , θ ) p(y_{1:N},s_{1:N},t_{1:N},\boldsymbol{\theta}) = p(\theta)\prod_{n=1}^{N_T}p(y_n|s_n, t_n,\theta) p ( y 1 : N , s 1 : N , t 1 : N , θ ) = p ( θ ) n = 1 ∏ N T p ( y n ∣ s n , t n , θ ) Param. Est. - GP Priors ¶ Examples: RBF, Matern
Joint : p ( f , α , z , θ ) = p ( f ∣ z ) p ( z ∣ α , θ ) p ( θ ) Posterior : q ( α , z , f , θ ) = q ( α ) ∏ n = 1 N q ( z ∣ f , α ) \begin{aligned}
\text{Joint}: && &&
p(f, \alpha, z, \theta) &= p(f|z)p(z|\alpha,\theta)p(\theta) \\
\text{Posterior}: && &&
q(\alpha, z, f, \theta)
&=
q(\alpha) \prod_{n=1}^N
q(z|f,\alpha) \\
\end{aligned} Joint : Posterior : p ( f , α , z , θ ) q ( α , z , f , θ ) = p ( f ∣ z ) p ( z ∣ α , θ ) p ( θ ) = q ( α ) n = 1 ∏ N q ( z ∣ f , α ) State Estimation
q ( y 1 : T , z 1 : T ∗ , α ∗ , θ ) = q ( θ ∗ ) q ( ) q(y_{1:T},z_{1:T}^*,\alpha^*,\theta) = q(\theta^*)q() q ( y 1 : T , z 1 : T ∗ , α ∗ , θ ) = q ( θ ∗ ) q ( ) Param. Est. - Simulations ¶ Param. Est. - Spatial Field Priors ¶ Joint : p ( y , z , θ ) = p ( θ ) ∏ n = 1 N p ( y n ∣ z n ) p ( z n ∣ θ ) Posterior : q ( z , y , θ ) = q ( θ ) ∏ n = 1 N q ( z ∣ y , θ ) \begin{aligned}
\text{Joint}: && &&
p(y,z,\theta) &= p(\theta)\prod_{n=1}^Np(y_n|z_n)p(z_n|\theta) \\
\text{Posterior}: && &&
q(z,y,\theta) &= q(\theta)\prod_{n=1}^N
q(z|y,\theta)
\end{aligned} Joint : Posterior : p ( y , z , θ ) q ( z , y , θ ) = p ( θ ) n = 1 ∏ N p ( y n ∣ z n ) p ( z n ∣ θ ) = q ( θ ) n = 1 ∏ N q ( z ∣ y , θ ) State Estimation ¶ q ( z , y , θ ) = q ( z ∣ θ , ϕ ∗ ) q ( θ ∣ ϕ ∗ ) q(z,y,\theta) = q(z|\theta,\phi^*)q(\theta|\phi^*) q ( z , y , θ ) = q ( z ∣ θ , ϕ ∗ ) q ( θ ∣ ϕ ∗ ) Equilibrium Models ¶ z ∗ = f ( z ∗ , y , x , θ ) z^* = f(z^*, y, x, \theta) z ∗ = f ( z ∗ , y , x , θ ) Param. Est. - Strong-Constrained Dynamical Priors ¶ Joint : p ( y 1 : T , z 0 : T , θ ) = p ( θ ) p ( z 0 ) ∏ n = 1 N p ( y t ∣ z t , θ ) Variational : q ( z 0 : T , y 1 : T , θ ) = q ( θ ) q ( z 0 ) ∏ n = 1 N q ( z t ∣ y t , θ ) \begin{aligned}
\text{Joint}: && &&
p(y_{1:T},z_{0:T},\theta)
&=
p(\theta)p(z_0)
\prod_{n=1}^N
p(y_t|z_t,\theta) \\
\text{Variational}: && &&
q(z_{0:T},y_{1:T},\theta) &=
q(\theta) q(z_0)
\prod_{n=1}^N
q(z_t|y_t,\theta)
\end{aligned} Joint : Variational : p ( y 1 : T , z 0 : T , θ ) q ( z 0 : T , y 1 : T , θ ) = p ( θ ) p ( z 0 ) n = 1 ∏ N p ( y t ∣ z t , θ ) = q ( θ ) q ( z 0 ) n = 1 ∏ N q ( z t ∣ y t , θ ) State Estimation ¶ Param. Est. - Weak-Constrained Dynamical Prior ¶ Joint : p ( y 1 : T , z 0 : T , θ ) = p ( θ ) p ( z 0 ) ∏ n = 1 N p ( y t ∣ z t ) p ( z t ∣ z t − 1 , θ ) Variational : q ( z 0 : T , y 1 : T , θ ) = q ( θ ) q ( z 0 ) ∏ n = 1 N q ( z t ∣ z t − 1 , y t ) q ( z t ∣ z t − 1 , θ ) \begin{aligned}
\text{Joint}: && &&
p(y_{1:T},z_{0:T},\theta)
&=
p(\theta)p(z_0)
\prod_{n=1}^N
p(y_t|z_t)p(z_t|z_{t-1},\theta) \\
\text{Variational}: && &&
q(z_{0:T},y_{1:T},\theta) &=
q(\theta) q(z_0)
\prod_{n=1}^N
q(z_t|z_{t-1},y_t)
q(z_t|z_{t-1},\theta)
\end{aligned} Joint : Variational : p ( y 1 : T , z 0 : T , θ ) q ( z 0 : T , y 1 : T , θ ) = p ( θ ) p ( z 0 ) n = 1 ∏ N p ( y t ∣ z t ) p ( z t ∣ z t − 1 , θ ) = q ( θ ) q ( z 0 ) n = 1 ∏ N q ( z t ∣ z t − 1 , y t ) q ( z t ∣ z t − 1 , θ ) State Estimation ¶ q ( y 1 : T , z 1 : T ∗ , θ ∗ ) = q ( θ ∗ ) q ( z 0 ∗ ) ∏ t = 1 T q ( z t ∣ z t − 1 , y t ) q(y_{1:T},z_{1:T}^*,\theta^*) = q(\theta^*)q(z_0^*)
\prod_{t=1}^Tq(z_t|z_{t-1},y_t) q ( y 1 : T , z 1 : T ∗ , θ ∗ ) = q ( θ ∗ ) q ( z 0 ∗ ) t = 1 ∏ T q ( z t ∣ z t − 1 , y t ) We download some
Applications ¶ Extreme Values .
Here, we learn how we can estimate the parameters of a model for extreme values.
Ocean Surface Height .
We will learn how to create mapping products for Sea Surface Height (SSH) and Sea Surface Temperature (SST).
Instrument-2-Instrument .
We will learn how to map one set of spatio-spectral channels to a different spatio-spectral channels.
Application I - Extremes ¶ In this application, we are interested in modeling temperatures, and in particular, temperature extremes.
Datasets .
We will use some standard datasets that are found within the AEMET database.
D = { ( t n , s n ) , x n , y n } n = 1 N , y n ∈ R D y x n ∈ R D y s n ∈ R D s t n ∈ R + \begin{aligned}
\mathcal{D} &= \left\{(t_n, \mathbf{s}_n), \mathbf{x}_n, \mathbf{y}_n \right\}_{n=1}^{N},
&& &&
\mathbf{y}_n\in\mathbb{R}^{D_y} &&
\mathbf{x}_n\in\mathbb{R}^{D_y} &&
\mathbf{s}_n\in\mathbb{R}^{D_s} &&
t_n\in\mathbb{R}^{+}
\end{aligned} D = { ( t n , s n ) , x n , y n } n = 1 N , y n ∈ R D y x n ∈ R D y s n ∈ R D s t n ∈ R + Covariante, x n x_n x n
Measurement, y n y_n y n
Extreme Events .
D = { y n } n = 1 N \mathcal{D} = \left\{y_n\right\}_{n=1}^N D = { y n } n = 1 N We will use some standard techniques to extract the extreme events from the time series.
Data Likelihood .
y ∼ p ( y ∣ z ) y \sim p(y|z) y ∼ p ( y ∣ z ) We need to decide which likelihood we want to use.
For the mean data, we can assume that it follows a Gaussian distribution or a long-tailed T-Student distribution.
However, for the maximum values, we need to assume an EVD like the GEVD , GPD or TPP .
Mean -> Gaussian, T-Student
Maximum -> GEVD , GPD , TPP
Parameterizations .
p ( y , z , θ ) = p ( y ∣ z ) p ( z ∣ θ ) p ( θ ) p(\mathbf{y},\mathbf{z},\boldsymbol{\theta}) = p(y|z)p(z|\theta)p(\theta) p ( y , z , θ ) = p ( y ∣ z ) p ( z ∣ θ ) p ( θ ) We have to decide the parameterization we wish to use.
IID
Hierarchical
Strong-Constrained Dynamical, i.e., Neural ODE/PDE
Weak-Constrained Dynamical, i.e., SSM, EnsKF
State Estimation
z ∗ ( θ ) = arg min J ( z ; θ ) z^*(\theta) = \argmin \hspace{2mm}\boldsymbol{J}(z;\theta) z ∗ ( θ ) = arg min J ( z ; θ ) Predictions
u ∼ p ( u ∣ z ∗ ) p ( z ∗ ∣ D ) u \sim p(u|z^*)p(z^*|\mathcal{D}) u ∼ p ( u ∣ z ∗ ) p ( z ∗ ∣ D ) Our final task is to use a our new model to make predictions.
Feature Representation
Interpolation
HindCasting
Forecasting
Application II - Sea Surface Height Mapping ¶ In this application, we will investigate different ways that we can create maps of sea surface height (SSH).