Skip to article frontmatterSkip to article content
Instrument-2-Instrument

Instrument-2-Instrument

CSIC
UCM
IGEO

Remote Sensing

From a high level, we have two types of satellites: stationary and orbiting.

Stationary satellites have a very good temporal frequency typically around from 15mins. However they have a poor spatial resolution and spatial coverage because they only look at a single point along the Earth at one time. In addition, orbiting satellites are typically further away from the Earths surface (e.g. ~35,000 km) which enables a worse spatial resolution but a good compromise on the spatial coverage.

Orbiting satellites have a good spatial resolution and spatial coverage because they wrap around the entire globe. However, they have a poor temporal resolution and temporal coverage because their revisit time can be quite infrequent compared to a stationary satellite. For example, MODIS has a revisit time of 2 days (source) whereas LANDSAT has a revisit time of 8 days. In addition, orbiting satellites are typically closer to the Earths surface (e.g. 200-1,000 km) which enables a better spatial resolution.

Problem Formulation

We are given datasets within two different domains

Dataset I:u=u(s,t),u:RDs×R+→RDus∈Ωu⊂RDst∈Tu⊂R+Dataset II:a=a(s,t),a:RDs×R+→RDas∈Ωa⊂RDst∈Ta⊂R+\begin{aligned} \text{Dataset I}: && && \boldsymbol{u} &= \boldsymbol{u}(\mathbf{s},t), && && \boldsymbol{u}: \mathbb{R}^{D_s}\times\mathbb{R}^+\rightarrow\mathbb{R}^{D_u} && \mathbf{s}\in\Omega_u\subset\mathbb{R}^{D_s} && t\in\mathcal{T}_u\subset\mathbb{R}^+ \\ \text{Dataset II}: && && \boldsymbol{a} &= \boldsymbol{a}(\mathbf{s},t), && && \boldsymbol{a}: \mathbb{R}^{D_s}\times\mathbb{R}^+\rightarrow\mathbb{R}^{D_a} && \mathbf{s}\in\Omega_a\subset\mathbb{R}^{D_s} && t\in\mathcal{T}_a\subset\mathbb{R}^+ \end{aligned}
(1)

Our objective is to find a transformation that maps dataset I to dataset II.

f:{u:Ωu×Tu}→{a:Ωa×Ta}\boldsymbol{f}: \left\{\boldsymbol{u}:\Omega_u\times\mathcal{T}_u\right\} \rightarrow\left\{\boldsymbol{a}:\Omega_a\times\mathcal{T}_a\right\}
(2)

In general, we have a

Encoder:zu=Encoder(x,θ)Transformation:za=Transformer(zu),s∈RDzuDecoder:a=Decoder(za,θ),\begin{aligned} \text{Encoder}: && && \mathbf{z}_u &= \text{Encoder}(\mathbf{x},\boldsymbol{\theta}) \\ \text{Transformation}: && && \mathbf{z}_a &= \text{Transformer}(\mathbf{z}_u), && && \mathbf{s}\in\mathbb{R}^{D_{z_u}}\\ \text{Decoder}: && && \mathbf{a} &= \text{Decoder}(\mathbf{z}_a,\boldsymbol{\theta}), && &&\\ \end{aligned}
(3)

Problem Approaches

  • Cycle-GAN
  • Inverse Problem w/ Plug-in-Play Prior

Foundational Models

This pipeline is a general pipeline to be able to translate between different satellites. However, we can go further and

Detection

A common subset of methods include detection problems. These are problems where we want to estimate a discrete variable. These can include items like buildings or cars. They can also include more physics-based things like clouds or cars.

Estimation

In all cases, we can derive many variables just with the radiance values.

Temperature.

Sea Surface Temperature.

Colour

Levels of Difficulty

  • Collocated Images
  • Take Intersection of Channels
  • Unpaired Image-2-Image
  • Geostationary vs Orbiting
  • NextGEMS (Model Development) --> Destination Earth (AI Component)
  • IFS -> ECMWF
  • ICON -> Germany
Inference
Inference - TOC
Background
Theory