Appendices

Appendix¶

Database [DVC,GDrive]. We will need a database to store our raw and geoprocessed data. In addition, we will need to store our model parameters and resulting figures.

Representation [Raster, Point Cloud]. We will need to represent our weather stations as point clouds and we may also need to represent them as Rasters. So a blog post will be decided to showcase how we can move between them.

Masks [Country, Land/Ocean]. We need to mask our data

Masked Likelihoods [GPD, TPP].

Sensitivity Analysis [MC, Gauss Approx, Taylor, Unscented, Moment Matching]

Numpyro + PPL I - Model [Prior, Likelihood, Posterior, Prior Predictive Posterior].

Numpyro + PPL II - Guide [MLE, MAP, Laplace, VI, MCMC, HMC].

Missing Data. e.g., Convolutions, Gaussian Processes, Masked-Likelihoods

CRS, Transform, Bounds, Resolution.

Preprocessing¶

Extreme Values [BM, POT, TPP]

Algorithms¶

Gaussian Processes [GP, Kriging, Kernel Methods].

Sparse Gaussian Processes [SGP].

Ensemble Kalman Filter [EnsKF]

Other¶

• Differentiation

  • Symbolic - Blog

  • AutoDifferentiation - Blog

  • Finite Difference --> Convolutions

• Fourier --> Convolutional FFT

• Poisson Solver --> CG, FFT, DST, DCT

• Gaussian Kernel Matrix --> Process Convolutions

• Kernel Matrix --> NN Tapering

• Nearest Neighbours --> Gaussian Distance

• Woodbury, Nystrom, Inducing Points

• PCA --> SVD --> rSVD

• GPs: Moment, Spectral, Markovian

• Plug-N-Play Priors: GP, PCA

• Minimization

• Missing Values: Masks, Fill, Iterative, Numpyro Masks Dist, Zeros RS

• DMD, Convolution, Spectral Convolution

• Finite Elements --> Graphs, Adjacency Matrix, Spatial Weights, Gaussian Distance

Algorithms

• Gaussian Processes

  • Scratch: Numpyro + JAX

  • Custom Library: TinyGP + Lineax + Numpyro

Architectures

• Convolutions

  • Convolutions & Finite Differences

  • More on Convolutions - 1x1, FOV, Separable, DepthWise

  • FFT Convolutions via PseudoSpectral Methods

  • Missing Values

    • Masks

    • Partial Convolutions

• Transformers

  • Attention is All You Need

  • Transformers and Kernels

  • Missing Data + Masked Transformers

• Graphical Models

  • Graphs and Finite Element Methods

  • Missing Data

ROM

• Dimensionality Reduction - What is it and why do we need it, e.g., (SWM, Linear SWM, ROM)

• AutoEncoders

  • Linear - PCA/EOF/POD/SVD

  • Convolutions

  • Spectral Convolutions

  • Transformers (Masked AutoEncoder)

  • Graphs

Multiscale

• Introduction to Multiscale - Power Spectrum Approach

• U-Net I - CNN

• U-Net II - Spectral Convolution

• U-Net III - Transformers

• U-Net IV - Graphs

Objective-Based Approaches

• Implicit Models

  • Fixed Point & Root-Finding

  • Argmin Differentiation

  • Deep Equilibrium Models

• Implementation

  • From Scratch - Blog

  • JaxOpt, Optimistix

• Adjoints

  • Scratch - Blog

  • Optimistix - Docs

  • Diffrax - Docs

Conditional Generative Models

• Latent Variable Models

• Bijective

• Surjective

• Stochastic

• Gradient Flows, Stochastic Interpolants

Engineering Tricks

• Scaling - MinMax, StandardScaler

• TemporalFeatures - Coords, Cyclic, Splines, Fourier Features, Sinusoidals

• SpatialFeatures

  • Coordinate Transforms (Cartesian, Spherical, Cylindrical)

  • Cycle, Splines, Fourier Features

  • Spherical Harmonics

Numpyro Tutorials

• Simple IID Model

• Equinox Integration

• Bayesian Hierarchical Model

• Inference

  • Custom Variational Posterior

• Gaussian Processes

• Sparse Gaussian Processes

• Neural ODE

• Linear State Space Model

• Kalman Filter

• Structured State Space Model

• Deep Markov Model

Keras Tutorial