plumax — Roadmap & Architecture

Mathematical models for plume simulation, methane retrieval, source identification, and emission estimation.

This page is the index for the architecture roadmap. The detail for each tier lives in its own file so they can grow independently as design decisions land. The high-level overview (philosophy, tier table, principles) stays here; each tier page expands the math, module layout, validation strategy, and open questions.

The Data-Driven Modeling Cycle¶

Every tier in plumax follows the same loop:

┌─────────────────────────────────────────────────────────────────┐
│   (1) Simple Model                                              │
│       ↓                                                         │
│   (2) Model-Based Inference                                     │
│       ↓                                                         │
│   (3) Model Emulator          ← skip if model is cheap          │
│       ↓                                                         │
│   (4) Emulator-Based Inference                                  │
│       ↓                                                         │
│   (5) Amortized Inference (Predictor)                           │
│       ↓                                                         │
│   (6) Improve  ───────────────────────────────────────────────┐ │
│       ↑         upgrade model / data / emulator / posterior   │ │
│       └───────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘

• Step 1 gives you a generative story — a known mathematical structure you can simulate from.

• Step 2 gives you ground truth inference — slow but exact, used to validate everything downstream.

• Step 3 makes Step 2 tractable at scale — replace the expensive forward model with a fast surrogate.

• Step 4 is Step 2 again, but now running in seconds instead of hours.

• Step 5 collapses the inference loop entirely — the predictor learns the posterior map directly.

• Step 6 closes the loop — every component is independently upgradable, with the previous step as ground truth.

Tier overview¶

plumax tier table — forward models, complexity, and links to detail pages.

The build order is roughly: Prerequisites → Tier I → RTM stack (parallel) → Tier II → Tier III → Tier IV → Tier V. RTM is independent of transport tier, so it can be developed in parallel by a different person without coordination cost. Tier V depends on at least Tier I being usable end-to-end (per-event posteriors are the input), but does not need Tiers II–IV — it can launch with Tier I posteriors and absorb richer ones later.

Architectural principles¶

Status snapshot (2026-09-15)¶

Module-level status is tracked per tier; every scheduled ☐ / 🚧 row on a tier page links to its GitHub issue, grouped under one epic per tier (#69–#75); the unlinked ☐ rows — Step 3 / 5 / 6 items in Tiers III–IV and the RTM stack, plus Tier III diagnostics and multi-species coupling — are later-phase work deliberately not filed yet. Since the previous snapshot (2026-04-29) the July correctness epics (#19 les_fvm BCs / staggering / CFL, #20 plume / puff, #21 gaussx / finitevolx primitives, #22 retrieval robustness, #23 Lagrangian / assimilation) closed and v0.1.1 shipped. The overall picture: Tiers I–III have Steps 1–2 (forward model + model-based inference) at v1; the RTM stack and Tier IV have Step 1 plus a linear Step 2 (matched filter, closed-form column fusion) with the iterated optimal-estimation retrieval and the nonlinear end-to-end inversion still open; Tier V has its v1 core with the importance-corrected likelihood open; no tier has reached Step 3 (emulator) or Step 5 (amortized predictor); of the Tier 0 data layer only the WRF loader exists.

• Tier 0 — Prerequisites: ✓ MetField container + wrfout loader in plumax.met (per-column η→z regrid, bilinear onto the les_fvm stagger, to_prescribed_wind feeds the Eulerian solver). ☐ ERA5 loader, PBL / Monin–Obukhov diagnostics, coordinate frames, inventory prior and L2 ingest. Every tier’s real-data benchmark is blocked here.

• Tier I — Gaussian: ✓ plume + puff forward models, ✓ MAP / MCMC inversion. ☐ plume rise, column + AK pipeline, likelihoods module, masked-K multi-source, MO dispersion, NPE predictor. Emulator deliberately skipped (model is cheap).

• Tier II — Lagrangian: ✓ Steps 1–2 in plumax.lagrangian: Markov-1 Langevin particles, turbulence (homogeneous + Hanna adapter), forward residence-time concentration, backward footprint on the exact reverse clock, closed-form Gaussian / lognormal inversion with a Matérn-3/2 prior. ☐ gridded wind interpolation, met ensembles, footprint emulator (Steps 3–4).

• Tier III — Eulerian FV: ✓ les_fvm advection / diffusion / dynamics with open-wall + periodic BCs, C-grid wind staggering, CFL guard, and the strong-constraint 4D-Var loop (les_fvm/fourdvar.py) — emission → column-obs forward, whitened (temporal Matérn-3/2) control space, L-BFGS with the exact discrete adjoint via reverse-mode AD, convergence surfaced, Gauss-Newton Laplace posterior (posterior_covariance / laplace_sample via gaussx). v1 inverts a time-resolved scalar emission rate at a known source. ☐ spatial control vector, IC / background / BC-scaling term, incremental Gauss-Newton solve (the assimilation.solve optimisers exist but are not wired), MO + Smagorinsky diffusivity, FV emulator.

• RTM stack: ✓ hapi_lut LUT generator + Beer–Lambert, radtran instrument / SRF / forward, matched_filter detection pipeline, gaussx linear solve. ☐ optimal-estimation retrieval, quality flags + information content, SWIR / TIR surface models, factorised LUT, neural RTM.

• Tier IV — Coupled: ✓ v1 multi-instrument fusion in plumax.coupled: Tier I plume + averaging-kernel forward per instrument (CoupledForward, Instrument), closed-form joint posterior over (Q, bias_inst) (fuse_observations), and the additive RTM observation operator (coupled/rtm.py) mapping column enhancement → ΔVMR → band-integrated radiance. ☐ Tier II / III transport backends, nonlinear end-to-end inversion through the radiance operator, Q(t) OU process, masked-K multi-source, quality-flag aggregator; emulators and the operational predictor follow those.

• Tier V — Population & forecasting: ✓ v1 core in plumax.population: tier-agnostic posterior catalog (event_from_posterior over GaussianPosterior / LognormalPosterior / FusionPosterior), V.A hierarchical lognormal size-distribution fit, V.B Gamma-Poisson rate + log-linear inhomogeneous intensity. ✓ The former standalone methane_pod library is in tree: plumax.population.intensity (13 λ(t) kernels), plumax.population.pod (10 probability-of-detection models), plumax.population.paradox (missing-mass Monte Carlo) and plumax.population.fitting (POD-modified power-law NUTS fit), with its theory pages and notebooks. ☐ Rebuild the generative layer on xtremax point processes (ThinningProcess for POD thinning) and add the heavy-tail mark families. Then: extended per-event payload (samples + prior recall), importance-corrected TMTPP likelihood, mark registry, multi-satellite POD union, catalog ingest, persistency metrics, total-emission estimator, Hawkes / LGCP.

See each tier page for the module-level breakdown and the linked issues.