Tier V — Source population & forecasting

Forward model: thinned marked temporal point process (TMTPP) over emission events, with per-event marks drawn from Tier I–IV posteriors and per-event detection thinning by per-satellite POD models.

This tier sits above the per-event physics tiers. Tiers I–IV answer “what’s the emission rate from this plume right now?” — single overpass, single source. Tier V answers a different family of questions:

Inventory accounting: “Given a population of detected plumes (and the ones we missed), what’s the true total emitted mass?”
Forecasting: “For a given facility class, when will the next emission event happen, and how big will it be?”
Bias diagnosis: “How biased is the per-overpass average rate when satellites only see the big leaks?”

Inventory and forecasting are co-equal products of Tier V — not just totals. The inverted intensity $\lambda(t)$ directly powers operational forecasts (dispatch windows, occurrence probabilities); see Persistency.

Sub-pages:

Instantaneous emission estimation — single-overpass $Q$ ; the cross-tier interface that turns per-event posteriors into mark likelihoods.
Point process model (TMTPP) — the generative foundation: temporal intensity $\lambda(t)$ , marks $f(Q)$ , and detection thinning $P_d(\cdot)$ .
Persistency — operational forecasts from inverted $\lambda(t)$ : wait times, dispatch windows, occurrence probabilities.
Total emission estimation — the missing-mass paradox and POD-corrected regional/national totals.

TMTPP foundations — the three-term log-likelihood¶

The full population log-likelihood has three terms (derived in 06b; foundations in [daley2003,daleyVereJones2008]):

\log L \;=\; \underbrace{\sum_{i \in \mathcal{D}} \log p(\text{detected}_i \mid f, \lambda, P_d)}_{\text{mark contribution}} \;+\; \underbrace{\sum_{i \in \mathcal{D}} \log \lambda(t_i)}_{\text{detection-time intensity}} \;-\; \underbrace{\int_{0}^{T} \lambda(t) \left[ \int P_d(Q)\, f(Q)\, \mathrm{d}Q \right] \mathrm{d}t}_{\text{integrated thinned rate}}

(1)

The third term is what makes $\lambda$ and $P_d$ jointly identifiable — without it the two trade off.

Mark contribution and the soft-observation framing¶

The per-event posterior from Tiers I–IV is a soft observation of the (unknown) true mark $Q_i$ . This is the same Bayesian-deconvolution / errors-in-variables structure used in measurement-error regression. The per-event mark contribution is:

p(\text{detected}_i \mid f, \lambda, P_d) \;=\; \int P_d(Q)\, L_i(Q)\, f(Q)\, \mathrm{d}Q

(2)

where $L_i(Q) = p(\text{observation}_i \mid Q)$ is the per-event likelihood, not the posterior. In sample-based practice (per-event posterior samples $Q_i^{(s)} \sim p(Q \mid \text{observation}_i)$ ):

p(\text{detected}_i \mid f, \lambda, P_d) \;\approx\; \frac{1}{S}\, \sum_{s=1}^{S}\, P_d(Q_i^{(s)})\, \frac{f(Q_i^{(s)})}{\pi_\text{per-event}(Q_i^{(s)})}

(3)

with $\pi_\text{per-event}(Q)$ the per-event prior used at Tier I–IV. The ratio $f / \pi_\text{per-event}$ is the importance weight that re-points the per-event posterior at the population mark distribution.

This is the central math of cross-tier inference. Currently the prototype in plumax.population.fitting summarises per-event posteriors to point estimates before the population fit, side-stepping the importance correction. Formalising this is the v1 deliverable for 06_tier5a_instantaneous.md.

How the cycle adapts at population scale¶

The six-step cycle still applies, but the objects change:

Six-step cycle adaptation: Tier I–IV (per event) vs. Tier V (population).

Step	Tier I–IV (per event)	Tier V (population)
1 — Simple model	Forward physics (plume / PDE / RTM)	Generative TMTPP: $\lambda(t)$ + mark $f(Q)$ + POD $P_d(\cdot)$
2 — Model-based inference	MAP / MCMC over source params	NumPyro NUTS over $(\lambda \text{ params}, \text{mark params}, \text{POD params})$ . Cheap at $O(10^{4})$ events (minutes); hours-to-days at $O(10^{6})$ events (national catalog)
3 — Model emulator	FNO / neural ODE on the PDE	Skip when NUTS fits in budget. Optionally a normalising flow over the population posterior for repeated re-fits or sensitivity studies
4 — Emulator-based inference	PDE-free 4D-Var	Variational fit (`numpyro.infer.SVI`) or flow-based posterior approximation; required at national catalog scale
5 — Amortized predictor	Per-overpass $Q$ predictor	$(\text{basin tile}, \text{history window}) \to$ posterior over $(\lambda, f(Q), \text{total mass}, \text{next-event time})$ conditioned on per-event evidence and met-region context
6 — Improve	Better physics	Spatial point process (links to Tier III); multi-satellite fusion; varying-coefficient POD (per-(basin, season, scene class) hierarchy); non-Poisson clustering (Hawkes / Cox)

Tile definition: an H3 hex-resolution-7 cell (~5 km²) for sub-basin work, or a basin polygon for inventory accounting. History window: 30–365 days, hierarchical prior on the cutoff.

Varying-coefficient POD: $P_d$ parameters indexed by $(\text{basin}, \text{season}, \text{scene class})$ with hierarchical shrinkage to the global POD. Captures regional / seasonal detection differences without inflating parameter count.

Cross-tier interface — the load-bearing contract¶

Payload schema¶

Every per-event posterior consumed by Tier V must carry:

Per-event posterior payload — fields, types, notes.

Field	Type	Notes
`posterior_samples`	`(S,)` array of $Q$ draws	OR `posterior_summary` for Gaussian shorthand
`posterior_summary`	$(\mu_{\log Q}, \sigma_{\log Q})$	lognormal quick form when full samples are too heavy
`per_event_prior_logpdf`	callable $Q \to \log \pi(Q)$	required for the importance correction; without it the population fit is biased
`instrument_id`	str	dispatch into per-instrument POD
`t_detection`	float (UTC seconds)	for $\lambda(t)$
`x0_posterior`	$(\boldsymbol{\mu}_{xy}, \boldsymbol{\Sigma}_{xy})$	for spatial Cox-process upgrade
`quality`	dict	confidence flags from the Tier I–IV quality bitmask

Independence assumption — the v1 caveat¶

The factorised likelihood above assumes detections at different overpasses are independent. Two overpasses of the same physical leak (e.g. GHGSat then TROPOMI two days later) violate this.

Validation strategy¶

Population SBC. Generate $(\lambda^{*}, f^{*}, P_d^{*})$ , simulate the full thinned-and-marked catalog with a synthetic per-event posterior layer, fit, check rank statistics across all hyperparameters. The Tier V analogue of Tier-I synthetic recovery.
Importance-weight ESS diagnostic. Per detection $i$ , the IS estimator’s effective sample size $\operatorname{ESS}_i = (\sum_s w_s)^{2} / \sum_s w_s^{2}$ is a health metric. Low ESS (e.g. $< S/10$ ) signals that the per-event posterior is far from the population mark distribution; the population fit is unreliable for that event. Report the ESS distribution as a fit diagnostic.
Per-event-prior swap-out. Refit the population using a different per-event prior at Tier I–IV (re-run Tiers I–IV with $\pi_\text{per-event} = \operatorname{LogNormal}(0, 2)$ instead of the inventory-anchored prior). The population posterior on $f$ should not move beyond IS noise. If it does, the importance correction is mis-implemented.
Real-data benchmark. Compare corrected total emission for a well-studied basin (Permian) to published bottom-up inventories ([epa_ghgi,scarpelli2020sectoral], GHGRP) and top-down inverse-modelling estimates ([maasakkers2023ghgi,jacob2022quantifying], Sherwin et al.) — see 06d.

Module layout¶

The formerly standalone methane_pod library (intensity and POD registries, missing-mass simulator, NUTS fitter) now lives in tree under plumax.population. Generic point-process machinery — compensators, thinning samplers, Hawkes, goodness-of-fit — is meant to come from xtremax.point_processes rather than be re-implemented here (#106); only the methane-domain primitives stay in plumax.

Tier V module layout — concern, target module, status.

Concern	Module	Status
Intensity registry $\lambda(t)$	`plumax.population.intensity` (13 kernels)	✓ in tree; `xtremax` generative layer ☐ #106
POD registry $P_d(\cdot)$	`plumax.population.pod` (10 models)	✓ in tree
Missing-mass MC simulator	`plumax.population.paradox`	✓ in tree
NUTS fitter	`plumax.population.fitting` (POD-modified power law)	✓ in tree; importance-correction integration ☐ #108
Per-event posterior summariser	`plumax.population.catalog` (`PerEventPosterior.samples`)	☐ #107
Per-event prior recall ( $\pi_\text{per-event}$ lookup)	`plumax.population.catalog` (`PerEventPosterior.prior_logpdf`)	☐ #107 — required for importance weighting
Importance-weight calculator	`plumax.population.importance`	☐ #108
Multi-satellite POD union	`plumax.population.pod.pod_union`	☐ #110
Catalog schema (in-memory cross-tier)	`plumax.population.catalog` (`EmissionEvent` / `EmissionCatalog`)	🚧 — in-memory Gaussian-summary catalog landed; CSV / parquet ingest #111
Real-data CSV ingestion	`plumax.population.ingest`	☐ #111
Population SBC harness	`plumax.population.validation`	☐ #114
Importance-weight ESS diagnostic	`plumax.population.importance.ess_report`	☐ #108
Per-event-prior swap-out test	`plumax.population.validation.prior_swap_test`	☐ #114
Spatial Cox-process extension (v2)	`plumax.population.spatial`	☐ v2 (temporal LGCP first: #115)

The plumax.population subpackage holds the v1 core (catalog, size_distribution, point_process) and the methane-domain primitives ported from the former methane_pod library (intensity, pod, paradox, fitting). The port is self-contained; the remaining part of #106 is to rebuild the generative machinery on a git-pinned dependency on xtremax.point_processes (which already covers intensities, compensators, thinning samplers, Hawkes, marked processes, a ThinningProcess whose retention function is exactly $P_d(Q)$ , goodness-of-fit and survival / wait-time primitives). xtremax is not yet a declared dependency. The remaining rows are tracked under the Tier V epic #75.

Connection to Tier III — spatial structure¶

Tier III’s distributed source field $S(\mathbf{x},t)$ is exactly a spatial inhomogeneous Poisson rate at the population level — temporally aggregated, this is the spatial intensity of a Cox process over emission events. The v2 spatial extension of Tier V is the same mathematical object Tier III already inverts at the per-event timescale, just averaged over a longer horizon. The two tiers should share the parameterisation: a Matérn GP prior on $\log S(\mathbf{x},t)$ plays the role of both Tier III’s source-field prior and Tier V.v2’s spatial Cox-process intensity.

This isn’t a coincidence — it’s why plumax’s tier structure works: the same mathematical objects appear at different scales.

Status snapshot¶

Theory. ✓ TMTPP foundations (TMTPP theory), the intensity zoo and persistency are theory pages; the missing-mass paradox, intensity gallery and POD gallery are executed notebooks.
methane_pod library: ✓ in tree — intensity, POD, paradox simulator and NUTS fitter now live in plumax.population.{intensity,pod,paradox,fitting}. Rebuilding the generative layer on xtremax.point_processes is what remains of #106. What neither library has is the importance-weighted soft-observation mark likelihood — that stays #108.
plumax.population subpackage: 🚧 — the v1 cross-tier catalog adapter (catalog.py), V.A hierarchical lognormal size-distribution fit (size_distribution.py), and V.B point-process core (point_process.py — closed-form Gamma-Poisson rate + log-linear inhomogeneous intensity) have landed (#14). The Gaussian-summary (emission_rate, emission_std) representation is consumed; the importance-corrected full-sample path is #107 + #108.
Cross-tier integration: 🚧 — the tier-agnostic catalog (event_from_posterior over GaussianPosterior / LognormalPosterior / FusionPosterior) and per-event uncertainty propagation into the population fit have landed. Per-event posteriors still enter as Gaussian summaries; the extended PerEventPosterior payload (samples + prior recall) is #107 and the importance correction — Tier V’s main outstanding code deliverable — is #108.
Synthetic validation. The stationary NumPyro MCMC notebook recovers POD parameters on synthetic data without the soft-observation layer; the in-tree SBC harness is #114.
Real-data fit. ☐ No real-data fit notebook yet; IMEO / Tanager / Carbon Mapper / GHGSat ingestion is #111. Persistency metrics (V.C) are #112, the total-emission estimator (V.D) is #113, Hawkes / LGCP intensities are #115.