Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

Part II — Building the mathematical objects

Each section introduces exactly one new idea and names the phase it serves.

§ObjectServes
II.1Measures and the point/area splitall phases
II.2Random measures, complete randomnessall phases
II.3Poisson processA
II.4Completely random measures (Poisson, Gamma, generalised Gamma, Beta)C (B for Beta)
II.5Normalisation: Dirichlet and Pitman–Yor processesC
II.6Random fields, Gaussian processesA, C
II.7Cox processA
II.8Scale: coarsening, transport, averagingall phases
II.9Thinning and the full observation operatorA, B
II.10TimeB, C

II.1 Regions and measures — and the point/area split (all phases)

Region

A⊂SA \subset S is any subset you could draw on a map.

Pairwise disjoint

A1,…,AnA_1,\dots,A_n with Ai∩Aj=∅A_i \cap A_j = \varnothing for i≠ji \neq j. Adjacent cells: disjoint. A cell and the basin containing it: not.

Partition

Pairwise-disjoint regions whose union is SS. The 2×22 \times 2 grid of §I.5 is one.

Measure

mm assigns to each region AA a number m(A)≥0m(A) \ge 0 with m(∅)=0m(\varnothing) = 0 and additivity over pairwise-disjoint regions:

m(A1∪A2∪⋯ )=m(A1)+m(A2)+⋯m(A_1 \cup A_2 \cup \cdots) = m(A_1) + m(A_2) + \cdots

Measures on SS: area ∣A∣\lvert A\rvert; emission rate μ(A)\mu(A); source count N(A)N(A). Not measures: max rate in AA; mean XCH4\mathrm{XCH_4} over AA.

Point mass

δx(A)=1\delta_x(A) = 1 if x∈Ax \in A, else 0.

Why the split matters for modelling. Atoms are described by a point process (§II.3–§II.5). A density e(x)e(x) is a field (§II.6). They are different random objects and get different priors. Everything else in this section is organised around that fact.

II.2 Randomising the inventory (all phases)

Random measure. Before looking at data, the number, locations, rates, and density are unknown. A random measure is a random variable whose value is an entire measure; for each fixed AA, μ(A)\mu(A) is an ordinary positive random variable.

Complete randomness — the one structural assumption. μ\mu is completely random if for any pairwise-disjoint A1,…,AnA_1,\dots,A_n, the totals μ(A1),…,μ(An)\mu(A_1),\dots,\mu(A_n) are mutually independent. Learning A1A_1 tells you nothing about A2A_2.

II.3 Counting point sources — the Poisson process (Phase A)

Ignore rates. N(A)N(A) is a Poisson process with intensity measure λ\lambda if

  1. N(A)∼Poisson(λ(A))N(A) \sim \mathrm{Poisson}(\lambda(A)), where λ(A)\lambda(A) is the expected number of sources in AA;

  2. counts in pairwise-disjoint regions are independent.

With density: λ(A)=∫Aλ(x) dx\lambda(A) = \int_A \lambda(x)\,\mathrm{d}x, λ(x)\lambda(x) in km−2\mathrm{km^{-2}}. Homogeneous: λ\lambda constant. Inhomogeneous: high over active fields, ≈0\approx 0 over rangeland.

NN is itself a completely random measure: the case where every atom weighs 1.

II.4 Attaching rates — the completely random measure (Phase C)

Work with dimensionless marks uk=sk/s0u_k = s_k/s_0, so that the physical rate is sk=s0uks_k = s_0 u_k. Treat the pairs (xk,uk)(x_k, u_k) as a Poisson process on S×(0,∞)S \times (0,\infty) with intensity

ν(dx,du)=λ(dx) ρ(du)\nu(\mathrm{d}x, \mathrm{d}u) = \lambda(\mathrm{d}x)\,\rho(\mathrm{d}u)

Reading. ν(A×[a,b])\nu(A \times [a,b]) = expected number of sources in AA with rate in [as0,bs0][a s_0, b s_0]. λ\lambda handles where, ρ\rho handles how big.

What ρ\rho is. Not a probability density. ρ(du)\rho(\mathrm{d}u) is the expected number of sources, per unit of λ\lambda, with dimensionless rate in du\mathrm{d}u. It may have infinite mass near 0; what is required is

∫0∞min⁡(u,1) ρ(du)<∞\int_0^\infty \min(u, 1)\,\rho(\mathrm{d}u) < \infty

Infinitely many negligible sources, finite total. This is the nonparametric part: the number of sources is never fixed.

The atomic random measure is μpt(A)=∑k: xk∈Ask=s0∑k: xk∈Auk\mu_{\mathrm{pt}}(A) = \sum_{k:\,x_k\in A} s_k = s_0 \sum_{k:\,x_k\in A} u_k, and its law in every region follows from ρ\rho via the Laplace functional (for dimensionless θ≥0\theta \ge 0)

E ⁣[exp⁡ ⁣(−θ μpt(A)s0)]=exp⁡ ⁣(−λ(A)∫0∞(1−e−θu) ρ(du))\mathbb{E}\!\left[\exp\!\left(-\theta\,\frac{\mu_{\mathrm{pt}}(A)}{s_0}\right)\right] = \exp\!\left(-\lambda(A)\int_0^\infty \bigl(1 - e^{-\theta u}\bigr)\,\rho(\mathrm{d}u)\right)

The choice of Lévy measure ρ\rho selects the member of the family:

II.4.1 Poisson process

ρ=δ1\rho = \delta_1. Every atom weighs 1; μpt(A)/s0=N(A)\mu_{\mathrm{pt}}(A)/s_0 = N(A). A prior over counts.

II.4.2 Gamma process — inventory prior with light tails

Take a prior inventory HH (kg h−1\mathrm{kg\,h^{-1}} per region, e.g. from a bottom-up product) and set λ(dx)=H(dx)/s0\lambda(\mathrm{d}x) = H(\mathrm{d}x)/s_0 and ρ(du)=u−1e−u du\rho(\mathrm{d}u) = u^{-1}e^{-u}\,\mathrm{d}u. Then

μpt(A)s0∼Gamma ⁣(shape=H(A)s0, rate=1),E[μpt(A)]=H(A)\frac{\mu_{\mathrm{pt}}(A)}{s_0} \sim \mathrm{Gamma}\!\left(\text{shape}=\frac{H(A)}{s_0},\ \text{rate}=1\right), \qquad \mathbb{E}[\mu_{\mathrm{pt}}(A)] = H(A)

II.4.3 Generalised Gamma — inventory prior with super-emitters

ρ(du)∝u−1−σe−τu du,σ∈(0,1), τ≥0\rho(\mathrm{d}u) \propto u^{-1-\sigma} e^{-\tau u}\,\mathrm{d}u, \qquad \sigma \in (0,1),\ \tau \ge 0

Power-law body, exponential cut-off at scale 1/τ1/\tau.

LimitResult
σ→0\sigma \to 0recovers the Gamma process
τ→0\tau \to 0pure σ\sigma-stable law: infinite mean, too heavy
σ∈(0,1), τ>0\sigma \in (0,1),\ \tau > 0tempered stable: heavy body, finite mean. Use this.

II.4.4 Beta process — persistence prior (Phase B)

ρ(dπ)=c π−1(1−π)c−1 dπ  on (0,1),λ=B0\rho(\mathrm{d}\pi) = c\, \pi^{-1}(1-\pi)^{c-1}\,\mathrm{d}\pi \ \text{ on } (0,1), \qquad \lambda = B_0

Every atom is now a probability πk∈(0,1)\pi_k \in (0,1): the fraction of time source kk is on. For overpass ii draw Zik∼Bernoulli(πk)Z_{ik} \sim \mathrm{Bernoulli}(\pi_k):

src 1src 2src 3src 4
pass 1●●○●
pass 2●●●●
pass 3●●○●
pass 4○●○●
pass 5●●●●
true πk\pi_k0.91.00.31.0

The overpass ×\times source matrix ZZ (filled = on, hollow = off) is the feature allocation; cc controls how many sources are persistent versus flickering. Integrating out the πk\pi_k gives the IBP on ZZ.[2]

Known candidate sites — the finite version. When the infrastructure database supplies candidate coordinates c1,…,cJc_1,\dots,c_J, there is nothing nonparametric about those: model each with Zj∼Bernoulli(πj)Z_j \sim \mathrm{Bernoulli}(\pi_j), πj∼Beta(a,b)\pi_j \sim \mathrm{Beta}(a,b). Sources not in the database (unknown-unknowns) need one marked process: locations from the Poisson/Cox process (§II.3, §II.7), with each atom carrying its own persistence mark πk∼G(dπ∣uk)\pi_k \sim G(\mathrm{d}\pi \mid u_k). This is the construction used in §III.1.

Known-unknowns → Phase B
  • sites cjc_j given

  • state ZjZ_j random

  • finite Beta–Bernoulli

Unknown-unknowns → Phase A
  • sites random

  • state and site random

  • Poisson/Cox process with persistence marks

II.5 Fractions and attribution — normalisation (Phase C)

If 0<μpt(S)<∞0 < \mu_{\mathrm{pt}}(S) < \infty, then G(A)=μpt(A)/μpt(S)G(A) = \mu_{\mathrm{pt}}(A)/\mu_{\mathrm{pt}}(S) is the fraction of the basin’s point total from AA. For the Gamma process, ratios of independent Gammas are Dirichlet, so for any partition:

(G(A1),…,G(An))∼Dirichlet ⁣(H(A1)s0,…,H(An)s0)\bigl(G(A_1),\dots,G(A_n)\bigr) \sim \mathrm{Dirichlet}\!\left(\frac{H(A_1)}{s_0},\dots,\frac{H(A_n)}{s_0}\right)

This is the Dirichlet process DP(H/s0)\mathrm{DP}(H/s_0): its base measure is the dimensionless H/s0H/s_0, whose total mass H(S)/s0H(S)/s_0 is the concentration.

Attribution reading. Each detected plume is assigned to a source.

Dirichlet process
Pitman–Yor

The number of distinct sources is unbounded and grows like log⁡n\log n with nn plumes; a source already holding many plumes attracts the next (rich-get-richer).

Attribution matters when two plumes in one fine-imager scene could come from one source or two.

II.6 Fields — for concentrations and for area sources (Phases A, C)

Three quantities in this problem are fields, not measures:

FieldMeaningUnitsWhere
f(x)f(x)XCH4\mathrm{XCH_4} enhancementppbwhat the satellite retrieves
e(x)e(x)diffuse flux densitykg h−1 km−2\mathrm{kg\,h^{-1}\,km^{-2}}area sources, §II.1
g(x)g(x)log source density—where point sources cluster, §II.7

None of them is additive over regions. All three need the same construction.

Random field. Specify, for every finite set of locations x1,…,xnx_1,\dots,x_n, the joint law of (f(x1),…,f(xn))(f(x_1),\dots,f(x_n)), subject to consistency: integrating the (n+1)(n+1)-point law over one coordinate must reproduce the nn-point law. A consistent family determines a unique stochastic process in distribution.[3] Consistency is what lets you say “the field”.

Pathwise quantities such as ∫Ae(x) dx\int_A e(x)\,\mathrm{d}x need more than the law: they need a measurable, integrable modification of the process. For the kernels used here (exponential, Matérn, squared-exponential), the Kolmogorov continuity criterion supplies a continuous one, which is what makes those integrals well defined.

Gaussian process GP(m,k)\mathcal{GP}(m,k).

(f(x1),…,f(xn))∼N(m,K),mi=m(xi),Kij=k(xi,xj)\bigl(f(x_1),\dots,f(x_n)\bigr) \sim \mathcal{N}(\mathbf{m}, \mathbf{K}), \qquad m_i = m(x_i),\quad K_{ij} = k(x_i, x_j)

with kk positive semi-definite. Consistency is automatic.

Concentration field
Area source

Beyond Gaussian.

Student-t process
Multivariate-tt finite-dimensional laws with common νt\nu_t; consistent; robust to outlier retrievals. νt→∞\nu_t \to \infty gives the GP.
Wishart process
Σ(x)=A(x)A(x)T\Sigma(x) = A(x)A(x)^{\mathsf T} with GP columns; a spatially varying covariance across jointly retrieved species.

II.7 Composing field and point process — the Cox process (Phase A)

Repair §II.2’s independence assumption: let the source density be a random field,

log⁡λ(x)=g(x),g∼GP(mg,kg);N∣λ∼Poisson process with intensity λ\log\lambda(x) = g(x),\quad g \sim \mathcal{GP}(m_g, k_g); \qquad N \mid \lambda \sim \text{Poisson process with intensity } \lambda

Given λ\lambda, disjoint cells are independent. Marginally they are correlated, because the same gg shaped both:

Cov(λ(x),λ(x′))=E[λ(x)] E[λ(x′)] (ekg(x,x′)−1),\mathrm{Cov}\bigl(\lambda(x), \lambda(x')\bigr) = \mathbb{E}[\lambda(x)]\,\mathbb{E}[\lambda(x')]\,\bigl(e^{k_g(x,x')} - 1\bigr),

positive wherever kg>0k_g > 0, as it is everywhere for the usual exponential, Matérn, and squared-exponential kernels, and negative where a kernel with negative lobes has kg<0k_g < 0. This is the LGCP, and it is the natural discovery prior: more detections in A1A_1 than its search effort predicts raise the posterior source density in adjacent A3A_3 before A3A_3 has been searched, and fewer lower it. The update depends on the count relative to its expectation, because the Poisson likelihood pairs the observed points with the integrated-intensity penalty −∫p λ-\int p\,\lambda.

II.8 Scale — coarsening, transport, and averaging (all phases)

This is where §I.2’s “point versus area depends on the instrument” becomes mathematics.

II.8.1 Spatial coarsening

Define the coarsening operator for pixel size Δx\Delta x:

CΔ μ=(μ(P1),μ(P2),… ),{Pj} the Δx-grid partition of SC_\Delta\,\mu = \bigl(\mu(P_1), \mu(P_2), \dots\bigr), \qquad \{P_j\} \text{ the } \Delta x\text{-grid partition of } S

CΔC_\Delta maps a measure to a vector of pixel totals in kg h−1\mathrm{kg\,h^{-1}}. Two facts:

  1. Coarsening forgets the atomic/diffuse split. μ(Pj)=μpt(Pj)+μdf(Pj)\mu(P_j) = \mu_{\mathrm{pt}}(P_j) + \mu_{\mathrm{df}}(P_j) is one number; a pixel containing twenty 20 kg h−120\ \mathrm{kg\,h^{-1}} pads is indistinguishable from a pixel with 400 kg h−1400\ \mathrm{kg\,h^{-1}} of diffuse flux.

  2. A point source is an atom relative to Δx\Delta x. Source kk is “point” for an instrument if its physical extent ≪Δx\ll \Delta x and it is the dominant contributor to μ(Pj)\mu(P_j) for its pixel; otherwise it is part of an aggregate.

Fine Δx (0.03 km)
+---+---+---+---+
|   | o |   |   |   atoms resolve,
+---+---+---+---+   each in its
|   |   |   | o |   own pixel
+---+---+---+---+
→ point-process model
Coarse Δx (7 km)
+---------------+
|  o    o       |   atoms merge
|     ::::::    |   with diffuse
|   :::::::     |   into one total
+---------------+
→ field model of C_Δ μ

II.8.2 Atmospheric transport — the field-side observation operator

The satellite does not retrieve μ\mu; it retrieves the raw column y(x)=XCH4(x)y(x) = \mathrm{XCH_4}(x). The link is a transport operator

y(x)=(Tμ)(x)+fbg(x)+εr(x),εr∼N(0,σr2)y(x) = (\mathcal{T}\mu)(x) + f_{\mathrm{bg}}(x) + \varepsilon_r(x), \qquad \varepsilon_r \sim \mathcal{N}(0, \sigma_r^2)

where T\mathcal{T} convolves emissions with a plume kernel driven by wind UU, and fbgf_{\mathrm{bg}} is background. The enhancement product of §I.4 has the background already removed, f=y−fbg=Tμ+εrf = y - f_{\mathrm{bg}} = \mathcal{T}\mu + \varepsilon_r. Model one or the other, never fbgf_{\mathrm{bg}} twice.

Per-plume rate estimation (IME and cross-sectional methods) is the local inversion of T\mathcal{T} for one atom; it returns a noisy skηks_k\eta_k with ηk\eta_k log-normal (§II.9.3). In plumax the forward side of T\mathcal{T} is the Tier I–III dispersion models (Gaussian plume/puff, Eulerian FV).

II.8.3 Temporal scale — averaging and sampling

Two time scales matter: the instrument revisit Δt\Delta t, and the source’s own on/off timescale.

Time-averaged rate. Over a window much longer than the on/off cycle,

sˉk=πksk[kg h−1]\bar{s}_k = \pi_k s_k \quad [\mathrm{kg\,h^{-1}}]

This is the quantity an inventory wants; sks_k (the on-state rate) is what a single fine-imager scene measures.

Averaging area sources
Sampling intermittent point sources

Retrieval noise σr\sigma_r averages down as 1/n1/\sqrt{n} over nn clear overpasses while a persistent diffuse enhancement does not, so area sources emerge from stacks that no single scene shows. The coarse mapper’s daily revisit is what makes this work: 30 clear days gives noise ≈σr/30≈2\approx \sigma_r/\sqrt{30} \approx 2 ppb against a field-DD enhancement of a few ppb.

The 1/n1/\sqrt{n} law assumes independent, zero-mean errors. Errors shared across scenes (background subtraction, calibration, transport) give a stack variance of 1TΣ 1/n2\mathbf{1}^{\mathsf T}\Sigma\,\mathbf{1}/n^2, which approaches a nonzero floor, so treat 2 ppb as a lower bound unless the temporal error covariance Σ\Sigma is carried through.

II.9 Observing point sources — thinning and the full operator (Phases A, B)

II.9.1 Thinning

Definition. Given a point process NN with points {xk}\{x_k\} and a retention function p:S→[0,1]p: S \to [0,1], keep each xkx_k independently with probability p(xk)p(x_k). The kept points form NobsN_{\mathrm{obs}}.

II.9.2 Rate-dependent thinning and the detection limit

Let the detection probability depend on the rate, writing p(x,u)p(x,u) for a source at xx with rate s=s0us = s_0 u. A CRM with intensity λ(dx)ρ(du)\lambda(\mathrm{d}x)\rho(\mathrm{d}u) is observed as

λ(dx) p(x,u) ρ(du)\lambda(\mathrm{d}x)\,p(x,u)\,\rho(\mathrm{d}u)

Take p(u)p(u) to be the POD curve, rising around smin⁡/s0s_{\min}/s_0, and require

∫0∞p(u) ρ(du)<∞.\int_0^\infty p(u)\,\rho(\mathrm{d}u) < \infty .

Under that condition:

  1. ρobs=pρ\rho_{\mathrm{obs}} = p\rho has finite mass: you observe finitely many sources. “Infinitely many” was about the truth, never the data.

  2. The observed total is biased low by the factor

    ∫u p(u) ρ(du)∫u ρ(du)\frac{\int u\,p(u)\,\rho(\mathrm{d}u)}{\int u\,\rho(\mathrm{d}u)}

    This is the below-detection-limit gap, in one line.

II.9.3 The full observation operator for point sources

Each stage preserves Poisson-ness (thinning theorem; displacement theorem; superposition), so the whole operator is tractable.

Three differences from a data-assimilation H\mathcal{H}:

  1. Random, not deterministic.

  2. Dimension-changing.

  3. Not invertible. Only the product of the four thinning probabilities with λ\lambda is identifiable. Separating “source off”, “not overhead”, “cloudy”, and “too small” requires each pp calibrated outside the data: controlled releases, orbit files, cloud masks.

II.10 Time (Phases B, C)

Nothing above required XX to be spatial. Time adds order.

II.10.1 Cumulative view — subordinators

Model discrete releases at tkt_k with masses mkm_k [kg]:

M(t)=∑k: tk≤tmkM(t) = \sum_{k:\,t_k \le t} m_k

MM is always nondecreasing. It is a subordinator (independent, stationary increments, i.e. a CRM on the time axis) only if the pairs (tk,mk)(t_k, m_k) form a Poisson random measure with time-homogeneous intensity dt ρ(dm)\mathrm{d}t\,\rho(\mathrm{d}m), so that release times carry no memory and no seasonality. Under that assumption: Poisson gives a counting process; Gamma gives the Gamma subordinator; stable gives rare huge jumps.

Renewal, Hawkes, and time-varying intensities (§II.10.2) break the assumption, and then M(t)M(t) is not a subordinator.

II.10.2 Conditional intensity — history dependence

λ∗(t)=lim⁡dt→0P(event in [t,t+dt)∣Ht)dt[d−1]\lambda^*(t) = \lim_{\mathrm{d}t \to 0} \frac{P(\text{event in } [t, t+\mathrm{d}t) \mid \mathcal{H}_t)}{\mathrm{d}t} \quad [\mathrm{d^{-1}}]

Poisson: λ∗\lambda^* ignores Ht\mathcal{H}_t (memoryless). Others:

Renewal
Hawkes
Cox in time
λ∗(t)=h(t−tlast)\lambda^*(t) = h(t - t_{\mathrm{last}})

Regular schedules (blowdowns every ≈7\approx 7 d) are under-dispersed relative to Poisson.

One likelihood covers all:

log⁡L=∑ilog⁡λ∗(ti)−∫0Tendλ∗(t) dt\log L = \sum_i \log\lambda^*(t_i) - \int_0^{T_{\mathrm{end}}} \lambda^*(t)\,\mathrm{d}t

II.10.3 GPs in time are SDEs — the Kalman link

Matérn GPs over tt with half-integer smoothness ν=p+12\nu = p + \tfrac12 are exactly linear SDEs with a (p+1)(p+1)-dimensional state. General ν\nu has no finite state-space form. The simplest case, ν=12\nu = \tfrac12, with WW standard Brownian motion:

k(t,t′)=σf2exp⁡ ⁣(−∣t−t′∣ℓt)⟺df=−1ℓtf dt+2σf2ℓt dWk(t,t') = \sigma_f^2 \exp\!\left(-\frac{\lvert t - t'\rvert}{\ell_t}\right) \quad\Longleftrightarrow\quad \mathrm{d}f = -\frac{1}{\ell_t} f\,\mathrm{d}t + \sqrt{\frac{2\sigma_f^2}{\ell_t}}\,\mathrm{d}W

(Ornstein–Uhlenbeck). For these kernels GP smoothing becomes a Kalman filter + smoother, O(n)O(n). Squared-exponential kernels (and Matérn with non-half-integer ν\nu) get no exact such structure, only approximations.

II.10.4 Space × time

X=S×TX = S \times T. Kernels are separable (kS⋅kTk_S \cdot k_T) or non-separable; the advective kernel depends on the displacement x−x′−v(t−t′)x - x' - \mathbf{v}(t - t') with velocity vector v=86.4 U km d−1\mathbf{v} = 86.4\,\mathbf{U}\ \mathrm{km\,d^{-1}}, a plume carried by the wind in the wind’s direction. Per-source rate functions sk(t)s_k(t): the continuous-time version of §II.4.4 is a two-state Markov (or semi-Markov, for non-exponential durations) switching process. This is the right model for a leak that stays on until repair: the on-state duration is the repair time.

II.10.5 Clustering over time

Dependent DP: stick-breaking with time-varying Vk(t)V_k(t), so attribution fractions drift smoothly instead of being re-estimated monthly.

II.10.6 The observation clock is a thinned point process

Overpass times form a point process on TT (deterministic per satellite, near-Poisson for a constellation), thinned by cloud. That is the “clock” and “clear sky” stages of §II.9.3, seen from the time axis.

Footnotes
  1. The Lebesgue decomposition, with respect to area, into atomic, absolutely continuous, and singular-continuous parts.

  2. Griffiths & Ghahramani’s Indian Buffet Process: the exchangeable distribution over binary feature matrices with an unbounded number of columns.

  3. The Kolmogorov extension theorem.