Each section introduces exactly one new idea and names the phase it serves.
| § | Object | Serves |
|---|---|---|
| II.1 | Measures and the point/area split | all phases |
| II.2 | Random measures, complete randomness | all phases |
| II.3 | Poisson process | A |
| II.4 | Completely random measures (Poisson, Gamma, generalised Gamma, Beta) | C (B for Beta) |
| II.5 | Normalisation: Dirichlet and Pitman–Yor processes | C |
| II.6 | Random fields, Gaussian processes | A, C |
| II.7 | Cox process | A |
| II.8 | Scale: coarsening, transport, averaging | all phases |
| II.9 | Thinning and the full observation operator | A, B |
| II.10 | Time | B, C |
II.1 Regions and measures — and the point/area split (all phases)¶
- Region
is any subset you could draw on a map.
- Pairwise disjoint
with for . Adjacent cells: disjoint. A cell and the basin containing it: not.
- Partition
Pairwise-disjoint regions whose union is . The grid of §I.5 is one.
- Measure
assigns to each region a number with and additivity over pairwise-disjoint regions:
Measures on : area ; emission rate ; source count . Not measures: max rate in ; mean over .
- Point mass
if , else 0.
Running example
Why the split matters for modelling. Atoms are described by a point process (§II.3–§II.5). A density is a field (§II.6). They are different random objects and get different priors. Everything else in this section is organised around that fact.
II.2 Randomising the inventory (all phases)¶
Random measure. Before looking at data, the number, locations, rates, and density are unknown. A random measure is a random variable whose value is an entire measure; for each fixed , is an ordinary positive random variable.
Complete randomness — the one structural assumption. is completely random if for any pairwise-disjoint , the totals are mutually independent. Learning tells you nothing about .
II.3 Counting point sources — the Poisson process (Phase A)¶
Ignore rates. is a Poisson process with intensity measure if
, where is the expected number of sources in ;
counts in pairwise-disjoint regions are independent.
With density: , in . Homogeneous: constant. Inhomogeneous: high over active fields, over rangeland.
Running example
Suppose over active acreage, elsewhere. (, mostly active) has : we expect about 15 point sources there, standard deviation .
Our four “true” sources are the ones large enough to matter; the Poisson prior also expects many tiny ones (see §II.4).
is itself a completely random measure: the case where every atom weighs 1.
II.4 Attaching rates — the completely random measure (Phase C)¶
Work with dimensionless marks , so that the physical rate is . Treat the pairs as a Poisson process on with intensity
Reading. = expected number of sources in with rate in . handles where, handles how big.
What is. Not a probability density. is the expected number of sources, per unit of , with dimensionless rate in . It may have infinite mass near 0; what is required is
Infinitely many negligible sources, finite total. This is the nonparametric part: the number of sources is never fixed.
The atomic random measure is , and its law in every region follows from via the Laplace functional (for dimensionless )
The choice of Lévy measure selects the member of the family:
II.4.1 Poisson process¶
. Every atom weighs 1; . A prior over counts.
II.4.2 Gamma process — inventory prior with light tails¶
Take a prior inventory ( per region, e.g. from a bottom-up product) and set and . Then
Running example
gives : mean 500, standard deviation . The true 410 sits standard deviations below the mean: this prior all but rules the truth out.
II.4.3 Generalised Gamma — inventory prior with super-emitters¶
Power-law body, exponential cut-off at scale .
| Limit | Result |
|---|---|
| recovers the Gamma process | |
| pure -stable law: infinite mean, too heavy | |
| tempered stable: heavy body, finite mean. Use this. |
Running example
Source 3 is 66 % of . A Gamma prior calls that an outlier; a tempered-stable prior with expects exactly this shape.
II.4.4 Beta process — persistence prior (Phase B)¶
Every atom is now a probability : the fraction of time source is on. For overpass draw :
| src 1 | src 2 | src 3 | src 4 | |
|---|---|---|---|---|
| pass 1 | ● | ● | ○ | ● |
| pass 2 | ● | ● | ● | ● |
| pass 3 | ● | ● | ○ | ● |
| pass 4 | ○ | ● | ○ | ● |
| pass 5 | ● | ● | ● | ● |
| true | 0.9 | 1.0 | 0.3 | 1.0 |
The overpass source matrix (filled = on, hollow = off) is the feature allocation; controls how many sources are persistent versus flickering. Integrating out the gives the IBP on .[2]
Known candidate sites — the finite version. When the infrastructure database supplies candidate coordinates , there is nothing nonparametric about those: model each with , . Sources not in the database (unknown-unknowns) need one marked process: locations from the Poisson/Cox process (§II.3, §II.7), with each atom carrying its own persistence mark . This is the construction used in §III.1.
sites given
state random
finite Beta–Bernoulli
sites random
state and site random
Poisson/Cox process with persistence marks
II.5 Fractions and attribution — normalisation (Phase C)¶
If , then is the fraction of the basin’s point total from . For the Gamma process, ratios of independent Gammas are Dirichlet, so for any partition:
This is the Dirichlet process : its base measure is the dimensionless , whose total mass is the concentration.
Running example
gives fractions centred on .
Attribution reading. Each detected plume is assigned to a source.
The number of distinct sources is unbounded and grows like with plumes; a source already holding many plumes attracts the next (rich-get-richer).
Attribution matters when two plumes in one fine-imager scene could come from one source or two.
II.6 Fields — for concentrations and for area sources (Phases A, C)¶
Three quantities in this problem are fields, not measures:
| Field | Meaning | Units | Where |
|---|---|---|---|
| enhancement | ppb | what the satellite retrieves | |
| diffuse flux density | area sources, §II.1 | ||
| log source density | — | where point sources cluster, §II.7 |
None of them is additive over regions. All three need the same construction.
Random field. Specify, for every finite set of locations , the joint law of , subject to consistency: integrating the -point law over one coordinate must reproduce the -point law. A consistent family determines a unique stochastic process in distribution.[3] Consistency is what lets you say “the field”.
Pathwise quantities such as need more than the law: they need a measurable, integrable modification of the process. For the kernels used here (exponential, Matérn, squared-exponential), the Kolmogorov continuity criterion supplies a continuous one, which is what makes those integrals well defined.
Gaussian process .
with positive semi-definite. Consistency is automatic.
Beyond Gaussian.
- Student-t process
- Multivariate- finite-dimensional laws with common ; consistent; robust to outlier retrievals. gives the GP.
- Wishart process
- with GP columns; a spatially varying covariance across jointly retrieved species.
II.7 Composing field and point process — the Cox process (Phase A)¶
Repair §II.2’s independence assumption: let the source density be a random field,
Given , disjoint cells are independent. Marginally they are correlated, because the same shaped both:
positive wherever , as it is everywhere for the usual exponential, Matérn, and squared-exponential kernels, and negative where a kernel with negative lobes has . This is the LGCP, and it is the natural discovery prior: more detections in than its search effort predicts raise the posterior source density in adjacent before has been searched, and fewer lower it. The update depends on the count relative to its expectation, because the Poisson likelihood pairs the observed points with the integrated-intensity penalty .
Running example
With ’s correlation length 20 km, a search of moves by a fraction set by . Because this exponential kernel is positive, moves in the direction of the surprise. Five detections against 4.5 expected would raise it; the single detection of §III.1 against 4.5 expected lowers both and . Under plain Poisson, would not move either way.
II.8 Scale — coarsening, transport, and averaging (all phases)¶
This is where §I.2’s “point versus area depends on the instrument” becomes mathematics.
II.8.1 Spatial coarsening¶
Define the coarsening operator for pixel size :
maps a measure to a vector of pixel totals in . Two facts:
Coarsening forgets the atomic/diffuse split. is one number; a pixel containing twenty pads is indistinguishable from a pixel with of diffuse flux.
A point source is an atom relative to . Source is “point” for an instrument if its physical extent and it is the dominant contributor to for its pixel; otherwise it is part of an aggregate.
+---+---+---+---+
| | o | | | atoms resolve,
+---+---+---+---+ each in its
| | | | o | own pixel
+---+---+---+---+
→ point-process model+---------------+
| o o | atoms merge
| :::::: | with diffuse
| ::::::: | into one total
+---------------+
→ field model of C_Δ μRunning example
The coarse mapper’s 7 km pixel over the north-east of contains source 2 () and roughly a quarter of field (): for that pixel, and the mapper cannot tell the two apart. The fine imager resolves source 2 geometrically (its footprint would be a compact plume), but at it is below the detection limit, so it goes undetected (§II.9.2); field produces no plume at all. Resolution and detection are different properties.
II.8.2 Atmospheric transport — the field-side observation operator¶
The satellite does not retrieve ; it retrieves the raw column . The link is a transport operator
where convolves emissions with a plume kernel driven by wind , and is background. The enhancement product of §I.4 has the background already removed, . Model one or the other, never twice.
For a single atom, is a compact plume of peak enhancement roughly .
For a diffuse patch, is a broad, low enhancement.
Per-plume rate estimation (IME and cross-sectional methods) is the local inversion of for one atom; it returns a noisy with log-normal (§II.9.3). In plumax the forward side of is the Tier I–III dispersion models (Gaussian plume/puff, Eulerian FV).
II.8.3 Temporal scale — averaging and sampling¶
Two time scales matter: the instrument revisit , and the source’s own on/off timescale.
Time-averaged rate. Over a window much longer than the on/off cycle,
This is the quantity an inventory wants; (the on-state rate) is what a single fine-imager scene measures.
Retrieval noise averages down as over clear overpasses while a persistent diffuse enhancement does not, so area sources emerge from stacks that no single scene shows. The coarse mapper’s daily revisit is what makes this work: 30 clear days gives noise ppb against a field- enhancement of a few ppb.
The law assumes independent, zero-mean errors. Errors shared across scenes (background subtraction, calibration, transport) give a stack variance of , which approaches a nonzero floor, so treat 2 ppb as a lower bound unless the temporal error covariance is carried through.
The fine imager at d samples . If overpass times are independent of the source’s state, the fraction of “on” scenes is an unbiased estimate of .
Running example — source 3
, , . Ten fine-imager overpasses, eight clear, three showing a plume, each estimating (the same sequence as §III.2). The raw fraction estimates , not (§II.9.3). Dividing by the POD, , gives , , and against a truth of 123. The gap is sampling noise on eight scenes (§III.2 gives the interval), and the estimate is unbiased only if the overpasses are a fair sample of the source’s schedule.
II.9 Observing point sources — thinning and the full operator (Phases A, B)¶
II.9.1 Thinning¶
Definition. Given a point process with points and a retention function , keep each independently with probability . The kept points form .
Running example
, over on a given day, so and the undetected , independent.
II.9.2 Rate-dependent thinning and the detection limit¶
Let the detection probability depend on the rate, writing for a source at with rate . A CRM with intensity is observed as
Take to be the POD curve, rising around , and require
Under that condition:
has finite mass: you observe finitely many sources. “Infinitely many” was about the truth, never the data.
The observed total is biased low by the factor
This is the below-detection-limit gap, in one line.
Running example
Fine imager, : sources 2 and 4 (35, 60) are essentially invisible; sources 1 and 3 are visible when on. Observable point total of instantaneous, and a further factor of once averaged.
II.9.3 The full observation operator for point sources¶
Each stage preserves Poisson-ness (thinning theorem; displacement theorem; superposition), so the whole operator is tractable.
Three differences from a data-assimilation :
Random, not deterministic.
Dimension-changing.
Not invertible. Only the product of the four thinning probabilities with is identifiable. Separating “source off”, “not overhead”, “cloudy”, and “too small” requires each calibrated outside the data: controlled releases, orbit files, cloud masks.
II.10 Time (Phases B, C)¶
Nothing above required to be spatial. Time adds order.
II.10.1 Cumulative view — subordinators¶
Model discrete releases at with masses [kg]:
is always nondecreasing. It is a subordinator (independent, stationary increments, i.e. a CRM on the time axis) only if the pairs form a Poisson random measure with time-homogeneous intensity , so that release times carry no memory and no seasonality. Under that assumption: Poisson gives a counting process; Gamma gives the Gamma subordinator; stable gives rare huge jumps.
Renewal, Hawkes, and time-varying intensities (§II.10.2) break the assumption, and then is not a subordinator.
II.10.2 Conditional intensity — history dependence¶
Poisson: ignores (memoryless). Others:
Regular schedules (blowdowns every d) are under-dispersed relative to Poisson.
Each emission initiation raises the near-term rate of new, distinct initiations: a process upset that triggers follow-on venting, or cascading equipment failures.
A single leak that stays on until repair is not a Hawkes process. It is one initiation followed by an on-state duration, modelled with the switching process of §II.10.4. Repeat satellite detections of the same leak are observations of that state, not new events.
One likelihood covers all:
II.10.3 GPs in time are SDEs — the Kalman link¶
Matérn GPs over with half-integer smoothness are exactly linear SDEs with a -dimensional state. General has no finite state-space form. The simplest case, , with standard Brownian motion:
(Ornstein–Uhlenbeck). For these kernels GP smoothing becomes a Kalman filter + smoother, . Squared-exponential kernels (and Matérn with non-half-integer ) get no exact such structure, only approximations.
II.10.4 Space × time¶
. Kernels are separable () or non-separable; the advective kernel depends on the displacement with velocity vector , a plume carried by the wind in the wind’s direction. Per-source rate functions : the continuous-time version of §II.4.4 is a two-state Markov (or semi-Markov, for non-exponential durations) switching process. This is the right model for a leak that stays on until repair: the on-state duration is the repair time.
II.10.5 Clustering over time¶
Dependent DP: stick-breaking with time-varying , so attribution fractions drift smoothly instead of being re-estimated monthly.
II.10.6 The observation clock is a thinned point process¶
Overpass times form a point process on (deterministic per satellite, near-Poisson for a constellation), thinned by cloud. That is the “clock” and “clear sky” stages of §II.9.3, seen from the time axis.