How code in the stack is packaged (function, Module, Operator), what data it sees (array, named array, carrier), and the role each name plays in its library's API, from structures up to recipes. This page holds the problem, the objects in symbols, the role ladder, who works where, one vocabulary per axis, and the rules that follow. The layer map keeps the layers, the imports and the boundaries of each library, and links here.
Take one running example: a methane plume in an EMIT hyperspectral scene over an oil and gas field. A matched filter turns the radiance into a methane enhancement, the plume is masked, and its emission rate comes from the mass of methane in the plume and the wind. Every step of it can go wrong in a way that raises no error.
| Problem | What goes wrong, silently | What fixes it |
|---|---|---|
| Axes by position | EMIT's radiance file stores (downtrack, crosstrack, band), which is (y, x, band); geoarray and most operators expect (band, y, x). mean(axis=0), meant as the scene's mean spectrum, averages down the rows instead and returns an array of a plausible size. | named array: dims by name, mean(("y", "x")) |
| Lost georeference | The radiance comes in sensor geometry, with its geolocation in a separate lookup table. Orthorectify it by hand, or crop the bare array, np.asarray(scene)[..., 100:164, 100:164], and the transform no longer matches the pixels: the crop claims to start at the scene's corner, and the plume is attributed to a facility kilometres away. | carrier: slicing and cropping move the transform with the data |
| Mismatched grids | The wind for the emission rate comes from ERA5 on a 0.25° grid, the enhancement on a 60 m UTM grid. Index the wind array with the plume's pixel indices and you read the wind somewhere else on Earth, with no error as long as the indices fit. | carrier: require_grid_match refuses; values on another grid are read at coordinates (interp_at) |
| Nodata | A radiance fill of -9999 enters the background mean and covariance as if it were data, and biases the filter over the whole scene, not only where it sits. | carrier: fill_value and valid_mask |
| Maths locked in a container | The matched filter written against xarray can't run on a GeoArray, and neither can be jitted: a CRS is not something JAX can trace. Each container gets its own copy of the algorithm. | function and Module on plain arrays; the container is unwrapped outside them |
| Setup that can't be traced | The background statistics (a mean, a shrunk covariance from scikit-learn, a solve with it) are refitted on every call, or forced under jit where they fail. | Module: fit once at fit, call jittably after |
| Pipelines held together by scripts | The chain from radiance to emission rate lives in one notebook; its settings (spectral window, threshold, wind source) can't be saved, swept or rerun exactly. | Operator: config round-trip, |, Graph |
The fixes split cleanly. The first four problems are about the data; the last three are about how the code is packaged. That is why the stack has two axes.
The whole stack rests on two ideas: a function of arrays, , and an Operator that lifts that function onto geographic data, . Everything else is bookkeeping about what the data knows and where the settings live. The symbols, each defined in full with its shape in the sections below:
Two axes follow from those ideas: what the data knows ( or ), and how the code is packaged (, a Module, or ).
The running example is an orthorectified EMIT scene over an oil and gas field in the Permian Basin: bands from 381 to 2493 nm, and pixels of 60 m on a UTM grid. A matched filter turns its radiance into a methane enhancement in ppm m; a threshold and connected components mask the plume; the integrated mass enhancement and an ERA5 wind give the emission rate. Its dimension names follow geoarray, ("band", "y", "x"); the sizes are written and so that stays the output of a function.
A block of numbers with a shape and nothing else:
Example The scene is , so . Which axis is the band is a convention the code has to remember. EMIT's radiance file stores (y, x, band) and geoarray (band, y, x); a mean over axis 0 meant as the mean spectrum averages down the rows instead, and runs without an error.
In code a NumPy or JAX array. Its axis order lives in a jaxtyping annotation and in each einx pattern, so the convention is written down at every call, but nothing checks it against the data: a scene read as (y, x, band) passes the same call and averages over the wrong axes.
Reading the sketches. Each block opens with a # Shapes: line naming its dimensions; the column on the right tracks shapes, (B, R, C) → (B,), in those letters. A call into the stack has a marker above it: # today (pkg) ships now, # proposed (pkg) does not yet, and # user is a function or class this page defines. The equation it computes sits above each line.
import einx
import jax.numpy as jnp
import numpy as np
from jaxtyping import Array, Float
# Shapes: B = N_band = 285 bands; R × C = N_row × N_col = 256 × 256 pixels
def mean_spectrum(radiance: Float[Array, "band y x"]) -> Float[Array, " band"]:
"""Mean radiance spectrum of a scene, averaged over its pixels.
x̄_b = (1 / R C) ∑_ij x_bij
Algorithm:
1. Average over the axes the pattern names y and x.
Args:
radiance: x, the scene, with the band axis first.
Returns:
x̄, one value per band. Nothing checks that the band axis really is
first: a (y, x, band) array gives a wrong answer and no error.
"""
# x̄_b = (1 / R C) ∑_ij x_bij; the axis names live in this pattern only
spectrum = einx.mean("band [y x]", radiance) # (B, R, C) → (B,)
return spectrum
# x ∈ ℝ^(B × R × C): a crop of an EMIT scene, on the host
radiance = np.zeros((285, 256, 256), dtype=np.float32) # → (B, R, C)
# the same numbers on the device
scene = jnp.asarray(radiance) # (B, R, C) → (B, R, C)
# user
# x̄
spectrum = mean_spectrum(scene) # (B, R, C) → (B,)
The same numbers with a name and a coordinate vector for each axis:
An operation addresses an axis by name, and the array finds its position. Take a reduction (a mean, a sum, a maximum) over axis of a plain array: it maps to the same space with the factor left out. On a named array the same reduction is asked for by a name . The array looks up its position, , reduces there, and removes that axis's name and coordinates, since the output no longer has that axis:
Example , with the band wavelengths. The mean spectrum reduces by name twice, over at and then over , and returns names with the wavelengths still attached, whatever order the reader used.
In code pipekit-array's DimArray: the data, one name per axis, and static labels on the dims that have them (here the wavelengths), with no other coordinates. It is a JAX pytree whose only leaf is the data, so new values never recompile. An xarray DataArray is the same object with full coordinates, and pipekit's dim functions (reduce, sel, isel, transpose) run on both through their adapters.
import einx
import numpy as np
import pipekit_array as pa
# Shapes: B = N_band = 285 bands; R × C = N_row × N_col = 256 × 256 pixels
radiance = np.zeros((285, 256, 256), dtype=np.float32) # → (B, R, C)
# c_1: the centre wavelength of each band, nm
wavelengths = np.linspace(381.0, 2493.0, 285) # → (B,)
rounded = np.round(wavelengths, 1) # (B,) → (B,)
WAVELENGTHS_NM = tuple(rounded.tolist()) # (B,) → B labels
# today (pipekit-array)
# x_named = (x, d, C): one name per axis, the band dim labelled by wavelength
scene = pa.DimArray(radiance, ("band", "y", "x"), {"band": WAVELENGTHS_NM})
# today (pipekit-array)
# r_y, then r_x: the mean spectrum, found by name
spectrum = pa.reduce(scene, "mean", ("y", "x")) # (B, R, C) → (B,)
# the file's own order, (y, x, band)
as_stored = einx.id("band y x -> y x band", radiance) # (B, R, C) → (R, C, B)
# today (pipekit-array)
# the same numbers, named in that order
stored = pa.DimArray(as_stored, ("y", "x", "band"), {"band": WAVELENGTHS_NM})
# today (pipekit-array)
# r_y, then r_x: the same call gives the same answer
same_spectrum = pa.reduce(stored, "mean", ("y", "x")) # (R, C, B) → (B,)
A named array plus its georeference :
The transform says where each pixel is. The top-left corner of the pixel in row and column sits at map coordinates , and its centre at in pixel units:
The fill value says which pixels are missing. A pixel is valid when every band there is finite and differs from the fill, which gives the valid mask , with indexing bands:
For a swath or a curvilinear grid, as EMIT's radiance is before orthorectification, is replaced by per-pixel coordinates ; the rest is the same.
Example The scene is a carrier on a 60 m grid in UTM zone 13N with , the band names and centre wavelengths in , and the sun and view zenith angles in . The well pad at map coordinates is found at pixel by solving for , which is to_pixel. Slice the bare array instead, np.asarray(scene)[..., 100:164, 100:164], and still describes the full scene: the window claims to start at the scene's corner, and the plume is attributed to a facility several kilometres away.
In code geotoolz-array's GeoArray: the data (NumPy or JAX), its dims ending in ("y", "x"), coordinates for the other dims (SpectralAxis, TimeAxis), the transform, CRS, fill value and frozen attrs. Slicing, crop and read_window move the transform with the pixels; a reduction over y and x leaves no grid, so it returns a named array. GeoSwath carries per-pixel latitude and longitude instead of a transform, and GeoPoints a set of points; xrops plays the same role for a DataArray whose coordinates and CRS mean geography.
import numpy as np
import pipekit_array as pa
from geoarray import GeoArray, crop, to_pixel
from geoarray.coords import SpectralAxis
# Shapes: B = N_band = 285 bands; R × C = N_row × N_col = 256 × 256 pixels
radiance = np.zeros((285, 256, 256), dtype=np.float32) # → (B, R, C)
# the centre wavelength of each band, nm, and a name for each
wavelengths = np.linspace(381.0, 2493.0, 285) # → (B,)
rounded = np.round(wavelengths, 1) # (B,) → (B,)
WAVELENGTHS_NM = tuple(rounded.tolist()) # (B,) → B labels
BAND_NAMES = tuple(f"{wavelength:.1f}nm" for wavelength in WAVELENGTHS_NM)
# today (geoarray)
# C: the band names with their centre wavelengths
bands = SpectralAxis(names=BAND_NAMES, wavelength_nm=WAVELENGTHS_NM)
# today (geoarray)
# x_geo = (x_named, Γ), Γ = (crs, A, x_fill, M): 60 m in UTM zone 13N
scene: GeoArray["band y x"] = GeoArray( # (B, R, C) → (B, R, C)
radiance,
(60.0, 0.0, 630000.0, 0.0, -60.0, 3550000.0),
"EPSG:32613",
fill_value=-9999.0,
dims="band y x",
coords=(bands,),
attrs={"solar_zenith_deg": 28.4, "view_zenith_deg": 6.1},
)
# today (geoarray)
# m_ij = ∏_b 1[x_bij ≠ x_fill] 1[x_bij is finite]
valid = scene.valid_mask() # (B, R, C) → (R, C)
# s: the well pad in map coordinates, m
well_pad_xy = np.array([[637530.0, 3542210.0]]) # → (1, 2)
# today (geoarray)
# s = A (j + ½, i + ½, 1)ᵀ, solved for the fractional (i, j)
well_pad_pixel = to_pixel(scene.transform, well_pad_xy) # (1, 2) → (1, 2)
# today (geoarray)
# a 64 × 64 window from row 100, column 100; A moves with the pixels
window = crop(scene, (100, 100), (64, 64)) # (B, R, C) → (B, 64, 64)
# today (pipekit-array)
# r_y, then r_x: no grid is left, so a named array comes back
spectrum = pa.reduce(scene, "mean", ("y", "x")) # (B, R, C) → (B,)
A pure function of arrays, with its parameters passed in on every call:
never sees a name, a CRS or a fill value. That is what lets it run under jit, on any carrier, in any library below GeoStack.
Example The matched filter, the first step of the running example. With indexing pixels and the spectrum at pixel , it estimates the methane enhancement , in ppm m, that best explains how departs from the background. Here the background is given, as the parameters :
The target signature is the radiance change for one ppm m of methane. Linearising the Beer–Lambert law, for band , gives
With the background given, the estimate is the generalised least-squares fit of a simple model, a pixel's spectrum as the background plus ppm m of methane along , with Gaussian clutter ; it is also the maximum-likelihood estimate:
Setting the derivative in to zero gives the closed form above; its standard error is , the same at every pixel.
The output keeps a singleton band axis, .
import einx
import lineax as lx
from jaxtyping import Array, Float
# Shapes: B = N_band bands; R × C = N_row × N_col pixels
def matched_filter(
radiance: Float[Array, "band y x"],
unit_absorption: Float[Array, " band"],
air_mass: float,
*,
mean: Float[Array, " band"],
covariance: lx.AbstractLinearOperator,
) -> Float[Array, "1 y x"]:
"""Methane enhancement of every pixel by a matched filter, background given.
Model: x_n = μ + y_n t + ε_n, ε_n ~ 𝒩(0, Σ), with θ = (μ, Σ) known. The
estimate is the generalised least-squares fit of y_n, which is also its
maximum-likelihood estimate:
y_n = argmin_y (x_n − μ − y t)ᵀ Σ⁻¹ (x_n − μ − y t)
= tᵀ Σ⁻¹ (x_n − μ) / (tᵀ Σ⁻¹ t), t = −M (k ⊙ μ)
Algorithm:
1. t = −M (k ⊙ μ): the linearised Beer–Lambert response to 1 ppm m.
2. s = Σ⁻¹ t, by a solve; a function repeats it on every call.
3. y_n = sᵀ (x_n − μ) / (tᵀ s) at every pixel.
Args:
radiance: x, the scene's spectra.
unit_absorption: k, methane absorption per ppm m at each band; part of u.
air_mass: M, the geometric air-mass factor; part of u.
mean: μ, the background mean; part of θ.
covariance: Σ, the background covariance as a PSD operator; part of θ.
Returns:
y in ppm m, with a singleton band axis; its standard error is
(tᵀ Σ⁻¹ t)^(−1/2) at every pixel.
"""
# k ⊙ μ
absorbed = unit_absorption * mean # (B,), (B,) → (B,)
# t = −M (k ⊙ μ): the radiance change for 1 ppm m of methane
target = -air_mass * absorbed # (B,) → (B,)
# s = Σ⁻¹ t
solution = lx.linear_solve(covariance, target) # (B, B) PSD op, (B,)
filter_direction = solution.value # → (B,)
# tᵀ s
normalisation = einx.dot("band, band ->", target, filter_direction) # → ()
# r_n = x_n − μ
residual = einx.subtract( # (B, R, C), (B,) → (B, R, C)
"band y x, band -> band y x", radiance, mean
)
# sᵀ r_n
projection = einx.dot( # (B, R, C), (B,) → (R, C)
"band y x, band -> y x", residual, filter_direction
)
# y_n = sᵀ r_n / (tᵀ s)
enhancement = projection / normalisation # (R, C) → (R, C)
# a singleton band axis
return einx.id("y x -> 1 y x", enhancement) # (R, C) → (1, R, C)
A function with state. A setup step builds the state once, from data and what it needs to know about them, , and may do work that cannot be traced (a scikit-learn estimator, a k-NN graph, a factorisation). The call is then pure and jittable:
Example The matched filter as it is used: the plume covers a small fraction of the pixels, so the scene is its own background. Setup estimates and a shrunk from (Ledoit–Wolf, in scikit-learn), forms , and solves once for the filter direction ; the state is . Each call is then two contractions:
What each method solves. Both rest on one model of a pixel's spectrum, the background plus ppm m of methane along :
fit solves for the background and the filter, once per scene and off the trace: a covariance shrunk towards a scaled identity by the amount that minimises its expected error, and the direction that maximises the signal-to-clutter ratio,
The call solves for each pixel's enhancement with that state fixed. It is the generalised least-squares fit of the model, which is also the maximum-likelihood estimate under the Gaussian background:
The docstrings below carry the same split: each method states its objective, its steps and its arguments.
import einx
import equinox as eqx
import jax.numpy as jnp
import numpy as np
import scipy.linalg
import sklearn.covariance
from jaxtyping import Array, Float
# Shapes: B = N_band bands; R × C = N_row × N_col pixels; P = N_valid valid pixels
class MatchedFilter(eqx.Module):
"""Matched filter whose background is fitted once, from the scene's own pixels.
Model: a pixel's spectrum is the background plus y ppm m of methane along t,
x_n = μ + y_n t + ε_n, ε_n ~ 𝒩(0, Σ)
`fit` estimates the background and the filter that model implies; the call
estimates y_n at every pixel with them. The state is ψ = (μ, s, tᵀ s).
"""
mean: Float[Array, " band"]
filter_direction: Float[Array, " band"]
normalisation: Float[Array, ""]
@classmethod
def fit(
cls,
spectra: Float[np.ndarray, "pixel band"],
unit_absorption: Float[np.ndarray, " band"],
air_mass: float,
) -> "MatchedFilter":
"""Fit the background and the filter direction, once per scene.
Two objectives, solved in turn. The background covariance is shrunk
towards a scaled identity by the amount that minimises its expected
squared error,
Σ̂ = (1 − ρ) S + ρ ν I, ρ = argmin_ρ 𝔼 ‖Σ̂ − Σ‖²_F
and the filter direction maximises the signal-to-clutter ratio,
s = argmax_s (sᵀ t)² / (sᵀ Σ̂ s) ∝ Σ̂⁻¹ t, t = −M (k ⊙ μ̂)
Algorithm:
1. μ̂, Σ̂: the sample mean and the Ledoit–Wolf shrunk covariance of X.
2. t = −M (k ⊙ μ̂): the radiance change for 1 ppm m of methane.
3. s = Σ̂⁻¹ t, one solve, and the normalisation tᵀ s.
Args:
spectra: X, the spectra of the valid pixels, one row each; the plume
is a small fraction of them, so the scene is its own background.
unit_absorption: k, methane absorption per ppm m at each band.
air_mass: M, the geometric air-mass factor.
Returns:
The filter with its state ψ = (μ̂, s, tᵀ s); S is the sample
covariance, ν = tr(S) / B and ρ ∈ [0, 1] the shrinkage.
"""
# --- setup: host-side NumPy, SciPy and scikit-learn, runs once ---
# μ̂, Σ̂ = (1 − ρ) S + ρ ν I
background = sklearn.covariance.LedoitWolf().fit(spectra) # (P, B) → fitted
mean = background.location_ # → (B,)
covariance = background.covariance_ # → (B, B)
# k ⊙ μ̂
absorbed = unit_absorption * mean # (B,), (B,) → (B,)
# t = −M (k ⊙ μ̂)
target = -air_mass * absorbed # (B,) → (B,)
# s = Σ̂⁻¹ t, solved once
filter_direction = scipy.linalg.solve( # (B, B), (B,) → (B,)
covariance, target, assume_a="pos"
)
# tᵀ s
normalisation = einx.dot("band, band ->", target, filter_direction) # → ()
# ψ = (μ̂, s, tᵀ s), as device arrays
return cls(
mean=jnp.asarray(mean),
filter_direction=jnp.asarray(filter_direction),
normalisation=jnp.asarray(normalisation),
)
def __call__(self, radiance: Float[Array, "band y x"]) -> Float[Array, "1 y x"]:
"""Estimate the methane enhancement at every pixel, with ψ fixed.
The generalised least-squares fit of the model to each pixel, which is
the maximum-likelihood estimate under the Gaussian background:
y_n = argmin_y (x_n − μ − y t)ᵀ Σ⁻¹ (x_n − μ − y t)
= sᵀ (x_n − μ) / (tᵀ s)
with standard error σ_y = (tᵀ s)^(−1/2), the same at every pixel.
Algorithm:
1. r_n = x_n − μ: the departure from the background.
2. sᵀ r_n: its projection on the filter direction.
3. y_n = sᵀ r_n / (tᵀ s): scaled to ppm m.
Args:
radiance: x, the scene's spectra, any number of pixels.
Returns:
y in ppm m, with a singleton band axis. Pure and jittable: no fit.
"""
# --- every call: two contractions, pure and jittable ---
# r_n = x_n − μ
residual = einx.subtract( # (B, R, C), (B,) → (B, R, C)
"band y x, band -> band y x", radiance, self.mean
)
# sᵀ r_n
projection = einx.dot( # (B, R, C), (B,) → (R, C)
"band y x, band -> y x", residual, self.filter_direction
)
# y_n = sᵀ r_n / (tᵀ s)
enhancement = projection / self.normalisation # (R, C) → (R, C)
# a singleton band axis
return einx.id("y x -> 1 y x", enhancement) # (R, C) → (1, R, C)
A pipeline step that lifts onto carriers. The Operator is built from the function and its parameters; it is called on a carrier with its config , a JSON record (which bands, which dimension, the fill policy). It unwraps the carrier with , calls on plain arrays, and rewraps with :
Read the first line from the inside out: unwraps, computes, rewraps. The unwrap hands down the array and what needs to know about it, , with every name in already resolved against the carrier's names and coordinates. passes through untouched; never reaches . The rewrap builds the output carrier:
Operators compose: is what pipekit writes step1 | step2, each step holding its own and .
Example The settings are the band dim and a spectral window, . resolves them against the carrier: it keeps the bands whose wavelengths in fall in the window, computes from the angles in and those wavelengths, and stacks from the pixels where . The Module's setup runs on , its call on the window. returns a carrier on the same grid, , with one band named ch4 and NaN where . geoops ships this step as geoops.matched_filter.MatchedFilter, which fits its background on the first call.
import einx
import numpy as np
import pipekit
from geoarray import GeoArray
from geoops.carrier import wrap_like
from radtrax.atmosphere import air_mass_factor
# Shapes: B = N_band bands; W window bands; R × C = N_row × N_col pixels; P valid
# MatchedFilter is the Module above; ch4_unit_absorption is proposed for radtrax
class MethaneEnhancement(pipekit_ops.Operator):
"""CH₄ enhancement on a carrier: the MatchedFilter Module lifted onto GeoArrays.
h[f, ψ](x_geo; α) = ι_α(f(π_α(x_geo); ψ), x_geo)
α = (dim, window_nm) says what to read; π_α resolves it against the
carrier's coordinates C and attrs 𝓜, f is the Module, ι_α rewraps.
"""
def __init__(
self, *, dim: str = "band", window_nm: tuple[float, float] = (2122.0, 2488.0)
) -> None:
"""Hold α: names and settings only, so the config round-trips through JSON.
Args:
dim: the name of the band dimension.
window_nm: the spectral window, nm, where methane absorbs strongly.
"""
self.dim = dim
self.window_nm = window_nm
def _apply(self, scene: GeoArray["band y x"]) -> GeoArray["1 y x"]:
"""Unwrap, fit the filter, filter, rewrap.
π_α(x_geo) = (x_W, u, X), u = (M, k)
ψ = setup(X, u), y = f(x_W; ψ)
y_geo = ι_α(y, x_geo), Γ_y = Γ
Algorithm:
1. x_W: the bands whose centre wavelengths fall inside the window.
2. u = (M, k): M from the sun and view angles in 𝓜, k at the
window's wavelengths.
3. X: the spectra of the pixels where the valid mask m is 1.
4. Fit the Module on X and u, then call it on x_W.
5. Rewrap on the same grid: one band named ch4, NaN where m = 0.
Args:
scene: x_geo, the radiance, with band wavelengths in C and
solar_zenith_deg, view_zenith_deg in 𝓜.
Returns:
y_geo, the enhancement in ppm m on the scene's grid.
"""
# --- π_α: the names in α, resolved against C and M ---
# today (geoarray)
# the band coordinate: names and centre wavelengths
bands = scene.coord(self.dim)
lower, upper = self.window_nm
# the bands whose wavelengths fall inside the window
window = [
name
for name, wavelength in zip(bands.names, bands.wavelength_nm)
if lower <= wavelength <= upper
]
# today (geoarray)
swir = scene.sel({self.dim: window}) # (B, R, C) → (W, R, C)
# today (radtrax)
# M = 1 / cos ϑ_sun + 1 / cos ϑ_view, from the angles in the attrs 𝓜
air_mass = air_mass_factor( # → ()
swir.attrs["solar_zenith_deg"], swir.attrs["view_zenith_deg"]
)
# today (geoarray)
wavelengths = swir.coord(self.dim).wavelength_nm # → W labels
# proposed (radtrax)
# k: methane absorption per ppm m at each band's wavelength
unit_absorption = ch4_unit_absorption(wavelengths) # → (W,)
# today (geoarray)
# m: valid where every band is finite and differs from the fill
valid = swir.valid_mask() # (W, R, C) → (R, C)
# X: the spectra of the valid pixels, one row each
swir_values = np.asarray(swir) # (W, R, C) → (W, R, C)
pixels = einx.id("band y x -> (y x) band", swir_values) # → (R C, W)
is_valid = einx.id("y x -> (y x)", valid) # (R, C) → (R C,)
spectra = pixels[is_valid] # (R C, W) → (P, W)
# --- the Module: setup once, then a pure call on plain arrays ---
# user
# ψ = setup(X, u)
matched_filter = MatchedFilter.fit(spectra, unit_absorption, air_mass)
# user
# y = f(x; ψ)
enhancement = matched_filter(swir.data) # (W, R, C) → (1, R, C)
# --- ι_α: same grid, one band in ppm m, NaN where m = 0 ---
# y_n = NaN where m_n = 0
masked = np.where(valid, enhancement, np.nan) # (1, R, C) → (1, R, C)
# today (geoops)
return wrap_like(swir, masked, band_names=("ch4",), fill_value=np.nan)
The sketches follow the house style: shapes as jaxtyping types, the equation above each line, einx for reshapes and contractions. All six run as written (checked on 6 Nov 2026): the data-axis examples against geotoolz-array and pipekit-array, and the function, the Module and the Operator on a synthetic scene with lineax and scikit-learn. In the Operator, air_mass_factor is radtrax's (radtrax.atmosphere) and ch4_unit_absorption stands for a lookup table of methane absorption (radtrax, or plumax's hapi_lut); geoops ships the matching Operators, matched_filter.MatchedFilter, plume.PlumeMask, plume.ColumnToMass and plume.IMEEstimate.
The shape of an Operator, down, compute, up, is not new: neural operators and lifted reduced models use it too. Open for how they line up, and where they differ.
Operator learning. A neural operator learns a map between function spaces, from an input field to an output field , with the coordinates of the domain. Every architecture in the family factors into an encoder, an approximator and a reconstructor (Lanthaler, Mishra and Karniadakis, 2022; Kovachki, Lanthaler and Stuart, 2024):
DeepONet's sensor values are , its branch net and its trunk net the basis of (Lu et al., 2021). The Fourier neural operator raises the channels with a map , applies its kernel layers, and lowers them with (Li et al., 2021; Kovachki et al., 2023). A carrier is a discretised field, and its georeference says which discretisation, so plays , plays and plays . Discretisation invariance, one operator for any grid, is the counterpart of an Operator that runs on any carrier whose names resolve.
Lift & Learn and Koopman. These change the state's variables to ones in which the dynamics are simpler, learn the simple model there, and map back. Take the state as a vector . Lift & Learn (Qian, Kramer, Peherstorfer and Willcox, 2020) picks a lifting in which the dynamics are quadratic, and fits the two operators by least squares from snapshots, which is operator inference (Peherstorfer and Willcox, 2016):
Koopman's view (Koopman, 1931) lifts into a dictionary of observables in which the dynamics are linear; extended dynamic mode decomposition fits the finite Koopman matrix from snapshot pairs (Williams, Kevrekidis and Rowley, 2015):
What carries over, and what does not.
Four steps in one recipe, each a lifted function, in three stages: retrieval (the first line), segmentation (the second) and quantification (the last two).
Here is the methane enhancement in ppm m and the plume mask, both on the scene's grid; is ERA5's 10 m wind speed on its own 0.25° grid, read at the source's coordinates to give the wind in m s⁻¹; and is the emission rate in kg s⁻¹. is the Module above, with fitted on the scene; thresholds and keeps connected components; is the integrated mass enhancement method:
with the mask at pixel , its area and the plume length, both from , and the mass of methane per square metre in one ppm m. The last step takes as its parameter , and it is terminal: returns a table ( with its uncertainty), not a carrier. Only the four configs mention band names, windows, units, thresholds or the source's coordinates. In geoops the steps are MatchedFilter, PlumeMask, interp_at on the wind, and ColumnToMass followed by IMEEstimate.
Each line here owns everything it needs. The same job, refactored into pieces that are written once and reused, is worked through after the role ladder: the running example, up the ladder.
| Symbol | Is | Lives in |
|---|---|---|
| , | plain array; function | Core GeoNumerics and up |
| , | Module: setup runs once; and are its fields | Core GeoNumerics and up |
| , | named array; operations by name | Core (pipekit-array) and up |
| , , | carrier: CRS, transform, fill and attributes; valid mask | GeoStack (geoarray; xarray on the states side) |
| , , | unwrap, rewrap, Operator on carriers; is its config | GeoStack; GeoModels in a geo extra |
| geometry as plain arrays, handed down | computed by the GeoStack; consumed by functax, gridax, geophysix |
The three invariants, as equations. Arrays go everywhere: every rung takes . Carriers stop at the GeoStack: nothing below sees , only . Configs stop at Operators: is consumed by and and never reaches , which takes only , and .
Code in the stack is packaged as one of three kinds: a function, a Module or an Operator. Each kind can work on one of three kinds of data: plain arrays, named arrays, or carriers. These are two separate axes. geoarray is not a fourth kind; it sits on the data axis. Names in italics are proposed.
A function. No state, no setup: inputs in, outputs out.
to_pixel, cell_areaAn eqx.Module with state. The expensive, untraceable setup runs once at construction or fit (a k-NN graph, a Delaunay triangulation, a factorisation); the call is then pure and jittable.
AffineGrid, BilinearPlan, Matern52. *Operator only for a linear map or an observation operator HA pipekit_ops.Operator: composes with |, runs in a Graph, round-trips its config through JSON. Any Module can become one by wrapping.
Reproject, PlumeMask. *Op only for an Operator that wraps a Module: JaxModelOp, SomaxModelOpA NumPy or JAX array. Shapes live in jaxtyping annotations; geometry arrives as separate arguments.
Float[Array, "y x"], a transform passed alongside itAn array whose dims have names and coordinates, but no georeference: no CRS, no fill semantics.
DimArray; an xarray.DataArray used as a labelled arrayA named array plus its georeference: a grid with a CRS, a fill value, attrs; or a lazy reader of one.
GeoArray, GeoSwath, GeoPoints, RasterSource; a DataArray in xrops, where its coords and CRS mean geography▦ A plain array knows its shape.
▤ A named array knows which axis is which.
◆ A carrier also knows where each pixel is (), and which pixels are missing (, ).
So a carrier is a named array plus its georeference. The named array still earns its place: it is the protocol that lets one Operator (Reduce(dim="time")) run on DimArray, DataArray and GeoArray through their adapters, and it is the end product for data with dims but no geography: a DA ensemble (member, time, state), a sweep, a posterior summary.
"Kicking a Module up a notch" is two separate steps. Wrapping it in an Operator adds composition: a config, |, graph mode. Moving it to carriers adds georeference: the CRS, the fill, the attrs. The steps are independent. JaxModelOp and a DA cycle take the first step without the second; geoarray's crop and read_window take the second without the first. geoops Operators take both.
gaussx draws its API as a ladder: primitives, operators, strategies, distributions, recipes. The same ladder fits every library in the stack, and it sits across the three kinds rather than beside them. Read it as grammar, from ⓪ down: structures are the nouns (Kronecker, and the data itself: arrays, named arrays, carriers), primitives the verbs that act on them (solve), strategies the adverbs that say how (CGSolver), models the noun phrases (a Gaussian with this mean), algorithms the sentences (kalman_filter), steps (pipekit Operators) a sentence with its own settings, ready to chain (MatchedFilter), and recipes the paragraphs: steps composed into one job, from source to product. Above the ladder, orchestration runs recipes; it is not a rung. Module is not a rung; it is how three rungs are packaged: ⓪ structures, ② strategies and ③ models. Its setup step , which builds state from data outside the jitted path, is what gaussx draws as the sidebar of tools that produce operators. Rungs are marked with circled digits, ⓪ to ⑥, so they cannot be confused with tiers (T) or processing levels (L); names in italics are proposed.
▦ Array▤ DimArray▤ DataArray◆ GeoArray◆ GeoSwath◆ GeoPointsKroneckerBlockDiagLowRankUpdateToeplitzRBFAffineGridLegendresolvelogdetcholeskydiagtraceto_pixelcropAutoSolverCGSolverBBMMSolverNystromPreconditionerUnscentedIntegratordiffrax.Tsit5MultivariateNormalLGSSMGaussianExpFamBilinearPlansesmax modelskalman_filtersparse_conditionaletkf_transforminlaMatchedFilterReprojectRGBRecipeJaxModelOpSequentialGraphBranchRetryDACycleTrainingLoopSpatialPatcherpipekit_ops.dumpspipekit_ops.loadsloads_sandboxedrecipe.yamljit: a model's state (③), and structures (⓪) such as a low-rank factorrandomized_svdrandomized_nystromrp_choleskySRHTSketchk-NN graphDelaunayThe four lifted functions of the running example are right, but each step owns everything it needs, so nothing is shared. Rebuilt the way a library is built, the same job climbs the ladder: the data, then primitives, the strategies and models that use them, the algorithms over the models, one step per stage, and a recipe that strings the steps together, which orchestration then runs. Each piece is written once, and most are used by more than one piece above it.
| Piece | Rung | Kind | Used by |
|---|---|---|---|
| GeoArray, SpectralAxis, a PSD operator | ⓪ structure | ▣ / ◆ | every rung above: the scene and the wind are carriers, is a lineax operator |
| target_signature | ① primitive | ƒ | the retrieval step, and through the background's noise |
| pixel_area | ① primitive | ƒ | emission_rate, twice: the plume's mass and its length |
| large_components | ① primitive | ƒ | segment |
| air_mass_factor, valid_mask, interp_at | ① primitive (radtrax, geoarray) | ƒ | the retrieval step and the wind step; any SWIR retrieval, any sampling |
| LedoitWolf, SigmaThreshold | ② strategy | ▣ | Background.fit and segment; swap in another estimator or rule |
| Background | ③ model | ▣ | matched_filter (its solve) and the noise , which sets the segmentation threshold |
| matched_filter, segment, emission_rate | ④ algorithm | ƒ | one step each |
| MethaneRetrieval, PlumeSegmentation, WindAt, EmissionRate | ⑤ step | → | the recipe; each also runs alone, in a notebook or another recipe |
| plume_recipe | ⑥ recipe | → Graph | orchestration: a DVC stage, a GeoApps endpoint, a sweep over |
The scene and the wind are carriers, each on its own grid: the radiance on 60 m in UTM zone 13N with the sun and view angles in its attributes , the wind on ERA5's 0.25° grid in longitude and latitude. Nothing in the rungs above builds a grid or reads a file.
import numpy as np
from geoarray import GeoArray
from geoarray.coords import SpectralAxis
# Shapes: B = 285 bands; R × C = 256 × 256 pixels; G × H = 8 × 8 wind cells
# x: radiance from the orthorectified EMIT file
radiance = np.zeros((285, 256, 256), dtype=np.float32) # → (B, R, C)
# v: ERA5 10 m wind speed around the field, one band on a 0.25° grid
wind_speed = np.full((1, 8, 8), 3.5) # → (1, G, H)
# the centre wavelength of each band, nm, and a name for each
wavelengths = np.linspace(381.0, 2493.0, 285) # → (B,)
rounded = np.round(wavelengths, 1) # (B,) → (B,)
WAVELENGTHS_NM = tuple(rounded.tolist()) # (B,) → B labels
BAND_NAMES = tuple(f"{wavelength:.1f}nm" for wavelength in WAVELENGTHS_NM)
# today (geoarray)
# C: the band names with their centre wavelengths
bands = SpectralAxis(names=BAND_NAMES, wavelength_nm=WAVELENGTHS_NM)
# today (geoarray)
# x_geo: a 60 m grid in UTM zone 13N, with the sun and view angles in 𝓜
scene: GeoArray["band y x"] = GeoArray( # (B, R, C) → (B, R, C)
radiance,
(60.0, 0.0, 630000.0, 0.0, -60.0, 3550000.0),
"EPSG:32613",
fill_value=-9999.0,
dims="band y x",
coords=(bands,),
attrs={"solar_zenith_deg": 28.4, "view_zenith_deg": 6.1},
)
# today (geoarray)
# v_geo: the wind on its own grid, in longitude and latitude
wind_field: GeoArray["band y x"] = GeoArray( # (1, G, H) → (1, G, H)
wind_speed, (0.25, 0.0, -105.0, 0.0, -0.25, 33.0), "EPSG:4326", dims="band y x"
)
Functions on arrays that know nothing of methane plumes: the target signature, the area of a pixel of an affine grid, and the large connected components of a mask.
The third primitive keeps only the large connected pieces of a candidate mask , with the 4-connected component of that holds pixel :
import einx
import numpy as np
import scipy.ndimage
from jaxtyping import Array, Bool, Float
# Shapes: W = window bands; R × C = N_row × N_col pixels; K labelled components
def target_signature(
mean: Float[Array, " band"], unit_absorption: Float[Array, " band"], air_mass: float
) -> Float[Array, " band"]:
"""The radiance change for 1 ppm m of methane: the matched filter's target.
Linearising the Beer–Lambert law around the background,
x_b ≈ μ_b exp(−M k_b y) ≈ μ_b (1 − M k_b y), so ∂x/∂y at y = 0 is
t = −M (k ⊙ μ)
Args:
mean: μ, the background mean spectrum.
unit_absorption: k, absorption per ppm m along a vertical path.
air_mass: M, the geometric air-mass factor.
Returns:
t, in radiance units per ppm m; negative where methane absorbs.
"""
# k ⊙ μ
absorbed = unit_absorption * mean # (W,), (W,) → (W,)
# t = −M (k ⊙ μ)
target = -air_mass * absorbed # (W,) → (W,)
return target
def pixel_area(transform: Float[np.ndarray, " 6"]) -> float:
"""The area of every pixel of an affine grid.
a = |det A₂ₓ₂| = |a₀ a₄ − a₁ a₃|
for A = (a₀, a₁, a₂, a₃, a₄, a₅), the transform in (a, b, c, d, e, f) order.
Args:
transform: A, six numbers, in map units per pixel.
Returns:
a, in squared map units (m² on a projected grid), the same at every pixel.
"""
# det A₂ₓ₂ = a₀ a₄ − a₁ a₃
determinant = transform[0] * transform[4] - transform[1] * transform[3] # → ()
# a = |det A₂ₓ₂|
return abs(determinant)
def large_components(
mask: Bool[np.ndarray, "y x"], min_pixels: int
) -> Bool[np.ndarray, "y x"]:
"""The connected components of a mask that have at least N_min pixels.
b_n = c_n 1[|K(n)| ≥ N_min]
with K(n) the 4-connected component of c that holds pixel n.
Algorithm:
1. Label the components of c 1, …, K, and 0 off the mask.
2. Count the pixels of each label.
3. Keep the labels with at least N_min pixels; never label 0.
Args:
mask: c, the candidate pixels, above a threshold.
min_pixels: N_min.
Returns:
b, the mask with its small components removed.
"""
# the components of c, labelled 1, …, K; 0 off the mask
labels, count = scipy.ndimage.label(mask) # (R, C) → (R, C), K
# |component k|, in pixels
flat_labels = einx.id("y x -> (y x)", labels) # (R, C) → (R C,)
sizes = np.bincount(flat_labels, minlength=count + 1) # (R C,) → (K + 1,)
# 1[|component k| ≥ N_min], never the background, label 0
keep = sizes >= min_pixels # (K + 1,) → (K + 1,)
keep[0] = False
# b_n = c_n 1[|K(n)| ≥ N_min]
return keep[labels] # (K + 1,), (R, C) → (R, C)
The strategies say how: LedoitWolf fits a mean and a shrunk covariance, SigmaThreshold keeps the pixels above noise levels. The model holds what was fitted: Background keeps and , solves with , and gives the matched filter's standard error, which the segmentation threshold uses:
The threshold is a one-sided test of against . Under the Gaussian background is standard normal where there is no methane, so each pixel is a false alarm with probability
with the standard normal CDF: 0.13 % at , about 85 false pixels in a 256 × 256 scene, which the filter then removes. The candidates become the plume mask there.
This is where the kind axis's MatchedFilter Module splits: its state becomes the model Background, and its call becomes the algorithm matched_filter below. The split is what lets segmentation reuse .
import einx
import equinox as eqx
import jax.numpy as jnp
import lineax as lx
import numpy as np
import sklearn.covariance
from jaxtyping import Array, Bool, Float
# Shapes: W = window bands; P = valid pixels; R × C = N_row × N_col pixels
# --- ② strategies: how a model is fitted, or a rule applied ---
class LedoitWolf(eqx.Module):
"""Background mean and a shrunk covariance; swap for another estimator."""
def __call__(self, spectra: Float[np.ndarray, "pixel band"]) -> tuple:
"""Background mean and a shrunk covariance from the valid pixels' spectra.
μ̂ = (1 / P) ∑_n x_n
Σ̂ = (1 − ρ) S + ρ ν I, ρ = argmin_ρ 𝔼 ‖Σ̂ − Σ‖²_F
with S the sample covariance and ν = tr(S) / W. Shrinking keeps Σ̂ well
conditioned when W bands are estimated from few pixels.
Args:
spectra: X, one row per valid pixel.
Returns:
(μ̂, Σ̂).
"""
# μ, Σ by Ledoit–Wolf shrinkage
estimate = sklearn.covariance.LedoitWolf().fit(spectra) # (P, W) → fitted
return estimate.location_, estimate.covariance_ # → (W,), (W, W)
class SigmaThreshold(eqx.Module):
"""Keep pixels above k noise levels; swap for Otsu or a percentile."""
k: float = 3.0
def __call__(
self, enhancement: Float[Array, "y x"], noise: float
) -> Bool[Array, "y x"]:
"""Pixels whose enhancement exceeds k standard errors.
c_n = 1[y_n > k σ_y]
A one-sided test of y_n = 0 against y_n > 0: under the Gaussian
background a pixel is a false alarm with probability 1 − Φ(k), 0.13 %
at k = 3, with Φ the standard normal CDF.
Args:
enhancement: y, ppm m.
noise: σ_y, the matched filter's standard error, ppm m.
Returns:
c, the candidate pixels, before small components are removed.
"""
# k σ_y
level = self.k * noise # → ()
# c_n = 1[y_n > k σ_y]
return enhancement > level # (R, C) → (R, C)
# --- ③ models: state fitted once ---
class Background(eqx.Module):
"""The scene's background: x_n = μ + ε_n, ε_n ~ 𝒩(0, Σ), off the plume.
`fit` estimates μ and Σ once; `solve` and `noise` use them, pure and
jittable, for every pixel and every target t.
"""
mean: Float[Array, " band"]
covariance: lx.MatrixLinearOperator
@classmethod
def fit(
cls, spectra: Float[np.ndarray, "pixel band"], estimator=LedoitWolf()
) -> "Background":
"""Estimate μ and Σ from the valid pixels' spectra, once per scene.
(μ̂, Σ̂) = estimator(X)
With LedoitWolf, Σ̂ = (1 − ρ) S + ρ ν I, shrunk by the ρ that minimises
𝔼 ‖Σ̂ − Σ‖²_F. Σ̂ is kept as a PSD-tagged operator, a ⓪ structure.
Args:
spectra: X, one row per valid pixel.
estimator: the strategy that gives (μ̂, Σ̂).
Returns:
The fitted background.
"""
# user
# μ, Σ from the strategy
mean, covariance = estimator(spectra) # (P, W) → (W,), (W, W)
# Σ as a ⓪ structure: a PSD-tagged operator, so solves take the fast path
operator = lx.MatrixLinearOperator( # (W, W) → (W, W) PSD op
jnp.asarray(covariance), lx.positive_semidefinite_tag
)
return cls(mean=jnp.asarray(mean), covariance=operator)
def solve(self, vector: Float[Array, " band"]) -> Float[Array, " band"]:
"""Σ⁻¹ v, by a solve with the PSD operator, never an inverse.
Args:
vector: v.
Returns:
Σ⁻¹ v.
"""
# Σ⁻¹ v
solution = lx.linear_solve(self.covariance, vector) # (W, W) op, (W,)
return solution.value # → (W,)
def noise(self, target: Float[Array, " band"]) -> Float[Array, ""]:
"""The matched filter's standard error for target t.
σ_y² = Var(sᵀ ε) / (tᵀ s)² = sᵀ Σ s / (tᵀ s)² = 1 / (tᵀ Σ⁻¹ t)
with s = Σ⁻¹ t and ε ~ 𝒩(0, Σ) the background's departure.
Args:
target: t.
Returns:
σ_y, in ppm m.
"""
# Σ⁻¹ t
direction = self.solve(target) # (W,) → (W,)
# tᵀ Σ⁻¹ t
information = einx.dot("band, band ->", target, direction) # → ()
# σ_y = (tᵀ Σ⁻¹ t)^(−1/2), the matched filter's standard error
return information**-0.5 # () → ()
Functions over models: the matched filter, the segmentation, and the emission rate from the integrated mass enhancement, with the plume length taken as the square root of the plume's area:
The emission rate rests on a steady-state mass balance: the plume holds what the source emitted while the wind carried it across the plume's length, so the integrated mass enhancement is
and solving for gives the formula above. Taking as the square root of the plume's area is a convention; other lengths change by a factor near one.
import einx
import numpy as np
from jaxtyping import Array, Bool, Float
# Shapes: W = window bands; R × C = N_row × N_col pixels
def matched_filter(
radiance: Float[Array, "band y x"],
background: Background,
target: Float[Array, " band"],
) -> Float[Array, "y x"]:
"""Methane enhancement of every pixel, against a fitted background.
y_n = argmin_y (x_n − μ − y t)ᵀ Σ⁻¹ (x_n − μ − y t) = sᵀ (x_n − μ) / (tᵀ s)
with s = Σ⁻¹ t, from the model x_n = μ + y_n t + ε_n, ε_n ~ 𝒩(0, Σ).
Algorithm:
1. s = Σ⁻¹ t, through the model's solve.
2. r_n = x_n − μ and its projection sᵀ r_n.
3. y_n = sᵀ r_n / (tᵀ s).
Args:
radiance: x, the window's spectra.
background: the fitted model, μ and Σ.
target: t.
Returns:
y, in ppm m.
"""
# user
# s = Σ⁻¹ t, through the model
direction = background.solve(target) # (W,) → (W,)
# r_n = x_n − μ
residual = einx.subtract( # (W, R, C), (W,) → (W, R, C)
"band y x, band -> band y x", radiance, background.mean
)
# sᵀ r_n
projection = einx.dot("band y x, band -> y x", residual, direction) # → (R, C)
# tᵀ s
normalisation = einx.dot("band, band ->", target, direction) # → ()
# y_n = sᵀ r_n / (tᵀ s)
return projection / normalisation # (R, C) → (R, C)
def segment(
enhancement: Float[np.ndarray, "y x"],
noise: float,
rule: SigmaThreshold,
min_pixels: int,
) -> Bool[np.ndarray, "y x"]:
"""The plume mask: pixels above k σ_y, in components of at least N_min pixels.
c_n = 1[y_n > k σ_y], b_n = c_n 1[|K(n)| ≥ N_min]
Algorithm:
1. Threshold y with the rule, a strategy, so Otsu or a percentile can
take its place.
2. Remove the components smaller than N_min pixels, mostly noise.
Args:
enhancement: y, ppm m.
noise: σ_y, ppm m.
rule: the threshold strategy.
min_pixels: N_min.
Returns:
b, the plume mask.
"""
# user
# c_n = 1[y_n > k σ_y], by the strategy
above = rule(enhancement, noise) # (R, C) → (R, C)
above = np.asarray(above) # (R, C) → (R, C)
# user
# b_n = c_n 1[|K(n)| ≥ N_min]
return large_components(above, min_pixels) # (R, C) → (R, C)
def emission_rate(
enhancement: Float[np.ndarray, "y x"],
plume: Bool[np.ndarray, "y x"],
area: float,
wind: float,
kappa: float,
) -> float:
"""Emission rate from the integrated mass enhancement (Varon et al., 2018).
In steady state, the methane in the plume is the source rate times the
time the wind takes to carry it across the plume, IME = Q L / U, so
IME = κ a ∑_n b_n y_n, L = (a ∑_n b_n)^(1/2), Q = U IME / L
Algorithm:
1. IME: the plume's enhancement summed over its pixels, in kg.
2. L: the plume length, the square root of its area, in m.
3. Q = U IME / L, in kg s⁻¹; zero for an empty plume.
Args:
enhancement: y, ppm m.
plume: b, the plume mask.
area: a, the pixel area, m².
wind: U, the effective wind speed at the source, m s⁻¹.
kappa: κ, kg m⁻² per ppm m.
Returns:
Q, kg s⁻¹.
"""
# b_n y_n
plume_enhancement = np.where(plume, enhancement, 0.0) # (R, C) → (R, C)
# ∑ b_n y_n, ppm m
total = np.nansum(plume_enhancement) # (R, C) → ()
# IME = κ a ∑ b_n y_n, the mass of methane in the plume, kg
ime = kappa * area * total # → ()
# ∑ b_n, pixels
plume_pixels = np.sum(plume) # (R, C) → ()
# L = (a ∑ b_n)^(1/2), m
length = np.sqrt(area * plume_pixels) # → ()
if length == 0:
return 0.0
# Q = U IME / L, kg s⁻¹
rate = wind * ime / length # → ()
return float(rate)
One Operator per stage, each a , a call down the ladder and an . Retrieval writes into the enhancement's attributes, so segmentation reads it from its input. Quantification is two steps, because the wind lives on another grid: WindAt reads at the source's coordinates, and EmissionRate takes it as an input. In the four lifted functions was the parameter of ; here it is wired, as geotoolz's rule wants: a second input is an argument of _apply, never a constructor field, so a Graph can connect it.
import einx
import jax.numpy as jnp
import numpy as np
import pipekit
from geoarray import GeoArray, GeoPoints, interp_at
from geoops.carrier import wrap_like
from pipekit import Signature
from radtrax.atmosphere import air_mass_factor
# Shapes: B = 285 bands; W window bands; R × C pixels; P valid; G × H wind cells
# ch4_unit_absorption is proposed for radtrax: methane absorption per ppm m
class MethaneRetrieval(pipekit_ops.Operator):
"""Retrieval: radiance → methane enhancement, with its noise σ_y in the attrs.
y_geo = ι_α(matched_filter(π_α(x_geo); Background.fit(X)), x_geo)
"""
def __init__(self, *, window_nm: tuple[float, float] = (2122.0, 2488.0)) -> None:
"""Hold α, the spectral window in nm where methane absorbs strongly."""
self.window_nm = window_nm
def _apply(self, scene: GeoArray["band y x"]) -> GeoArray["1 y x"]:
"""Unwrap, fit the background, filter, rewrap.
π_α(x_geo) = (x_W, u, X), u = (M, k)
μ̂, Σ̂ = Background.fit(X), t = −M (k ⊙ μ̂)
y_n = sᵀ (x_n − μ̂) / (tᵀ s), s = Σ̂⁻¹ t, σ_y = (tᵀ s)^(−1/2)
Algorithm:
1. π_α: the window's bands, u from the attrs and the wavelengths,
and X from the valid pixels.
2. ③ fit the model, ① build t, ④ filter.
3. ι_α: one band in ppm m on the same grid, NaN where m = 0, and
σ_y in 𝓜 for the steps downstream.
Args:
scene: x_geo, the radiance, with band wavelengths in C and the
sun and view zenith angles in 𝓜.
Returns:
y_geo, the enhancement in ppm m, with noise_ppm_m = σ_y in 𝓜.
"""
# --- π_α: the window's bands, u = (M, k) and X ---
# today (geoarray)
bands = scene.coord("band")
lower, upper = self.window_nm
names = [
n for n, w in zip(bands.names, bands.wavelength_nm) if lower <= w <= upper
]
# today (geoarray)
swir = scene.sel(band=names) # (B, R, C) → (W, R, C)
# today (radtrax)
# M = 1 / cos ϑ_sun + 1 / cos ϑ_view, from the attrs 𝓜
air_mass = air_mass_factor( # → ()
swir.attrs["solar_zenith_deg"], swir.attrs["view_zenith_deg"]
)
# today (geoarray)
wavelengths = swir.coord("band").wavelength_nm # → W labels
# proposed (radtrax)
# k at the window's wavelengths
absorption = ch4_unit_absorption(wavelengths) # → (W,)
# today (geoarray)
# m: the valid pixels
valid = swir.valid_mask() # (W, R, C) → (R, C)
# X: the spectra of the valid pixels, one row each
swir_values = np.asarray(swir) # (W, R, C) → (W, R, C)
pixels = einx.id("band y x -> (y x) band", swir_values) # → (R C, W)
is_valid = einx.id("y x -> (y x)", valid) # (R, C) → (R C,)
spectra = pixels[is_valid] # (R C, W) → (P, W)
# --- ③ fit the model, ① build t, ④ filter ---
# user
background = Background.fit(spectra) # (P, W) → μ (W,), Σ (W, W)
# user
# t = −M (k ⊙ μ)
target = target_signature( # (W,), (W,) → (W,)
background.mean, jnp.asarray(absorption), air_mass
)
# user
# y_n = sᵀ (x_n − μ) / (tᵀ s)
enhancement = matched_filter(swir.data, background, target) # → (R, C)
# --- ι_α: one band in ppm m, NaN where m = 0, σ_y carried in 𝓜 ---
masked = np.where(valid, enhancement, np.nan) # (R, C) → (R, C)
# user
# σ_y = (tᵀ Σ⁻¹ t)^(−1/2)
noise = background.noise(target) # → ()
# today (geoops)
return wrap_like( # (R, C) → (1, R, C)
swir,
einx.id("y x -> 1 y x", masked),
band_names=("ch4",),
fill_value=np.nan,
attrs={"noise_ppm_m": float(noise)},
)
def compute_output_signature(self, sig: Signature) -> Signature:
"""The output's dims and dtype from the input's, without data."""
# today (pipekit)
# (band=B, y, x) → (band=1, y, x)
return sig.replace_dims({"band": 1})
class PlumeSegmentation(pipekit_ops.Operator):
"""Segmentation: enhancement → plume mask, at k times the retrieval's noise.
b_geo = ι_α(segment(y, σ_y; k, N_min), y_geo)
"""
def __init__(self, *, k: float = 3.0, min_pixels: int = 20) -> None:
"""Hold α: the threshold k in noise levels and the smallest plume N_min."""
self.k = k
self.min_pixels = min_pixels
def _apply(self, enhancement: GeoArray["1 y x"]) -> GeoArray["1 y x"]:
"""Unwrap, threshold, keep the large components, rewrap.
c_n = 1[y_n > k σ_y], b_n = c_n 1[|K(n)| ≥ N_min]
Algorithm:
1. π_α: the map y and σ_y from the retrieval's attrs.
2. ② the rule and ④ the algorithm give b.
3. ι_α: a boolean band on the same grid.
Args:
enhancement: y_geo, from MethaneRetrieval.
Returns:
b_geo, the plume mask.
"""
# --- π_α: the map, and σ_y from the retrieval's attrs ---
values = einx.id("1 y x -> y x", np.asarray(enhancement)) # → (R, C)
noise = enhancement.attrs["noise_ppm_m"] # → ()
# --- ② the rule, ④ the algorithm ---
rule = SigmaThreshold(k=self.k)
# user
# b_n = c_n 1[|K(n)| ≥ N_min], c_n = 1[y_n > k σ_y]
plume = segment(values, noise, rule, self.min_pixels) # (R, C) → (R, C)
# --- ι_α: a boolean band on the same grid ---
# today (geoops)
return wrap_like( # (R, C) → (1, R, C)
enhancement,
einx.id("y x -> 1 y x", plume),
band_names=("plume",),
fill_value=False,
)
def compute_output_signature(self, sig: Signature) -> Signature:
"""The same dims as the input, as booleans."""
# today (pipekit)
# same grid, one boolean band
return Signature(dims=sig.dims, dtype="bool")
class WindAt(pipekit_ops.Operator):
"""Quantification: the wind speed U at the source, read from its own grid.
U = v(s_source), bilinear in the wind's grid
"""
_terminal = True
def __init__(self, *, lon: float, lat: float) -> None:
"""Hold α: the source's longitude and latitude, degrees."""
self.lon = lon
self.lat = lat
def _apply(self, wind_field: GeoArray["band y x"]) -> float:
"""Read the wind at the source.
U = ∑_corners w_c v_c, w the bilinear weights of s_source
Args:
wind_field: v_geo, ERA5's 10 m wind speed on its own grid.
Returns:
U, m s⁻¹.
"""
# the source, in the wind's CRS
source_xy = np.array([[self.lon, self.lat]]) # → (1, 2)
# today (geoarray)
source = GeoPoints(np.zeros(1), xy=source_xy, dims="obs") # → (1,)
# today (geoarray)
# U = v(s_source), bilinear
sampled = interp_at(wind_field, source) # (1, G, H) → (1, 1)
values = np.asarray(sampled) # (1, 1) → (1, 1)
return float(values.item()) # (1, 1) → ()
def compute_output_signature(self, sig: Signature) -> Signature:
"""One number, whatever the field's shape."""
# today (pipekit)
# one number: U, in m s⁻¹
return Signature(dims=(), dtype="float64")
class EmissionRate(pipekit_ops.Operator):
"""Quantification: Q from the enhancement, the plume and the wind.
Q = emission_rate(y, b; a, U, κ), a = |det A₂ₓ₂|
"""
_terminal = True
def __init__(self, *, kappa: float = 6.56e-7) -> None:
"""Hold α: κ, kg m⁻² per ppm m, the near-surface value."""
self.kappa = kappa
def _apply(
self, enhancement: GeoArray["1 y x"], plume: GeoArray["1 y x"], wind: float
) -> dict:
"""Unwrap, compute Q, return a table: the recipe's last step.
Q = U κ a ∑_n b_n y_n / L, L = (a ∑_n b_n)^(1/2)
Args:
enhancement: y_geo, ppm m.
plume: b_geo, on the same grid.
wind: U, m s⁻¹, from WindAt.
Returns:
A table, {emission_rate_kg_s: Q}; terminal, so not a carrier.
"""
# --- π_α: u = a from A, and the two maps ---
# user
# a = |det A₂ₓ₂|
area = pixel_area(enhancement.transform) # (6,) → ()
values = einx.id("1 y x -> y x", np.asarray(enhancement)) # → (R, C)
mask = einx.id("1 y x -> y x", np.asarray(plume)) # → (R, C)
# --- ④ the algorithm; terminal, so ι_α returns a table, not a carrier ---
# user
# Q = U κ a ∑ b_n y_n / L
rate = emission_rate(values, mask, area, wind, self.kappa) # → ()
return {"emission_rate_kg_s": rate}
The recipe strings the steps into one job: retrieval, segmentation, quantification. The enhancement feeds both segmentation and quantification, and the wind enters on its own grid, so the recipe is a Graph rather than a Sequential:
In symbols, four lifted functions, with the wind now an input of the last one rather than its parameter:
plot_model: in and out are pipekit Signatures, the dims, sizes and dtype that recipe.summary propagates without data; α is the config dumps writes; ψ is the state fitted when the step runs. ψ plays the part of Keras's parameter count, with one difference: Keras's weights are trained once and saved, while the background is refitted on every scene and never enters the config. The dashed frame is the ⑥ recipe, a Graph because the enhancement feeds both segmentation and quantification and the wind enters on its own grid.Orchestration runs the recipe without changing it: built in Python, summarised without data, saved as JSON, loaded back, or swept over a setting. A DVC stage, an Airflow task and a GeoApps endpoint all do one of these.
import pipekit
def plume_recipe(*, lon: float, lat: float, k: float = 3.0) -> pipekit_ops.Graph:
"""Retrieval, segmentation and quantification, wired into one Graph.
y_geo = h[f_ret](x_geo; α₁), b_geo = h[f_seg](y_geo; α₂)
U = h[f_wind](v_geo; α₃), Q = h[f_ime](y_geo, b_geo, U; α₄)
Args:
lon: the source's longitude, degrees, for the wind.
lat: the source's latitude, degrees.
k: the segmentation threshold, in noise levels.
Returns:
A Graph with inputs scene and wind and the output emission. It holds
configs only, so pipekit_ops.dumps writes it and pipekit_ops.loads rebuilds it.
"""
# today (pipekit)
scene_in = pipekit_ops.Input("scene")
# today (pipekit)
wind_in = pipekit_ops.Input("wind")
# user
# retrieval: y_geo with σ_y
enhancement = MethaneRetrieval()(scene_in)
# user
# segmentation: b_geo
plume = PlumeSegmentation(k=k)(enhancement)
# user
# quantification: U, then Q
wind = WindAt(lon=lon, lat=lat)(wind_in)
# user
emission = EmissionRate()(enhancement, plume, wind)
# today (pipekit)
return pipekit_ops.Graph(
inputs={"scene": scene_in, "wind": wind_in}, outputs={"emission": emission}
)
# --- orchestration: build, summarise, save, load and run; the recipe never changes ---
# user
recipe = plume_recipe(lon=-103.52, lat=31.98)
# today (pipekit)
# the signatures of the data part's carriers, with no data read
scene_sig = pipekit_ops.signature_of(scene) # → (band=B, y=R, x=C)
# today (pipekit)
wind_sig = pipekit_ops.signature_of(wind_field) # → (band=1, y=G, x=H)
# today (pipekit)
# a Keras-style table, from signatures alone
print(recipe.summary({"scene": scene_sig, "wind": wind_sig}))
# today (pipekit)
# the recipe as data: JSON with the steps' configs, never the arrays
recipe_json = pipekit_ops.dumps(recipe)
# today (pipekit)
# loaded back, it runs on the carriers of the data part
loaded = pipekit_ops.loads(recipe_json)
result = loaded(scene=scene, wind=wind_field) # → {emission: table}
# user
# a sweep over the threshold, as a DVC stage or an Airflow task would run it
sweep = {
k: plume_recipe(lon=-103.52, lat=31.98, k=k)(scene=scene, wind=wind_field)
for k in (2.0, 3.0, 4.0)
}
recipe.summary prints the same signatures as a table, Keras-style, from signatures alone. A multi-input node shows ?: pipekit's compute_output_signature takes one input, so EmissionRate's output is not inferred.
Node Input sig Output sig -------------------------------------------------------------------------------------------------------------------- [0] MethaneRetrieval(scene) (band=285, y=256, x=256) float32 (band=1, y=256, x=256) float32 [1] PlumeSegmentation([0]) (band=1, y=256, x=256) float32 (band=1, y=256, x=256) bool [2] WindAt(wind) (band=1, y=8, x=8) float64 () float64 [3] EmissionRate([0], [1], [2]) (band=1, y=256, x=256) float32, (band=1, y=256, x=256) bool, () float64 ? → emission = [3] ?
The recipe as data, read for people: the steps by name, each with what it takes and its config. This is the record pipekit_ops.dumps writes, with the node numbers replaced by step names; pipekit itself writes and reads the JSON below it.
inputs: [scene, wind]
steps:
methane_retrieval:
op: MethaneRetrieval
in: [scene]
window_nm: [2122.0, 2488.0]
plume_segmentation:
op: PlumeSegmentation
in: [methane_retrieval]
k: 3.0
min_pixels: 20
wind_at:
op: WindAt
in: [wind]
lat: 31.98
lon: -103.52
emission_rate:
op: EmissionRate
in: [methane_retrieval, plume_segmentation, wind_at]
kappa: 6.56e-07
outputs:
emission: emission_rate
pipekit_ops.dumps writes it{
"class": "Graph",
"config": {
"inputs": ["scene", "wind"],
"nodes": [
{
"operator": 0,
"parents": [
{ "input": "scene" }
]
},
{
"operator": 1,
"parents": [
{ "node": 0 }
]
},
{
"operator": 2,
"parents": [
{ "input": "wind" }
]
},
{
"operator": 3,
"parents": [
{ "node": 0 },
{ "node": 1 },
{ "node": 2 }
]
}
],
"operators": [
{
"class": "MethaneRetrieval",
"config": {
"window_nm": [2122.0, 2488.0]
},
"module": "plume"
},
{
"class": "PlumeSegmentation",
"config": {
"k": 3.0,
"min_pixels": 20
},
"module": "plume"
},
{
"class": "WindAt",
"config": {
"lat": 31.98,
"lon": -103.52
},
"module": "plume"
},
{
"class": "EmissionRate",
"config": { "kappa": 6.56e-07 },
"module": "plume"
}
],
"outputs": {
"emission": { "node": 3 }
}
},
"module": "pipekit_ops._base.graph"
}The six parts run as written, in order, as one module (checked on 7 Nov 2026), on a synthetic 256 × 256 scene, the size the summary shows, with a 3000 ppm m disk and a uniform 3.5 m s⁻¹ wind, with stand-ins for radtrax's air_mass_factor and the absorption lookup. The recipe returns = 4.5 kg s⁻¹ against 4.6 kg s⁻¹ from the true plume, and the Graph loaded back from JSON returns the same. = 6.56 × 10⁻⁷ kg m⁻² per ppm m is the near-surface value of geoops' ColumnToMass.
Why structures sit below primitives. In gaussx, _primitives imports _operators 76 times and _operators imports _primitives once: the verbs exist to dispatch on the nouns. With the nouns at the floor, the ladder is also the build order. One entanglement is left: a primitive takes a strategy argument (solve(A, b, solver=CGSolver())) and some strategies call primitives (diag, trace); _primitives and _strategies import each other eight times each way. That pair is gaussx's dispatch core, now its only documented exception.
The data are nouns too. ▦ arrays, ▤ named arrays and ◆ carriers are the ⓪ structures of the data axis: DimArray is pipekit-array's noun, GeoArray, GeoSwath and GeoPoints are geoarray's, and geoarray's own verbs (crop, to_pixel, read_window) are the ① primitives that act on them. The kind and data axes say how a noun is packaged and what it knows; the ladder says it is a noun.
Steps and recipes. The test between ⑤ and ⑥: does it do one thing to its input, or does it put steps (its own, or ones you pass in) in order to make a job? MatchedFilter and Reproject are steps; load, orthorectify, matched filter, plume mask and emission estimate, from an EMIT L1B scene to an emission rate, is a recipe. What counts is how it is built, not its name or what it does inside: Reproject resamples and warps internally but is one closed class with settings, so it is a step, and so is geoops.viz.RGBRecipe despite its name (RGBComposite would say it better). Some recipes ship in libraries as templates, compositions with slots for your steps: DACycle (forecast then analysis, cycled), TrainingLoop, and the patcher's split, process and stitch. The combinators (Sequential, Graph, Branch, Retry) are the conjunctions that join steps, so they sit at ⑥ too.
Composition is closed; running is not. A recipe of recipes is a recipe, as a Sequential of Sequentials is a Sequential, so the ladder stops at ⑥. What sits above is not more composition but running it: which items, when, where, with what retries and storage. That is orchestration, the application's job (a DVC stage, an Airflow or Prefect flow, a FastAPI endpoint, GeoApps), and the same recipe should run unchanged in all of them. A recipe has two forms, code (a Sequential or Graph built in Python) and data (a YAML file that pipekit_ops.loads reads back). The data form needs every step in it to round-trip (a strict-JSON get_config, from_state, no live objects), and a recipe that comes from a user goes through loads_sandboxed with an allowlist: a recipe that names classes is code. Settings composed with Hydra override values inside a recipe; they do not become a second recipe format.
Against gaussx's own numbering: its L1 operators become ⓪ structures and its L0 primitives ① primitives (the swap above); L1.5, L2 and L3 become ② strategy, ③ model (its "distributions", renamed so the rung reads the same in libraries with none: a functax plan, a kernellib estimator, a sesmax model) and ④ algorithm; ⑤ step and ⑥ recipe are pipekit's. gaussx's word "recipes" moves up the ladder: in the stack a recipe is a composition of steps, as in a kitchen, in Pangeo Forge's ETL recipes and in torchtune's training recipes. UnscentedIntegrator moves from gaussx's recipes to the strategies: it is a choice of how to integrate, like CGSolver is a choice of how to solve.
Each kind of user works in a band of the ladder and treats the rung below it as atomic: their floor. A geoscientist calls MatchedFilter and never opens it; an ML researcher calls solve and never opens it. The word is floor, not primitive: ① stays one role, the same for everyone.
| Who | Works at | Their floor | Data |
|---|---|---|---|
| data engineers: ETL, APIs, research runs | orchestration, ⑥ recipes | ⑥ recipes, often as data | ◆ carriers and files |
| geoscience researchers | ⑤ steps, ⑥ recipes (④ on the xarray side) | ⑤ steps | ◆ carriers, ▤ named arrays |
| ML researchers | ② strategies, ③ models, ④ algorithms | ① primitives | ▦ arrays |
| numerics and CS | ⓪ structures to ④ algorithms | JAX, lineax, diffrax | ▦ arrays |
Carriers start at ⑤, but ⑤ does not need carriers. Nothing below ⑤ computes on a carrier: that is the second of the three invariants under rule 3, nothing below GeoStack sees , only (). A step is exactly where and happen, so geo territory begins there. The converse does not hold: JaxModelOp, a DA cycle and a training loop are steps and recipes on plain arrays. Geo territory is ⑤ and ⑥ with carrier data. geoarray's own nouns and verbs (GeoArray, crop, to_pixel) sit at ⓪ and ① but handle georeference only and compute no field. On the xarray side, geoscientists call functions on named arrays (xrops), so their floor is set by the data as much as by the rung.
Floors leak, so design for one rung down. When a step's settings fall short, a geoscientist should reach the algorithm it wraps without reading its source: every step's docstring names the function it calls (the in ). Errors speak the band's language: band names, CRS and fill at ⑤ and ⑥, shapes at ④ and below. Docs open by band, one entry point per kind of user, not one per library. And ⑤ and ⑥ are the API the most people touch, so they get the strictest deprecations.
Each axis gets its own words and its own mark, so no two axes share one. The letter L goes back to the axis that owns it in remote sensing: the processing levels of the data.
| Axis | Asks | Vocabulary | Mark |
|---|---|---|---|
| role | what job does it do in its library's API? | ⓪ structure (noun) · ① primitive (verb) · ② strategy (adverb) · ③ model (noun phrase) · ④ algorithm (sentence) · ⑤ step (sentence with its settings) · ⑥ recipe (paragraph); orchestration above, not a rung | circled digit |
| kind | how is it packaged? | ƒ function · ▣ Module · → Operator | glyph |
| data | what does it see? | ▦ array · ▤ named array · ◆ carrier | glyph |
| layer | where does it live? | Core · GeoNumerics · GeoStack (geotoolz + xrtoolz) · GeoModels · GeoBench · GeoApps | name |
| tier | what may it import? | T0 to T4, within a layer | T |
| level | how processed is the data? (data products, not code) | L0 raw · L1 calibrated · L2 geophysical · L3 gridded · L4 model output | L |
Two names on every axis. gaussx.CGSolver: ② strategy · ▣ Module · ▦ array · Core T0. geoops.matched_filter.MatchedFilter: ⑤ step · → Operator · ◆ carrier · GeoStack T3; it turns an L1B radiance scene into an L2 enhancement product.
How the axes constrain each other. The role fixes the kind: ① and ④ are functions, ⓪, ② and ③ are Modules (or, at ⓪, a third-party array), ⑤ and ⑥ are Operators (a recipe in code is a Sequential or Graph, itself an Operator; in data, the YAML that loads into one), so a library's chips only need the digit. The kind and the data fix the lowest layer. Inside one library, imports follow the roles upward, nouns first; the one entanglement left is between verbs and adverbs (a primitive takes a strategy; some strategies call primitives), which gaussx documents as its dispatch core.
What this retires. The kind once called "primitive" is now "function", so "primitive" names one role (①) only. geostack's docs/design/stack_levels.md L1 to L3 (primitives and machinery, operators on carriers, pipelines) were a mix of kind and data; the glyph pairs say the same without a number (ƒ ▣ on ▦; → on ◆; ⑤ steps composed into ⑥ recipes). Two words stay where the field uses them on purpose: "linear operator" (lineax's AbstractLinearOperator, a ⓪ structure) and "observation operator" (H). Capital-O Operator means pipekit's kind only.
Every public name sits in one cell of this grid, and each cell has a lowest layer it may live in.
| ▦ plain array | ▤ named array | ◆ carrier | |
|---|---|---|---|
| ƒ function | einx.dot, a functax stencil's apply, geoarray.to_pixel(transform, xy), geophysix.coriolis(lat)Core GeoNumerics and up |
pipekit_array.run_kernelCore (pipekit-array) and up |
geoarray's own verbs: crop, pad, read_window, stack. They move data and georeference together and never compute a new fieldGeoStack and up |
| ▣ Module | functax plans, gridax grids, gaussx and kernellib operators, finitevolX operators, the modelsCore GeoNumerics and up | empty by design: a Module works on plain arrays, and an Operator puts the names on | never: a Module never holds or takes a carrier |
| → Operator | JaxModelOp, DACycle, TrainingLoop, rolloutsCore GeoModels; never GeoNumerics |
ISel, Reduce, ArgReduce, MapOver, KernelOp (pipekit-array)Core (pipekit-array) and up |
geoops, xrops; each model's geo extraGeoStack; GeoModels in the geo extra |
GeoNumerics fills only the plain-array column, and only its first two rows. Core adds Operators on plain arrays and, through pipekit-array, the named-array column. The carrier column starts at GeoStack. This also settles the xarray adapter in pipekit-array without an exception: there a DataArray is a named array, which Core may hold. As a carrier, with a CRS, it belongs to xrops.
pipekit_ops.Operator? Then it is an Operator, whatever else it inherits; JaxOperator is also an eqx.Module and is still an Operator. Else, is it a class whose setup does work before a pure call? A Module. Else it is a function.Not in code; in prose, yes. In code the namespaces already separate the two kinds of operator (lineax calls its own AbstractLinearOperator), and renaming pipekit_ops.Operator would touch pipekit, every geoops and xrops class, vardax, plumax and sesmax, all for a prose problem. "Observation operator" is the field's own word for H and stays. In prose: function for the kind (not "kernel", which kernellib uses for covariance functions, nor "primitive", which now names the ① role), Module, Operator with a capital O for pipekit's, and carrier for the data. The full vocabulary, one per axis, is in the box under the role ladder. The suffix rules on the library cards of the layer map keep the class names apart as well.
Unchanged: a library imports any lower layer, and lower tiers of its own layer. A tier never changes which kinds a library may define. pipekit and gaussx are both Core tier 0, yet one defines Operators and the other Modules; the cells come from the layer.
Sequential or Graph from a Hydra configgeoops.SampleAt: _apply reads np.asarray(g) and g.gridfunctax.BilinearPlan, fitted on the grid and the pointseinx.dot with the stencil weightswrap_like(result, g): a carrier againArrays go everywhere. Every kind takes them.
Carriers stop at GeoStack. An Operator on carriers unwraps before it calls down and rewraps on the way back; nothing below GeoStack takes a carrier as input, and that includes xarray.DataArray. One narrow exception, listed by name with its reason:
An export at the edge. A method that only returns results as a labelled container (to_xarray), imports it lazily behind an extra, attaches no CRS, and is called by nothing else in the library. Carriers flowing out at the end couple nothing; carriers flowing in are what the rule stops.
Configs stop at Operators. Hydra and JSON configs build Operators and apps; Modules and functions take plain values (numbers, arrays, other Modules), never a DictConfig.
Every row of the boundaries table on the layer map carries the kinds and data that library exports. Each must be allowed in the library's layer (rule 1), and a public name sits in exactly one cell. "geo" marks Operators on carriers that live in a model's geo extra.
tests/test_dependency_graph.py gets a list of forbidden imports per layer, counted in src/ only (scripts and notebooks are exempt), with a KNOWN list that only shrinks:
| Layer | Never imports | Breaks today | Fix |
|---|---|---|---|
| GeoNumerics | pipekit*, pyproj, rasterio, xarray, geoarray, geoops, xrops, every GeoStack package | nothing in src/; xarray only in the scripts and notebooks of finitevolX and spectraldiffx | none needed |
| Core | pyproj, rasterio, xarray, geoarray, geoops, xrops, every GeoStack package | xtremax.extraction and xtremax.simulations (7 modules); vardax _src/utils/patches.py and masks.py; pyrox-lgm's lazy to_xarray in _result.py; pipekit_array.xarray | xtremax: extraction and simulations on xarray move to xrtoolz; the extremes modelling stays. vardax: extract_patches and the three masks take Datasets, so their xarray halves move to xrtoolz; the array halves (masks from a shape and a key, window starts) stay in vardax as primitives; trajectory_to_xr_dataset stays as an export at the edge.pyrox-lgm: to_xarray stays as an export at the edge.pipekit_array.xarray: not a break. It adapts DataArray as a named array, which Core may hold; Dataset and DataTree dispatch is already in xrops (PK-P5) |
The Operator contract itself is already enforced: test_operator_contract.py in geotoolz walks every geoops Operator, and geoops.testing.check_operator runs the same checks on Operators outside geotoolz, such as the models' geo extras.
geoarray stays a package of the geotoolz workspace and keeps every carrier. What moves out is the geometry maths it does today. Each geoarray grid becomes a CRS-free grid from below, with a pyproj.CRS attached. Affine raster grids are the CRS-free geometry functax already owns for interpolation; the grids that physics needs (curvilinear, swath, staggered, vertical) are gridax's; anything that knows the Earth's shape comes from geophysix.
| Concept | functax Core | gridax GeoNumerics | geoarray GeoStack |
|---|---|---|---|
| Affine raster grid | AffineGrid(transform, shape): res, bounds, shift, crop(window), window(bounds), pixel centres | none | Grid = an AffineGrid + a CRS; stays hashable, keeps isclose and get_config / from_config |
| World to pixel | to_pixel(grid, xy): fractional indices, the "find" step of a plan | none | reprojects the points with pyproj, then calls functax |
| Pixel coordinates | pixel centres and corners of an AffineGrid | none | pixel_coords(g) reads the grid and calls functax |
| Pixel windows | none | none | PixelWindow stays: it is the read contract of RasterSource. read_window, write_window, crop, pad move the data and call AffineGrid.shift |
| Curvilinear and swath grids | none | CurvilinearGrid(lat, lon), swath geometry: cell corners and areas, neighbours, nearest cell | CurvilinearGrid and SwathGrid add the CRS of lat/lon; GeoSwath keeps per-scene lat/lon as leaves |
| Physical grid structure | none | Arakawa staggering, metric arrays (dx, dy, areas), masks, vertical coordinates and remapping | Vertical, VerticalAxis: labels only |
| Areas on the Earth | none | cell areas of any grid, on the ellipsoid from geophysix | pixel_area (designed): reads the CRS, picks planar or ellipsoidal, calls gridax |
| Footprint in another CRS | bounds in grid coordinates | none | footprint: a Shapely polygon, densified and reprojected with pyproj |
| CRS, fill, attrs, dims, readers, georeader interop | never | never | only here |
A condition for the move: geoarray installs and runs on NumPy alone, so the kernels it calls must too. functax and gridax take the array namespace (xp) in those kernels and import JAX only when a JAX array arrives. If that costs more than it saves, geoarray keeps its four affine helpers, and a test pins them equal to functax's.