geopatcher

Where geopatcher sits¶

The repo’s README calls geopatcher the locality layer between catalogs and operators. It answers a single question: “what slice of the data does my operator see at once, and how do local outputs become a global field?”

                georeader  (GeoTensor, GeoData, readers)
                    ▲
       ┌────────────┼────────────┐
  geocatalog     pipekit     geopatcher ──┐
       ▲            ▲            ▲        │ optional [pipekit] extra
       │            └──── geopatcher.integrations.pipekit
       │                         ▲          (GridSampler, ApplyToChips, Stitch)
       └────────────┬────────────┘
              geotoolz · xrtoolz

The scoping report’s reading of this picture holds: geopatcher is a peer of geocatalog and pipekit, it never imports geocatalog (geocatalog’s CatalogDomain satisfies its Domain Protocol instead), and pipekit appears only in one 154-line integrations.pipekit module behind the [pipekit] extra. The stack-wide rules behind that layering are in Report 16.

The four-axis framework¶

The four axes¶

Each axis is a strategy object, a small dataclass with a single job, that the patcher composes at construction. The repo’s rule is that anything you can swap without touching the other three is on its own axis (Figure 1 draws each one).

Table 1 lists them: 38 spatial and 17 temporal classes, plus Stencil, TimeStencil and Closed for coordinate-space temporal stencils. The scoping report counted “60+”; the families, not the count, are what matters. The spatial aggregation family is where the research-grade work sits. Overlap-add with feathered windows, inverse-variance weighting, streaming histograms and approximate quantiles are the merge strategies that real sliding-window inference needs.

The patch lifecycle¶

A run touches each axis in turn. The sampler places anchors, the geometry turns each anchor into an index window, the field reads the window, the operator transforms the patch, and the aggregation merges the outputs, using the window’s weights (Figure 2):

Each Patch carries:

• data: the payload read from the field, usually an array or a GeoTensor;

• anchor: where the sampler put it, (row, col) for a raster;

• indices: the geometry’s index window, such as a rasterio.windows.Window or a list of neighbour ids;

• weights: the window’s per-cell weights;

• masked: set when an error policy replaced the data.

split is an iterator by design (the repo’s ADR-001), so a field larger than memory streams patch by patch, and list(…) gives the eager case.

Overlap-add, the canonical merge¶

The aggregation that matters most for inference is overlap-add. Let patch $p$ cover the cells $\Omega_p$, carry window weights $w_p(x)$, and produce operator output $\hat y_p(x)$ for each cell $x \in \Omega_p$. The stitched field is the weight-normalised sum over every patch that covers the cell:

A feathered window such as Hann tapers each patch towards its edges, so seams between patches disappear. The denominator in (1) makes the reconstruction exact wherever patches overlap: if every $\hat y_p$ is the identity, then $y(x)$ is the input. Figure 3 shows the Hann weights, their sum and the flat reconstruction.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
import numpy as np
import rasterio
from georeader.geotensor import GeoTensor

import geopatcher as gp
from geopatcher import spatial

# 1. Wrap any array as a Field.
ramp = np.linspace(0, 1, 64)
arr = np.outer(ramp, ramp).astype(np.float32)                 # (64, 64)
field = gp.RasterField(
    GeoTensor(values=arr, transform=rasterio.Affine.identity(), crs="EPSG:32630")
)

# 2. Compose the four axes.
patcher = gp.SpatialPatcher(
    geometry=spatial.geometry.Rectangular(size=(16, 16)),     # Ωₚ: 16×16 cells
    sampler=spatial.sampler.RegularStride(step=(8, 8)),       # 50 % overlap
    window=spatial.window.Hann(),                             # wₚ(x): feathered
    aggregation=spatial.aggregation.OverlapAdd(),             # y = Σ wₚŷₚ / Σ wₚ
)

# 3. Split → operate per patch → merge.
out = [p.with_data(np.asarray(p.data) * 2.0)                  # ŷₚ = 2 · patch
       for p in patcher.split(field)]                         # 49 patches, 16×16
stitched = patcher.merge(out, field.domain)                   # → (64, 64) float64
np.abs(stitched - 2 * arr).max()                              # → 4.4e-16
patcher.merge_to_field(out, field)                            # → GeoTensor (64, 64)

With no overlap (stride 16, a boxcar window), both OverlapAdd and Mean reconstruct the field exactly (np.array_equal is True). The Hann case is exact to round-off even at the outer edge, because the outermost Hann weights are small but not zero. One detail to know: merge returns float64 even for a float32 field.

Boundary policies¶

When an anchor sits close enough to the edge that its neighbourhood would overflow the field, the geometry’s boundary argument decides what happens (Figure 4; counts on a real run in Table 2):

The default deserves a warning of its own. With "drop", a stride that does not divide the field leaves a strip along the far edges that no patch covers, and merge returns NaN there. The repo’s concepts page still lists "reflect" as a planned follow-up; it landed in the standalone repo’s last commits in July, just before the merge.

The Field and Domain Protocols¶

The whole framework rests on two small Protocols, and the scoping report was right to call them the key abstraction:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
@runtime_checkable
class Domain(Protocol):
    """Metadata only: bounds and CRS, no I/O."""
    @property
    def crs(self) -> Any: ...
    @property
    def bounds(self) -> Any: ...

@runtime_checkable
class Field(Protocol):
    """A substrate the patcher reads patches out of."""
    @property
    def domain(self) -> Domain: ...
    def select(self, indexer: Any) -> Any: ...       # read one patch's data
    def with_data(self, array: Any) -> Any: ...      # rebuild around new data

A field may also provide placeholder(indexer, data), which is duck-typed and optional and is used by on_error="mask". AsyncField has the same shape with an awaitable select. Because the patcher only talks to these Protocols, it does not care what carrier sits underneath: you wrap the data once, at the boundary, and the four-axis machinery does the rest. Table 3 lists the adapters that ship.

The extras-gated adapters import lazily: geopatcher.fields.XarrayField resolves on first access, and a missing extra produces a message naming the install command. The domains (RasterDomain, GridDomain, VectorDomain, PointDomain) live in geopatcher.fields; RasterDomain is georeader’s GeoDataBase.

Public surface¶

Table 4 lists the public namespaces.

A patcher does more than split and merge now. SpatialPatcher also has:

• merge_to_field (returns a GeoTensor) and merge_to_xarray;

• patch_at for random access to one anchor;

• reduce and two_pass, the codified helpers for operators that need global context;

• to_delayed and to_dask_bag for dask;

• error handling: on_error="raise" | "skip" | "mask" | "retry", with max_retries and retry_on, and failures recorded in patcher.errors.

SpatioTemporalPatcher(spatial, temporal, coupling="product" | "coupled") composes the spatial and temporal patchers.

What shipped beyond the scope¶

Table 5 lists what the patcher gained beyond the scoping report.

The pipekit integration¶

The scoping report isolated the only pipekit-coupled code in geotoolz’s patcher (_src/ops.py, three operator wrappers) in geopatcher.integrations.pipekit, behind a [pipekit] extra. That is what shipped, with one rename (Table 6):

The three compose into a sliding-window inference pipeline:

1
2
3
4
5
6
7
8
9
10
import pipekit as pk
from geopatcher.integrations.pipekit import ApplyToChips, GridSampler, MergePatches

pipeline = pk.Sequential([
    GridSampler(patcher=patcher),                          # Field → Iterator[Patch]
    ApplyToChips(operator=model_op),                       # ŷₚ = model(patch)
    MergePatches(aggregation=spatial.aggregation.OverlapAdd(),
                 domain=field.domain),                     # y = Σ wₚŷₚ / Σ wₚ
])
result = pipeline(field)

The scoping report’s three reasons for keeping the bridge in geopatcher rather than geotoolz held:

• It is about the patcher: the wrappers take a patcher and an aggregation.

• xrtoolz could use it too. It doesn’t yet.

• The dependency stays opt-in. Without [pipekit], geopatcher installs without pipekit.

geotoolz’s patch_ops re-exports the same three objects.

Dependencies and extras¶

The scoping report’s core was numpy and scipy only: no field adapters, no pipekit. A user would wrap their own data in a Field and bring their own adapter. Table 7 and Table 8 set that against what shipped.

No geocatalog dependency. This holds exactly. You can patch your own data without geocatalog, and find files with geocatalog without geopatcher. When you want both, geocatalog.patch.field_for hands a catalog to geopatcher as a RasterField, and CatalogDomain satisfies Domain (Report 6). Neither package imports the other at module level.

Migration from geotoolz¶

The scoping report planned to:

• move geotoolz.patch into geopatcher, except _src/ops.py, which goes to integrations/pipekit;

• re-export everything from geotoolz (gz.SpatialPatcher, import geopatcher as patch) for two minor versions;

• leave a small geotoolz patch_ops module with GeoTensor-aware operators (ReadFromCatalogAsPatches, GeoTensorChipFilter).

Table 9 gives each step’s status.

The scoping report’s cross-library examples still work in current names:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
import xarray as xr
import geopatcher as gp
from geopatcher import spatial
from geopatcher.fields import XarrayField

field = XarrayField(xr.open_dataset("reanalysis.nc")["sst"])
patcher = gp.SpatialPatcher(
    geometry=spatial.geometry.Rectangular(size=(32, 32)),
    sampler=spatial.sampler.RegularStride(step=(16, 16)),
    window=spatial.window.Boxcar(),
    aggregation=spatial.aggregation.Mean(),
)
for patch in patcher.split(field):     # patch.data is an xr.DataArray
    ...

The scoping report’s JAX example also holds as a pattern: the patcher iterates in Python, the patches are stacked, and a jitted, vmapped function processes them all at once. run.batch_split and run.stack_patches now give that stacking step a name. A pipeline built from pipekit operators still cannot be jitted (Report 3).

Monorepo, release and timing¶

The scoping report picked a single “pipekit-ecosystem” monorepo for the patcher. Its reasons were especially strong here: six field adapters across five ecosystems, 60-odd classes to test across all of them, and a cross-package bridge. As with geocatalog, the patcher landed in the geotoolz monorepo instead, beside geocatalog and geocloud, with per-package releases (geotoolz-patcher-v0.10.2). There are 60 test modules (about 19,000 lines), including a pipekit integration test. Table 10 gives the release record.

The estimate was a week and a half of focused work, in one coordinated v1.0 with the pipekit and geocatalog extractions. It was gradual instead: one merge in July, then a series of breaking releases, each absorbed per package. What the split was meant to unblock did happen. Aggregations, adapters (CogField, DaskField), samplers (AlongTrack) and the run machinery all shipped on the patcher’s own release cycle.

Trade-offs¶

The scoping report’s list, with how each turned out.

What got better:

1. The patcher gets its own billing: in part. The repo and the import name are geopatcher, but the distribution is geotoolz-patcher, and the standalone repo that a search would find is archived (Section ??).

2. The Field / Domain Protocols get visibility: yes. They are the root’s first exports, and geocatalog’s bridge depends on them.

3. The aggregations get their own roadmap: yes. The family grew from about fifteen to 21 classes.

4. The async story is clean: yes. AsyncSpatialPatcher, AsyncRasterField, prefetch and backpressure are first-class.

What got worse, and the mitigations:

1. The test surface is big: 60 modules, in one monorepo CI.

2. More extras to coordinate: each adapter stays behind its own extra, as planned.

3. More documentation: the docs are tutorial-grade (concepts, a patching guide, recipes, ADRs, a notebook), but they live in the archived repo, with pre-0.9.0 names.

4. People forget [pipekit]: the import error says what to install, though its PyPI note is out of date.

Open questions¶

1. Where do the docs live now? The README, concepts, recipes, ADRs and figures are in an archived repo, and they use names removed in 0.9.0. Either they move into the geotoolz monorepo, or the archived repo is unarchived as the docs site. This mirrors geocatalog’s question 1 (Report 6).

2. geopatcher or geotoolz-patcher on PyPI? Resolved: geotoolz-patcher (Report 16 §F).

3. Should the core be georeader-free? The scoped core was numpy plus scipy, so that xarray, vector and point users would not pull in the raster stack. Moving the raster domain and fields behind a [raster] extra would restore that, at the cost of one more extra for the common case. The lazy half is Report 16 D3; the extra stays open.

4. Should the axis classes be reachable from the root again? With the prefixes gone, Rectangular, Mean and RegularStride exist on both the spatial and the temporal side, so the root cannot re-export them flatly. Module-qualified imports (spatial.geometry.Rectangular) are the price of that.

5. Should drop stay the default boundary? It silently leaves NaN strips when the stride does not divide the field (960 cells in the example above). "pad" or "reflect", or at least a warning on uncovered cells, would make the default safe.

6. Should the pipekit bridge stop using private API? Resolved: nested_config becomes public (Report 16 §A).

7. Should GridSampler and MergePatches be serialisable? They are flagged forbid_in_yaml, so a sliding-window pipeline cannot be saved and reloaded, which matters for the deployment use cases.

8. What do geotoolz and xrtoolz actually need from the patcher? The scoped GeoTensor-aware operators (ReadFromCatalogAsPatches, GeoTensorChipFilter) were never built. For xrtoolz, resolved: it uses geopatcher, not xrpatcher (Report 16 D4).

Upcoming work¶

Table 11 lists the follow-ups.

References¶

Sources¶

• The geopatcher repo (archived): the README, concepts, the patching guide, design decisions (ADR-001 to ADR-006) and recipes. The four figures on this page are its own.

• geotoolz-patcher 0.10.2 in the geotoolz monorepo, and its changelog.

Citations to be added: overlap-add reconstruction, and Hann and Tukey windows, as entries in references.bib.