Skip to content

pipekit

Composable scientific pipelines. Train, register, deploy — one operator interface end-to-end.

pipekit is a uv workspace of seven Python packages that together turn the pieces of a scientific ML workflow into composable Operators: data loading, QC, training, data assimilation, model registry, reproducibility. The same operator that returns a trained emulator drops into a forecast Cycle as a step function. One mental model, one composition surface, end to end.

from pipekit_cycle import Cycle, NeuralForward
from pipekit_experiment import LocalModelRegistry

# Reload a trained model by name and forecast 24 steps with it
# as the forward model — same Operator interface as inference.
registry = LocalModelRegistry("/models")
op = registry.load("methane_emulator_v3")

forecast = Cycle(step_op=NeuralForward(op, dt=3600.0), n_steps=24)
final_state, _ = forecast(x0, None)

Three doors

  • Getting Started

    From git clone to your first pipeline, first DA cycle, and first registered model. 5 minutes of copy-paste.

  • Concepts

    The 10-minute mental model. What's an Operator? Why carrier-agnostic? How does dispatch work?

  • Tutorials

    End-to-end walkthroughs. Train an emulator. Run a DA cycle. Persist via the registry.

  • Installation

    Per-package install matrix. Every extra spelled out.

  • API Reference

    mkdocstrings-generated reference, kept in sync with source.

  • Notebooks

    Executed Jupyter notebooks — MLP quickstart, train → forecast handoff on a toy oscillator.

The packages

Package Status Purpose
pipekit ✅ implemented Core: Operator, Sequential, Graph, control, observe, …
pipekit-array 🚧 partial Array-API operators (Phase A landed; B-D pending).
pipekit-cycle ✅ implemented Time-stepping, DA cycles, observation/forward protocols.
pipekit-experiment ✅ implemented Content-addressed model registry, tracker protocols, tool adapters.
pipekit-evaluate 🚧 partial Benchmark ladder; Unit × Lens × Stage scorers; EvaluationReport.
pipekit-train ✅ implemented Training pipelines — datasets, losses, loop, Equinox adapter.
pipekit-jax ✅ implemented JaxModelOp — eqx.Module wrapper with registry weight round-trip.