V6 Capstone — one Graph emits scores + diagnostic panels¶

This notebook demonstrates D15: a single Graph consumes prediction + reference Datasets and emits both numerical scores and validation panels as terminal outputs. Closes the V6 epic.

1. Synthetic SSH twin (truth + noisy + smoothed predictions).
2. Compute V1 score: SkillByLeadTime.
3. Compute a 1-D PSD-style spectral score (synthetic for the demo).
4. Build a single Graph whose terminal nodes include both score Datasets and LeadTimeSkillPanel / SpectralSkillPanel / ScaleSkillPanel.
5. Bonus: V6.4 EventVerificationPanel driven by hand-authored event masks (V5 detectors are stubs in this branch).

In [1]:

import numpy as np
import xarray as xr
import matplotlib.pyplot as plt

from pipekit import Graph, Input
from xrops.metrics import RMSE, SkillByLeadTime
from xrops.viz.validation import (
    LeadTimeSkillPanel,
    ScaleSkillPanel,
    SpectralSkillPanel,
    EventVerificationPanel,
)

_ = xr.set_options(  # HTML repr, every section collapsed
    display_style="html",
    display_expand_attrs=False,
    display_expand_coords=False,
    display_expand_data=False,
    display_expand_data_vars=False,
    display_expand_groups=False,
    display_expand_indexes=False,
)

import numpy as np import xarray as xr import matplotlib.pyplot as plt from pipekit import Graph, Input from xrops.metrics import RMSE, SkillByLeadTime from xrops.viz.validation import ( LeadTimeSkillPanel, ScaleSkillPanel, SpectralSkillPanel, EventVerificationPanel, ) _ = xr.set_options( # HTML repr, every section collapsed display_style="html", display_expand_attrs=False, display_expand_coords=False, display_expand_data=False, display_expand_data_vars=False, display_expand_groups=False, display_expand_indexes=False, )

1 — Synthetic SSH twin¶

In [2]:

rng = np.random.default_rng(0)
lead = np.arange(7)
lon = np.linspace(-30, 30, 41)
lat = np.linspace(20, 50, 31)
LON, LAT = np.meshgrid(lon, lat, indexing='xy')
truth_field = 0.1 * np.sin(np.deg2rad(2*LON)) * np.cos(np.deg2rad(LAT))
truth = np.broadcast_to(truth_field, (lead.size,) + truth_field.shape)
noise = rng.normal(0.0, 0.01 * (1 + lead[:, None, None] * 0.4), size=truth.shape)
pred = truth + noise
ds_pred = xr.Dataset({'ssh': (('lead_time', 'lat', 'lon'), pred)},
                     coords={'lead_time': lead, 'lat': lat, 'lon': lon})
ds_ref  = xr.Dataset({'ssh': (('lead_time', 'lat', 'lon'), truth)},
                     coords={'lead_time': lead, 'lat': lat, 'lon': lon})
ds_pred

rng = np.random.default_rng(0) lead = np.arange(7) lon = np.linspace(-30, 30, 41) lat = np.linspace(20, 50, 31) LON, LAT = np.meshgrid(lon, lat, indexing='xy') truth_field = 0.1 * np.sin(np.deg2rad(2*LON)) * np.cos(np.deg2rad(LAT)) truth = np.broadcast_to(truth_field, (lead.size,) + truth_field.shape) noise = rng.normal(0.0, 0.01 * (1 + lead[:, None, None] * 0.4), size=truth.shape) pred = truth + noise ds_pred = xr.Dataset({'ssh': (('lead_time', 'lat', 'lon'), pred)}, coords={'lead_time': lead, 'lat': lat, 'lon': lon}) ds_ref = xr.Dataset({'ssh': (('lead_time', 'lat', 'lon'), truth)}, coords={'lead_time': lead, 'lat': lat, 'lon': lon}) ds_pred

Out[2]:

<xarray.Dataset> Size: 72kB
Dimensions:    (lead_time: 7, lat: 31, lon: 41)
Coordinates: (3)
Data variables: (1)

xarray.Dataset

• Dimensions:
  • lead_time: 7
  • lat: 31
  • lon: 41
• Coordinates: (3)
  • lead_time
    (lead_time)
    int64
    0 1 2 3 4 5 6

    array([0, 1, 2, 3, 4, 5, 6])

  • lat
    (lat)
    float64
    20.0 21.0 22.0 ... 48.0 49.0 50.0

    array([20., 21., 22., 23., 24., 25., 26., 27., 28., 29., 30., 31., 32., 33.,
           34., 35., 36., 37., 38., 39., 40., 41., 42., 43., 44., 45., 46., 47.,
           48., 49., 50.])

  • lon
    (lon)
    float64
    -30.0 -28.5 -27.0 ... 28.5 30.0

    array([-30. , -28.5, -27. , -25.5, -24. , -22.5, -21. , -19.5, -18. , -16.5,
           -15. , -13.5, -12. , -10.5,  -9. ,  -7.5,  -6. ,  -4.5,  -3. ,  -1.5,
             0. ,   1.5,   3. ,   4.5,   6. ,   7.5,   9. ,  10.5,  12. ,  13.5,
            15. ,  16.5,  18. ,  19.5,  21. ,  22.5,  24. ,  25.5,  27. ,  28.5,
            30. ])

• Data variables: (1)
  • ssh
    (lead_time, lat, lon)
    float64
    -0.08012 -0.08013 ... -0.008556

    array([[[-0.08012247, -0.0801303 , -0.0696185 , ...,  0.08386248,
              0.09374357,  0.06878911],
            [-0.0657112 , -0.06483789, -0.06771513, ...,  0.09555217,
              0.08018183,  0.0745185 ],
            [-0.08407211, -0.08867164, -0.08778755, ...,  0.08288663,
              0.08620097,  0.08105241],
            ...,
            [-0.04847227, -0.04986281, -0.05713734, ...,  0.05380786,
              0.05783853,  0.07306559],
            [-0.06156972, -0.03840445, -0.06717082, ...,  0.06427136,
              0.05720907,  0.07051894],
            [-0.06505957, -0.04260129, -0.04291181, ...,  0.05492714,
              0.04445159,  0.0609619 ]],
    
           [[-0.06010987, -0.0967631 , -0.07516515, ...,  0.08034463,
              0.07883482,  0.07897306],
            [-0.06358921, -0.0921743 , -0.07728636, ...,  0.04678928,
              0.09291832,  0.10508643],
            [-0.05979914, -0.09201022, -0.06890721, ...,  0.1019033 ,
              0.0961179 ,  0.07906002],
    ...
            [-0.02520321, -0.05454524, -0.06758457, ...,  0.10699861,
              0.1099782 ,  0.10191527],
            [-0.04484254, -0.09257246, -0.02144516, ...,  0.00286813,
              0.08261383,  0.05149801],
            [-0.05019405,  0.02823295, -0.06031757, ...,  0.02082007,
              0.04173737,  0.02439708]],
    
           [[-0.05281393, -0.07564192, -0.12684879, ...,  0.10455432,
              0.08093202,  0.10403957],
            [-0.0611351 , -0.03329633, -0.09031306, ...,  0.07606461,
              0.0116016 ,  0.06618693],
            [-0.04319892, -0.09519993, -0.05122734, ...,  0.0499047 ,
              0.0233888 ,  0.07666828],
            ...,
            [-0.01587312, -0.05065766, -0.08815107, ...,  0.02875774,
              0.1185806 ,  0.04888021],
            [-0.05808658, -0.07029955, -0.08308145, ...,  0.03445105,
              0.00671909,  0.03987263],
            [-0.0709761 , -0.10991044, -0.04247588, ...,  0.06971486,
              0.02553555, -0.00855648]]], shape=(7, 31, 41))

2 — V1 SkillByLeadTime via a Graph emitting scores and a panel¶

In [3]:

rmse = RMSE('ssh', dims=('lat', 'lon'))
skill_op = SkillByLeadTime(metric=rmse, lead_dim='lead_time')
panel_op = LeadTimeSkillPanel(lead_dim='lead_time', ylabel='RMSE')

pred_in = Input(name='pred')
ref_in  = Input(name='ref')
scores_node = skill_op(pred_in, ref_in)
panel_node  = panel_op(scores_node)

g = Graph(inputs={'pred': pred_in, 'ref': ref_in},
          outputs={'scores': scores_node, 'panel': panel_node})
out = g(pred=ds_pred, ref=ds_ref)
print('scores:'); print(out['scores'])
out['panel']

rmse = RMSE('ssh', dims=('lat', 'lon')) skill_op = SkillByLeadTime(metric=rmse, lead_dim='lead_time') panel_op = LeadTimeSkillPanel(lead_dim='lead_time', ylabel='RMSE') pred_in = Input(name='pred') ref_in = Input(name='ref') scores_node = skill_op(pred_in, ref_in) panel_node = panel_op(scores_node) g = Graph(inputs={'pred': pred_in, 'ref': ref_in}, outputs={'scores': scores_node, 'panel': panel_node}) out = g(pred=ds_pred, ref=ds_ref) print('scores:'); print(out['scores']) out['panel']

scores:
<xarray.DataArray 'ssh' (lead_time: 7)> Size: 56B
0.009817 0.0142 0.01789 0.02185 0.0259 0.03086 0.03383
Coordinates: (1)

Out[3]:

3 — Spectral skill panel (1-D synthetic score)¶

In [4]:

freq = np.logspace(-3, -0.5, 30)
psd_score = np.clip(1 - 0.6 * (np.log10(freq) - np.log10(freq.min())) / (np.log10(freq.max()) - np.log10(freq.min())), 0, 1)
scores_da = xr.DataArray(psd_score, coords={'freq': freq}, dims=('freq',), name='psd_score')
spec_panel = SpectralSkillPanel(freq_dim='freq', log_x=True)
spec_panel(scores_da)

freq = np.logspace(-3, -0.5, 30) psd_score = np.clip(1 - 0.6 * (np.log10(freq) - np.log10(freq.min())) / (np.log10(freq.max()) - np.log10(freq.min())), 0, 1) scores_da = xr.DataArray(psd_score, coords={'freq': freq}, dims=('freq',), name='psd_score') spec_panel = SpectralSkillPanel(freq_dim='freq', log_x=True) spec_panel(scores_da)

Out[4]:

4 — Region skill panel¶

In [5]:

region_scores = xr.Dataset(
    {'rmse': ('region', [0.012, 0.025, 0.018, 0.031])},
    coords={'region': ['Gulf Stream', 'Mediterranean', 'Agulhas', 'Kuroshio']},
)
ScaleSkillPanel(region_dim='region', metric='rmse')(region_scores)

region_scores = xr.Dataset( {'rmse': ('region', [0.012, 0.025, 0.018, 0.031])}, coords={'region': ['Gulf Stream', 'Mediterranean', 'Agulhas', 'Kuroshio']}, ) ScaleSkillPanel(region_dim='region', metric='rmse')(region_scores)

Out[5]:

5 — Event verification panel (synthetic)¶

V5 detectors land in a future PR; the panel is fixture-driven here to demonstrate the V6.4 contract.

In [6]:

lon_e = np.linspace(-10, 5, 31)
lat_e = np.linspace(30, 45, 31)
pred_grid = np.zeros((lat_e.size, lon_e.size), dtype=int)
pred_grid[5:9, 5:9] = 1
pred_grid[18:22, 12:18] = 2
pred_grid[2:5, 22:27] = 3
ref_grid = np.zeros_like(pred_grid)
ref_grid[5:9, 5:9] = 10
ref_grid[19:23, 13:19] = 20
ref_grid[20:24, 1:5] = 30
objects_pred = xr.Dataset({'label': (('lat','lon'), pred_grid)}, coords={'lat': lat_e, 'lon': lon_e})
objects_ref  = xr.Dataset({'label': (('lat','lon'), ref_grid)},  coords={'lat': lat_e, 'lon': lon_e})
matches = {'hits': [(1, 10), (2, 20)], 'false_alarms': [3], 'misses': [30]}
scores = {'POD': 2/3, 'FAR': 1/3, 'CSI': 0.5, 'IoU': 0.61}
EventVerificationPanel(use_cartopy=False)(objects_pred, objects_ref, matches, scores)

lon_e = np.linspace(-10, 5, 31) lat_e = np.linspace(30, 45, 31) pred_grid = np.zeros((lat_e.size, lon_e.size), dtype=int) pred_grid[5:9, 5:9] = 1 pred_grid[18:22, 12:18] = 2 pred_grid[2:5, 22:27] = 3 ref_grid = np.zeros_like(pred_grid) ref_grid[5:9, 5:9] = 10 ref_grid[19:23, 13:19] = 20 ref_grid[20:24, 1:5] = 30 objects_pred = xr.Dataset({'label': (('lat','lon'), pred_grid)}, coords={'lat': lat_e, 'lon': lon_e}) objects_ref = xr.Dataset({'label': (('lat','lon'), ref_grid)}, coords={'lat': lat_e, 'lon': lon_e}) matches = {'hits': [(1, 10), (2, 20)], 'false_alarms': [3], 'misses': [30]} scores = {'POD': 2/3, 'FAR': 1/3, 'CSI': 0.5, 'IoU': 0.61} EventVerificationPanel(use_cartopy=False)(objects_pred, objects_ref, matches, scores)

Out[6]: