Pipeline driver

Everything in this repository runs through the single driver scripts/run_pipeline.py: a phase-structured, idempotent pipeline over one data-store working copy. Inputs are pulled from the shared ipf/data-store registry (fetch-store, sha256-verified) and every product carries an EOPF PSFD §3 name.

Modes

Mode	Phases	Products
--mode nominal (default)	fetch-store → l0-decode → radiometric → enhancement → toa → stats → report	PSFD-named L1A + L1B (TOA reflectance) + the per-band QA statistics table
nominal + --full	… toa → coregister → georeference → atmospheric → pansharpen …	+ L1C / L2A (demo geo/atmospheric ADFs — flagged in the report)
--mode calibration	fetch-store → cal-decode → radiometric-cal → cal-validate → report	the derived-NUC product (PSFD _NUC) + the coefficient cross-check

Usage

python scripts/run_pipeline.py <store>                        # nominal chain
python scripts/run_pipeline.py <store> --full                 # + L1C/L2A (demo geo ADFs)
python scripts/run_pipeline.py <store> --mode calibration     # derive + cross-validate the NUC
python scripts/run_pipeline.py <store> --phases stats         # QA table of the persisted L1B
python scripts/run_pipeline.py <store> --phases publish-store --publish-version <X.Y.Z>

Phases are idempotent and individually selectable (--phases); each writes its evidence under <store>/report/ and the report phase assembles pipeline_report.md. Chain phases need the eopf environment; fetch-store / stats / report / publish-store run with numpy+zarr alone.

Calibration mode

The calibration campaign arrives exactly like any downlink: the producer packages the dark (S02MSIDCA, operation mode DASC) and Lambertian sun-diffuser (S02MSISCA, ABSR) acquisitions as canonical L0 products (CCSDS-122 compressed ISPs, PSFD names, operation-mode metadata). cal-decode ground-decodes both through L0DecodeUnit (the REQ-F-L0-06 path in operational use), radiometric-cal derives the NUC in the radiometric unit’s calibration mode on the diffuser datatake, and cal-validate cross-checks it against the producer-derived cal-DB — both sides derive from the same (bit-exactly carried) frames, so agreement to float32 precision is expected. The producer-acquires → downlink → consumer-derives loop closes over the data-store.

CI

The manual pipeline-nominal and pipeline-calibration jobs (integration-tests stage) run the two modes end-to-end against the shared data-store and artifact the report/ evidence; see the results page for the current numbers.