Interface Control Document (ICD)#
Field |
Value |
|---|---|
Document |
ICD — Interface Control Document (FINAL / field-level) |
DRD ref |
ECSS-E-ST-40C Rev.1, Annex E |
Container |
Technical Specification (TS) — |
Project |
|
Software criticality |
Category C (ECSS-Q-ST-80C Rev.2 / ECSS-E-ST-40C Annex R) |
Baselined at |
CDR (final — supersedes the PDR preliminary issue) |
Status |
Draft for CDR |
Final issue (CDR). This is the final issue of the ICD, a major constituent of the Technical Specification of
msi-processor(Annex E.1.2). It gives the concrete field-level interface definitions that realise the interface requirementsREQ-IF-*of the IRD (RD-3) and bind the software requirementsREQ-*of the SRS (RD-4). It supersedes the PDR preliminary issue: the interface inventory, group/variable trees, field-level schemas, encodings, dtypes, chunking, CRS encoding, ADF content, the EOPF CPM software signatures, the triggering payload syntax and the sensor-profile schema are now locked and aligned with the detailed SDD (RD-5, CDR issue; componentsC-*, internal interfacesIF-*). The CPM software interfaces, the triggering payload keys, theAuxiliaryDataFiledataclass, theEOProduct/EOGroup/EOZarrStorecontracts, the computing-model schema and the validation/opening/error-policy enumerations of clauses <5.3.4> and <5.3.5> were re-confirmed against the pinnedeopf == 2.8.1wheel (the workingcpm_envinstall), resolving the prior “exact 2.8.1 surface”[TBC@CDR]flags. The remaining[TBC@impl]markers are confined to genuinely sensor-private encodings (the NDA-bound L0 on-wire packetisation/bit-codec, the viewing-model coefficient form) and a small number of details legitimately fixed during implementation (SDP WP-5, post-CDR); these are stated at interface + content level here. The EOPF Product Structure & Format Definition (PSFD) is the normative product-structure reference (AD-3). The footprint is tailored to a Category C, single-developer ground-segment processor.
<1> Introduction#
Purpose. (Annex E <1>a.) This document specifies and controls the external interfaces of
msi-processor at field level: the downlinked RAW Level-0 (L0c) input, the private
instrument-calibration Auxiliary Data Files (ADF), the Level-1/Level-2 cloud-native Zarr EOProduct
outputs, the EOPF CPM software interfaces (EOProduct/EOGroup, EOProcessingUnit, EOZarrStore,
triggering), the triggering payload (job order) and the sensor-profile/configuration interface.
Objective. The ICD turns the requirement-level interface envelope of the IRD into the concrete,
verifiable interface control that the detailed design (SDD, RD-5), implementation (SDP WP-5) and V&V
(RD-8) build against. Each interface is uniquely identified (ICD-IF-*), traced to its parent
REQ-IF-* / REQ-*, and assigned a validation method (clause <6>) and a forward/backward trace
(clause <7>).
Content. Clause <4> points to the software overview in the SRS/SSS (not duplicated, Annex E <4>a). Clause <5> is the body: <5.1> general provisions (identification, in-model id assignment, traceability), <5.2> the interface requirements — the inventory of external interfaces and the controlled requirements imposed on each — and <5.3> the interface design — the field-level definitions (data-item tables, group/variable trees, signatures, payload and profile schemas). Clause <6> gives the per-interface validation approach and matrix; clause <7> the traceability.
Reason for preparation. msi-processor is an integration and ECSS-productisation effort: the
processing chain is a sequence of EOPF CPM EOProcessingUnits exchanging EOProducts and writing
cloud-native Zarr. Because the chain is sensor-agnostic and the raw/calibration data are private, the
interfaces are fixed concretely — parameterised by the sensor profile and referenced by runtime URI.
This issue finalises that control at CDR so that implementation can begin against fixed, traceable
interfaces.
<2> Applicable and reference documents#
Applicable documents (AD)#
Id |
Document |
Reference |
|---|---|---|
AD-1 |
ECSS Space engineering — Software |
ECSS-E-ST-40C Rev.1 (30 April 2025) |
AD-2 |
ECSS Space product assurance — Software product assurance |
ECSS-Q-ST-80C Rev.2 |
AD-3 |
EOPF CPM — Product Structure & Format Definition (PSFD) / common data model |
EOPF CPM docs ( |
AD-4 |
EOPF CPM API — |
EOPF CPM ( |
Reference documents (RD)#
Id |
Document |
Reference |
|---|---|---|
RD-1 |
|
|
RD-2 |
|
|
RD-3 |
|
|
RD-4 |
|
|
RD-5 |
|
|
RD-6 |
|
|
RD-7 |
|
|
RD-8 |
|
|
RD-9 |
|
|
RD-10 |
Prior work — multispectral pushbroom preprocessing pipeline (calibration/format heritage; see SRF) |
|
RD-11 |
Cloud-native data conventions |
Zarr v2/v3, CF metadata, STAC, GeoZarr |
Verification note (Annex E.1.2 / project working principle “verify, don’t assume”). The CPM signatures, the
AuxiliaryDataFiledataclass, theEOProduct/EOGroup/EOZarrStorecontracts, the computing-model (EOProcessingModel) schema, the triggering-payload section keys and the opening/validation/error-policy enumerations in clause <5.3.4>/<5.3.5> were read from the installedeopf == 2.8.1package (the projectcpm_env:eopf/computing/abstract.py,eopf/product/eo_product.py,eopf/store/zarr.py,eopf/common/constants.py,eopf/triggering/parsers.py,eopf/computing/validation.py,eopf/exceptions/error_handling.py). Because the local runtime is the pinned target version, these items are resolved (no longer[TBC@CDR]). Items still flagged[TBC@impl]depend on the sensor-private instrument definition (NDA), not on the CPM, and are confirmed against real data during implementation (SDP WP-5).
<3> Terms, definitions and abbreviated terms#
Per Annex E <3>a, only terms/abbreviations not already in the SSS <3>, IRD <3>, SRS <3>, SDD <3>, DPM
<3> and ATBD <3> glossaries (which apply in full) are listed. Enumerated values below are the
confirmed eopf == 2.8.1 sets.
Term / abbr. |
Definition |
|---|---|
DataTree |
The |
EOObject |
Base type of any node in the product tree ( |
|
CPM product alias |
|
CPM alias `Mapping[str, DataType |
|
CPM alias |
grid_mapping |
CF/GeoZarr attribute carrying the CRS encoding of a gridded variable |
computing-model JSON |
The per-PU CPM declarative model (CPM |
|
CPM store/format selector in the triggering payload ( |
|
CPM output store mode ( |
|
CPM product-validation level ( |
|
CPM workflow error policy ( |
|
CPM location-transparent path type (local FS / POSIX / |
|
To-be-confirmed at implementation: a sensor-private encoding (NDA) or a code internal legitimately fixed in SDP WP-5 (post-CDR) |
<4> Software overview#
The software overview is given in SRS (RD-4) <4> and SSS <4> and is not duplicated here (Annex E <4>a
permits reference). In summary: msi-processor is a batch, non-interactive, sensor-agnostic processor
that transforms L0c RAW MSI data into Zarr EOProducts up to L2A through a chain of EOPF CPM
EOProcessingUnits, driven by a per-sensor profile and private calibration ADFs. The external
boundary (actors E1–E6) is defined in IRD <4.1> and SDD <4.4>; this ICD controls the data and software
interfaces crossing that boundary. The realising design components (C-PU-*, C-COM-*, C-SENSORS)
and the internal interfaces (IF-PROD-*, IF-CORE-*, IF-SVC-*, IF-TRIG-*) are in SDD <5.3>–<5.5>.
<5> Requirements and design#
<5.1> General provisions to the requirements in the IRD#
a. (Annex E <5.1>a — unique identification). Each interface is identified by
ICD-IF-<group>and each controlled requirement on it byICD-IF-<group>-NN. Groups:L0(L0 input),ADF(calibration ADF),OUT(L1/L2 output),SW(CPM software),TRIG(triggering payload),PROF(sensor profile),DIAG(status/diagnostics),HMI(CLI).b. (Annex E <5.1>b — identifiers within models). Where an interface is expressed as a model (Zarr DataTree, profile JSON schema, payload/computing-model schema), identifiers are assigned within the model by the dotted path of the node/field (e.g.
OUT:/measurements/reflectance/<band>,PROF:bands[].esun,TRIG:io.output_products[].opening_mode); these paths are the traceable interface item ids.c. (Annex E <5.1>c — traceability). Each interface and data item states its parent
REQ-IF-*(IRD) andREQ-*(SRS) inline; the consolidated matrices are clause <7> and RD-9.d. (units & conventions). SI / radiometric physical units are stated per data item; variable, band, dimension and field naming follow the EOPF data-model / PSFD conventions (SRS REQ-I-07). All product and ADF locations are URIs (
AnyPath, location-transparent local FS / POSIX / S3); private inputs are referenced, never embedded (IRD REQ-IF-SEC-02).
<5.2> Interface requirements#
This clause lists and describes the software item’s external interfaces (Annex E <5.2>a) and the
controlled requirements imposed on each; the field-level design of every interface is in <5.3>.
Per Annex E <5.2>b the three mandated interface classes are covered: software-to-software
(ICD-IF-SW, ICD-IF-TRIG), software-to-hardware (none — see ICD-IF-SW-05) and the man–machine
interface (ICD-IF-HMI, ICD-IF-DIAG). The Annex E <5.2>b.4 aspects — database structure, logical
interface architecture, signal, communication protocol, timing, behaviour-in-error, observable data —
are covered within the relevant interface (timing/signal: not applicable, no real-time/HW interface,
SDD <5.2>d).
<5.2.1> External interface inventory#
Interface id |
Name |
Direction |
Class |
Realises (IRD) |
Design |
Design component (SDD) |
|---|---|---|---|---|---|---|
|
Downlinked RAW |
in (E1→) |
data |
REQ-IF-IN-L0-01..03 |
<5.3.1> |
C-PU-L0, C-COM-IO |
|
Calibration / auxiliary ADF input |
in (E2→) |
data |
REQ-IF-IN-ADF-01..04 |
<5.3.2> |
C-COM-ADF |
|
|
out (→E5) |
data |
REQ-IF-OUT-01..04, REQ-IF-CAP-03 |
<5.3.3> |
C-COM-PRODUCT, C-COM-PROV |
|
EOPF CPM software interface (PU / product / store) |
host (E6) |
sw-to-sw |
REQ-IF-SW-01..04 |
<5.3.4> |
C-PU-*.unit, C-COM-PRODUCT |
|
Triggering payload (job order) |
in (E4→) |
sw-to-sw / comms |
REQ-IF-COM-01..03, REQ-IF-CAP-01 |
<5.3.5> |
C-COM-ORC, C-COM-CONFIG |
|
Sensor-profile / configuration |
in (E3→) |
data / adaptation |
REQ-IF-AD-01..04 |
<5.3.6> |
C-SENSORS, C-COM-PROFILE |
|
Completion status & structured diagnostics |
out (→E4) |
observable / MMI |
REQ-IF-CAP-05 |
<5.3.7> |
C-COM-ORC, C-COM-CLI |
|
Non-interactive CLI / programmatic entry |
in/out |
MMI |
REQ-IF-HMI-01 |
<5.3.7> |
C-COM-CLI |
<5.2.2> Controlled interface requirements#
ICD-IF-L0-01 — The
L0cinput shall be presented to the chain, after decode, as anL1AEOProductDataTree with the group layout of <5.3.1>; the decoder is profile-bound (sensor packetisation is private,[TBC@impl]). Trace: REQ-IF-IN-L0-01, REQ-F-L0-01/04. Verify: I, T.ICD-IF-L0-02 — The
L0cinput shall carry, or be accompanied by, the selection metadata of <5.3.1> table B (sensor id, acquisition time, instrument mode) enabling profile/ADF resolution. Trace: REQ-IF-IN-L0-02, REQ-F-L0-03. Verify: I, T.ICD-IF-L0-03 — The
L0csource shall be opened read-only by the reader (OpeningMode.OPEN); no write path to the L0 location shall exist. Trace: REQ-IF-IN-L0-03, REQ-F-L0-05. Verify: A, I.ICD-IF-ADF-01 — Each calibration input shall be supplied as a CPM
AuxiliaryDataFile(name,pathURI,store_params) with the content of <5.3.2>; gain/offset, dark and flat-field are mandatory, the PSF/MTF kernel (psf,DPM-ADF-PSF) is mandatory to the enhancement unit (C-PU-ENH), and the rest are profile-selected. Trace: REQ-IF-IN-ADF-01, REQ-F-RAD-01/02, REQ-F-TOA-01, REQ-F-ENH-02. Verify: I, T.ICD-IF-ADF-02 — Each ADF shall carry id, version and validity (sensor/profile, time range) in the attributes of <5.3.2> table B so the correct ADF is selectable for a given
L0c. Trace: REQ-IF-IN-ADF-02, REQ-S-04. Verify: I, T.ICD-IF-ADF-03 — ADF content shall be referenced by runtime URI only; no ADF content shall be committed to the repository or required by public CI. Trace: REQ-IF-IN-ADF-03, REQ-S-01. Verify: I, A.
ICD-IF-ADF-04 — ADFs shall be opened read-only. Trace: REQ-IF-IN-ADF-04. Verify: A, I.
ICD-IF-OUT-01 — Outputs shall be cloud-native Zarr
EOProducts written viaEOZarrStore, with the DataTree of <5.3.3> (measurements/conditions/quality+ metadata) and mandatory root fieldmeasurements. Trace: REQ-IF-OUT-01/02, REQ-F-PRD-01. Verify: T, I.ICD-IF-OUT-02 — Each output shall be chunked per <5.3.3> table C and writable to POSIX and S3-compatible backends via the EOPF store abstraction. Trace: REQ-IF-OUT-03, REQ-F-ORC-02. Verify: T.
ICD-IF-OUT-03 — Each output shall carry the product+provenance metadata of <5.3.3> table D (product id, baseline/version, input/ADF/profile ids+versions, parameters, timestamp) and no private calibration coefficients. Trace: REQ-IF-OUT-04, REQ-IF-CAP-03, REQ-F-PRD-02, REQ-S-05. Verify: I, T.
ICD-IF-SW-01 — Each stage shall be an
EOProcessingUnitsubclass with the class attributes andrun(...)signature of <5.3.4>A and a computing-model declaration (CPMEOProcessingModelschema, <5.3.4>D) declaring its per-mode inputs/ADFs/outputs/parameters. Trace: REQ-IF-SW-01, REQ-F-ORC-01. Verify: R, I.ICD-IF-SW-02 — Products shall be exchanged as
EOProduct/EOGroup(MappingDataType) and persisted only viaEOZarrStore; no product I/O outside these abstractions. Trace: REQ-IF-SW-02, REQ-D-06. Verify: R, T.ICD-IF-SW-03 — The software interfaces shall bind to
eopf == 2.8.1. Trace: REQ-IF-SW-03, REQ-R-03. Verify: I.ICD-IF-SW-04 — Each stage’s pure algorithmic core shall be callable without the CPM runtime through the plain signatures of <5.3.4>C (SDD
IF-CORE-01). Trace: REQ-IF-SW-04, REQ-D-03. Verify: T.ICD-IF-SW-05 — No software-to-hardware interface exists; compute/storage are reached only via the OS and the EOPF store abstraction (closes Annex E <5.2>b.2 / <5.3>c.2). Trace: REQ-IF-HW-01, REQ-R-02. Verify: I.
ICD-IF-TRIG-01 — A run shall be fully specified by the CPM triggering payload of <5.3.5> (
workflow+io+ optionalbreakpoints/dask_context/general_configuration/…), declaring inputs, ADFs, output target, profile selection and parameters. Trace: REQ-IF-COM-01, REQ-I-05, REQ-O-01. Verify: I, T.ICD-IF-TRIG-02 — All payload product/ADF/output references shall be URIs (
AnyPath) resolved through the EOPF store/mapper, location-transparent local/remote. Trace: REQ-IF-COM-02, REQ-I-06. Verify: T.ICD-IF-TRIG-03 — A local-filesystem payload variant (
store_type: zarr, local paths, nodask_context/S3) shall run on the CI shell runner. Trace: REQ-IF-COM-03, REQ-PORT-03. Verify: T.ICD-IF-TRIG-04 — Chain start/stop at level breakpoints shall be controlled via the payload
workflow(active/step) andbreakpointssections of <5.3.5>. Trace: REQ-IF-CAP-01, REQ-F-ORC-01. Verify: T.ICD-IF-PROF-01 — The sensor profile shall be a versioned JSON object validated on load against the schema of <5.3.6>; it shall externalise all sensor-specific interface content. Trace: REQ-IF-AD-01/04, REQ-AD-01, REQ-DAT-03. Verify: T, I.
ICD-IF-PROF-02 — The profile shall be uniquely identified and versioned and selectable per run via the payload (
profile_id/profile_versionin the active unitparameters). Trace: REQ-IF-AD-02, REQ-AD-02. Verify: I, T.ICD-IF-DIAG-01 — At its boundary the processor shall expose a completion status and the machine-readable diagnostics/QA structure of <5.3.7>. Trace: REQ-IF-CAP-05, REQ-O-03. Verify: T.
ICD-IF-HMI-01 — Human interaction shall be limited to the CLI of <5.3.7>, configuration files and logs/reports; no GUI. Trace: REQ-IF-HMI-01, REQ-HF-01. Verify: I, T.
Error behaviour (Annex E <5.2>b.4). All interfaces obey the fail-stop policy (SRS REQ-F-DEP-01, SDD <5.2>g) surfaced through
ICD-IF-DIAG: on any stage error the run exits non-zero, withholds/flags affected outputs and publishes no partial product as complete. The CPM error policy isFAIL_FAST(triggering__error_policy, <5.3.5>); the typed fault vocabulary is theMsiProcessorErrorhierarchy (SDD <5.4.1>).
<5.3> Interface design#
Per Annex E <5.3>a/d each interface definition gives at least the provided service, the
description (name, type, dimension), the range and the initial/default value; per Annex E
<5.3>e data items are organised as (Name, description, unique id (path/key), source→destination, unit,
limit/range, accuracy/precision where applicable, legality checks, data type, data representation).
External interfaces are also expressed as models where appropriate (Annex E <5.3>b: the Zarr
DataTree, the profile JSON-Schema, the CPM payload/computing-model schemas). Sensor-private numbers are
parameterised by the profile (<5.3.6>); only encodings that are genuinely private remain [TBC@impl].
<5.3.1> ICD-IF-L0 — Downlinked RAW L0c input product#
Provided service. Read-only ingestion of one downlinked RAW MSI acquisition (E1), decoded into an
L1A EOProduct DataTree in focal-plane geometry. The decode step is realised by a profile-registered
CPM store/reader (store_type selected in the payload, <5.3.5>) wrapped by C-PU-L0; the on-wire
packetisation/bit-codec is sensor-private (NDA) and [TBC@impl] — only the post-decode
interface is controlled here (SDD <5.4.2>; IF-PROD-01).
A. L1A DataTree after decode (source: E1 / L0 reader → destination: radiometric PU):
Path (item id) |
Description |
Type |
Dimension |
Range |
Data representation |
|---|---|---|---|---|---|
|
Raw detector samples, focal-plane geometry, per band |
|
|
|
Zarr/ |
|
Per-line acquisition timestamp |
|
|
UTC since epoch |
CF |
|
Platform ephemeris (orbit) ancillary |
|
|
ECEF m, m/s |
CF |
|
Platform attitude ancillary |
|
|
unit quaternion |
CF |
|
Initial QA (lost-packet / line-loss / fill) |
|
|
bit-flags (table <5.3.3>E) |
Zarr |
Number of bands
N_b, detectors/line, per-bandline_factorandbit_depthare profile parameters (PROF:bands,PROF:focal_plane;DPM-PRM-GEN-01,DPM-PRM-L0-01). Heritage note (RD-10): one band may be acquired at a 2× line rate — represented by a per-bandline_factorin the profile, not hard-coded.Legality checks (REQ-F-L0-03): mandatory groups present; band set equals the profile band list; dims consistent with the profile focal-plane geometry; selection metadata (table B) present — enforced by
l0_decode.core.check_legalityraisingInputValidationError(SDD <5.4.2>).Lost-packet detection (REQ-F-L0-02, heritage): an all-zero line following a non-zero line marks a line-loss; affected lines are truncated and flagged
LOST_PACKETin/quality/l0_flags. Non-zero corruption / CRC handling beyond the zero-line rule is[TBC@impl](ATBD <5.1>).
B. L0 selection metadata (drives profile/ADF resolution; source: E1 → destination: orchestrator/PU):
Item id |
Description |
Type |
Legality check |
|---|---|---|---|
|
Instrument/sensor identifier |
str |
matches a known profile |
|
Acquisition start (UTC) |
ISO-8601 str |
within an ADF validity range |
|
Instrument mode/configuration |
str |
in profile |
|
Unique L0 identifier |
str |
non-empty; recorded in provenance |
<5.3.2> ICD-IF-ADF — Calibration / auxiliary ADF input#
Provided service. Read-only supply of private instrument-calibration and auxiliary data as CPM
AuxiliaryDataFiles. Each ADF is passed to a PU in the adfs mapping keyed by the PU’s declared ADF
name. The 2.8.1 dataclass (confirmed, eopf/computing/abstract.py) is
AuxiliaryDataFile(name: str, path: str | AnyPath, store_params: dict | None = None, data_ptr: Any = None)
with a read-only .path -> AnyPath property; store_params may carry storage_options (e.g. S3
credentials). Storage form is cloud-native Zarr (preferred, PSFD ADF convention) or
NetCDF/GeoTIFF where heritage dictates; the content schema below is normative, the on-disk container
choice is a per-profile/heritage detail ([TBC@impl] only for the legacy container).
A. ADF content by type (source: E2 → destination: the named PU; SDD IF-SVC-02):
ADF key (item id) |
Mandatory |
Content / variables |
Type · dim |
Used by (SRS) |
|---|---|---|---|---|
|
yes |
per-band absolute gain |
float32 · |
REQ-F-TOA-01: |
|
yes |
per-band dark/offset (DSNU) reference frame(s) |
float32 · |
REQ-F-RAD-01 |
|
yes |
per-band flat-field (PRNU) reference |
float32 · |
REQ-F-RAD-02 (NUC) |
|
derived |
per-band NUC |
float32 · |
REQ-F-RAD-02/05 |
|
profile |
per-band defective-detector mask |
uint8/bool · |
REQ-F-RAD-03 |
|
profile |
per-band centre wavelength, ESUN, SRF ref |
float32 · |
REQ-F-TOA-02 |
|
profile |
viewing/geometric model coefficients (form sensor-private) |
model · |
REQ-F-GEO-01 |
|
profile |
digital elevation model |
int16/float32 · |
REQ-F-GEO-02 |
|
profile |
ground-control / reference points |
table |
REQ-F-GEO-02 |
|
profile |
AOT, water vapour, atmos-model params / RT-LUT |
float32 · grid/scalar/LUT |
REQ-F-ATM-01/02 |
|
yes (ENH) |
per-band PSF/MTF kernel (focal-plane geometry), normalised to unit DC gain (Σ=1) |
float32 · 2-D |
REQ-F-ENH-02: MTF compensation (PSF deconvolution), C-PU-ENH |
NUC derivation (heritage RD-10, REQ-F-RAD-05), informative: column-mean dark/flat →
gain = (mean(flat) − mean(dark)) / (flat − dark),offset = mean(flat) − gain·flat; detectors with out-of-range gain are flagged intobadpixel. The algorithm basis is the DPM (RD-6) / ATBD (RD-7) (ALG-RAD-NUC); the SDD core isradiometric.core.estimate_nuc(SDD <5.4.3>).PSF/MTF kernel (
psf, DPMDPM-ADF-PSF), mandatory to the enhancement unit: per-band 2-D kernels in focal-plane geometry, float32, normalised to unit DC gain (Σ=1) so radiometry is preserved; consumed by the mandatory MTF-compensation (PSF-deconvolution) sub-step of the enhancement stage (C-PU-ENH /ALG-ENH-DECONV, REQ-F-ENH-02). The kernel is per-band private calibration data referenced by ADF URI only (values not reproduced here) and opened read-only (ICD-IF-ADF-03/04).Change note (CR-3): the PSF/MTF kernel is declared here as the
psfcalibration ADF (DPM-ADF-PSF), having moved from an enhancement parameter to per-band calibration data; the optionaldarkADF (fft_dark only) is retained.
B. ADF identification & validity attributes (every ADF; legality-checked on selection by
C-COM-ADF.check_validity, SDD <5.4.11>):
Item id (attr) |
Description |
Type |
Legality check |
|---|---|---|---|
|
Unique ADF identifier |
str |
non-empty; recorded in provenance |
|
ADF version |
str (SemVer) |
non-empty |
|
Applicability |
str |
equals the run’s sensor/profile |
|
Applicability time range |
ISO-8601 |
spans |
|
One of the keys in table A |
enum |
in table A |
Access: opened read-only (
ICD-IF-ADF-04); referenced by URI only, never committed (ICD-IF-ADF-03); selection mismatch ⇒AdfResolutionError(REQ-S-04, SDD <5.4.1>).
<5.3.3> ICD-IF-OUT — L1B/L1C/L2A Zarr EOProduct output#
Provided service. Self-describing cloud-native Zarr EOProduct written via EOZarrStore,
consumable by E5. The product is an EOGroup tree whose mandatory root field is measurements
(confirmed CPM EOProduct.MANDATORY_FIELD = ("measurements",)). The level determines the
measurements content; conditions, quality and the metadata are common (SDD C-COM-PRODUCT,
IF-PROD-03/04/05).
A. EOProduct DataTree (group hierarchy) (source: writer PU → destination: E5):
/ EOProduct root (attrs: STAC + provenance, table D)
├── measurements/ (mandatory; EOProduct.MANDATORY_FIELD)
│ ├── radiance/<band> L1B: TOA spectral radiance [L1B]
│ └── reflectance/<band> L1B TOA refl. / L1C ortho TOA / L2A BOA [L1B|L1C|L2A]
├── conditions/
│ ├── geometry/{sun_zenith,sun_azimuth,view_zenith,view_azimuth} angles
│ ├── geolocation/{x,y | longitude,latitude} + spatial_ref(grid_mapping) [L1C+]
│ └── meteorology/{aot,water_vapour} [L2A]
└── quality/
├── mask/<band> per-pixel QA bit-flags (table E)
├── scene_classification [L2A] scene class map
└── metrics per-band/stage QA metrics (SNR,RMSE,PSNR,MSE,variance)
B. Measurement variable definition (per band; provided service: calibrated band raster):
Property |
Value |
|---|---|
Name / id |
`OUT:/measurements/{radiance |
Type |
|
Dimension |
|
Data type |
reflectance |
Range |
radiance ≥ 0 (sensor units); reflectance |
|
sensor no-data sentinel ( |
Initial value |
n/a (computed) |
Legality / accuracy |
per-profile |
C. Chunking & CRS encoding (REQ-IF-OUT-03 / REQ-F-ORC-02; SDD C-COM-CHUNK):
Item |
Definition |
|---|---|
Chunking |
per-variable Zarr chunks |
Compression |
Zarr codec from payload |
CRS encoding |
|
Grid |
output CRS, grid origin and |
D. Product & provenance metadata (root attrs; source: writer PU via C-COM-PROV;
REQ-IF-CAP-03 / REQ-F-PRD-02):
Item id (attr) |
Description |
Representation |
Legality |
|---|---|---|---|
|
Unique output product identifier |
str |
non-empty, unique |
|
|
enum |
matches PU |
|
Processor + baseline version |
str (SemVer) |
present |
|
Source product id(s) |
list[str] |
present |
|
ADF id(s)+version(s) used |
list[str] |
present; no coefficients (REQ-S-05) |
|
Sensor profile id+version |
str |
present |
|
Effective parameters |
JSON object |
present |
|
UTC timestamp |
ISO-8601 |
present |
|
CPM-appended history entry (processor/version/level) |
list[obj] |
appended by |
STAC properties |
|
STAC/CF |
GeoZarr/STAC valid ( |
CPM history note (confirmed 2.8.1). When invoked through
run_validating(...), the CPM appends a processing-history entry (processor name/version/level) to the output product attrs and validates the product (validate(validation_mode)/is_valid(validation_mode)); the project-level provenance of table D is additional and authored byC-COM-PROV.
E. Per-pixel QA flag bit definition (quality/mask/<band>, uint16 bitmask; propagated across all
stages by C-COM-QAFLAG, monotone OR-accumulation, REQ-F-QA-02; mirrors SDD QAFlag):
Bit |
Flag (SDD |
Set by |
|---|---|---|
0 |
|
L0, all stages |
1 |
|
L0 (REQ-F-L0-02) |
2 |
|
radiometric (REQ-F-RAD-04) |
3 |
|
radiometric (REQ-F-RAD-03) |
4 |
|
coregistration (REQ-F-COR-03) |
5 |
|
atmospheric (REQ-F-ATM-03) |
6 |
|
atmospheric (REQ-F-ATM-03) |
7 |
reserved |
|
<5.3.4> ICD-IF-SW — EOPF CPM software interface#
Provided service. The host CPM contracts (E6) the software realises. Re-confirmed verbatim
against the installed eopf == 2.8.1 (see <2> note); each stage is realised as a thin
EOProcessingUnit Wrapper over a pure Core (SDD <5.4.1>, IF-CORE-01).
A. EOProcessingUnit (stage) contract — each stage subclasses eopf.computing.EOProcessingUnit:
Element |
Definition (confirmed 2.8.1) |
|---|---|
Class attributes |
|
Computing model |
|
Core method |
|
Validated entry |
|
Modes |
|
Mandatory decls |
|
inputskeys = the PU’s declared input names; values areDataType(or an iterable ofDataType).adfskeys = the PU’s declared ADF names; values areAuxiliaryDataFile.**kwargs= the run parameters (sourced from the payload unitparameters, <5.3.5>).Return =
MappingDataTypeof output products keyed by the PU’s declared output names.
B. Product & store contracts (confirmed 2.8.1):
Element |
Definition |
|---|---|
|
dict-like product tree; |
|
holds sub-groups and |
|
|
|
dataclass |
|
|
|
|
C. Pure algorithmic-core signatures (callable without CPM, REQ-IF-SW-04 / REQ-D-03; SDD
IF-CORE-01; the PU is a thin adapter). The authoritative per-stage core signatures are in SDD <5.4.2>–
<5.4.10>; indicative cores:
Core (module, SDD) |
Signature (indicative) |
Returns |
|---|---|---|
radiometric ( |
|
corrected DN / NUC vectors |
toa ( |
|
radiance / reflectance |
coregistration ( |
|
aligned stack + residual QA |
georeference ( |
|
gridded product |
atmospheric ( |
|
BOA reflectance + masks |
qa ( |
|
metrics |
D. Computing-model declaration (CPM EOProcessingModel schema — confirmed 2.8.1). Each PU’s
computing model conforms to the eopf.computing.validation.EOProcessingModel schema, not a flat
input/output list. The schema and a worked example for msi_l0_decode:
// CPM EOProcessingModel: per-mode declaration. Dict keys MAY be regex.
{
"available_modes": ["nominal"], // all supported modes
"default_mode": "nominal", // must be in available_modes
"modes_config": { // one config per mode
"nominal": {
"inputs": { "l0": { "type": "product", "spec": { /* EOProductModel */ }, "iterable_allowed": false } },
"adfs": { }, // {<name>: {"spec": {"required": <bool>}}}
"outputs": { "l1a": { "type": "product", "spec": { /* EOProductModel */ } } },
"parameters": { // optional **kwargs; {"required": <bool>}
"profile_id": { "required": true },
"profile_version": { "required": true },
"bit_depth": { "required": false },
"line_factor": { "required": false }
}
}
}
}
inputs/outputsentries areDataTypeModel(type ∈ {product, container}, a typedspec(EOProductModel/EOContainerModel),iterable_allowed: bool);adfsentries carryspec.required;parametersentries carryrequired. The CPM validates a workflow unit’s wiring (mandatory inputs/ADFs present, kwargs valid) against this model (validate_run_parameters,validate_output_models). The per-PU logical declarations of the SDD (<5.4.2>–<5.4.10>: inputs/adfs/outputs/parameters/modes) map ontomodes_config[<mode>]of this schema; themandatoryflags map tospec.required(adfs) and to the input/output key being present in the mode config.
E. CPM entry points (confirmed 2.8.1).
Element |
Definition |
|---|---|
Console (CPM) |
|
Programmatic |
|
Project wrapper |
|
<5.3.5> ICD-IF-TRIG — Triggering payload (job order)#
Provided service. A single CPM payload (YAML or JSON; source: E4 → destination: CPM runner /
C-COM-ORC) fully specifies a run. Recognised top-level section keys (confirmed 2.8.1,
eopf/triggering/parsers.py; #opt may be absent):
Key |
Required |
Content |
|---|---|---|
|
yes |
ordered list of PU units to run (table A) — the acyclic stage graph |
|
yes |
|
|
no |
|
|
no |
|
|
no |
|
|
no |
logging conf files, config files, secret files, env files, dynamic-import modules, quality-control conf |
A. workflow[] unit (WorkflowRaw → WorkFlowUnitDescription; confirmed 2.8.1):
Field |
Type |
Default |
Meaning |
Legality |
|---|---|---|---|---|
|
str |
— (req) |
unique unit name in the run |
unique among active units |
|
str |
— (req) |
Python module exporting the PU |
importable |
|
str |
— (req) |
|
found in |
|
map |
|
|
mandatory inputs satisfied ( |
|
map |
|
|
mandatory ADFs present ( |
|
map |
|
|
— |
|
map |
|
passed as |
per PU computing model |
|
int |
|
execution-order index in the DAG |
— |
|
bool |
|
run this unit (inactive ⇒ skipped) |
drives sub-chain selection |
|
bool |
|
validate outputs when |
— |
|
str | null |
|
PU mode (defaults to |
in |
B. io.input_products[] (EOInputRaw) / io.adfs[] (EOADFRaw) (source: E1/E2; confirmed 2.8.1):
Field |
input_products |
adfs |
Notes |
|---|---|---|---|
|
yes |
yes |
referenced by |
|
yes (URI) |
yes (URI) |
local FS or |
|
yes |
— (n/a) |
|
|
no ( |
no ( |
|
|
|
— |
input resolution mode |
C. io.output_products[] (EOOutputRaw) (destination: E5; confirmed 2.8.1):
Field |
Value / range |
|---|---|
|
referenced by |
|
output URI (local FS / |
|
|
|
|
|
|
|
e.g. |
|
bool (default |
Profile selection (resolved). Carried as the
profile_id/profile_versionentries in the relevant active unit’sparametersmap (passed as**kwargstorun(...), SDD <5.4.2>); the profile JSON itself is referenced via aconfigfile or a profile URI parameter and loaded/validated byC-COM-PROFILE. Resolves the prior PDR placement[TBC](ICD-IF-PROF-02).Error policy / validation (resolved).
general_configuration.triggering__error_policy∈{FAIL_FAST, FAIL_ON_CRITICAL, BEST_EFFORT};msi-processormandatesFAIL_FAST(REQ-F-DEP-01).triggering__validate_runenables per-unit output validation attriggering__validate_mode(STRUCTURE/STAC/NONE).CI/local variant (
ICD-IF-TRIG-03):store_type: zarr, localpaths, nodask_context(the CPM defaults to a null context), nosecret/S3 — runnable on the shell runner.
<5.3.6> ICD-IF-PROF — Sensor-profile / configuration#
Provided service. A versioned JSON object (source: E3) that specialises the generic chain;
validated on load by C-COM-PROFILE (ICD-IF-PROF-01, REQ-DAT-03, SDD <5.4.11>/<5.4.12>).
JSON-Schema controlled; invalid/incomplete ⇒ ProfileValidationError with diagnostic. Top-level schema
(field id = dotted path):
Field (id) |
Type |
Description |
Legality |
|---|---|---|---|
|
str |
unique profile identifier |
required, unique |
|
str (SemVer) |
profile version |
required |
|
str |
instrument id (matches L0 |
required |
|
list[str] |
supported instrument modes |
non-empty |
|
list[obj] |
|
≥1; names unique |
|
obj |
|
bit_depth>0 |
|
obj |
default ADF ids per type ( |
mandatory types bound |
|
obj |
dark/NUC thresholds ( |
— |
|
obj |
|
method ∈ allowed set |
|
obj |
|
reference_band ∈ bands |
|
obj |
|
valid CRS |
|
obj |
|
if enabled, pan_band set |
|
obj |
|
mode ∈ enum |
|
obj |
|
chunking>0 |
|
obj |
per-stage enable flags ( |
bool |
|
obj |
default breakpoint levels |
— |
Adaptation rule (REQ-IF-AD-01 / REQ-AD-01): the processing core reads all sensor-specific data from this profile; adding a sensor = adding a profile (+ private ADFs), no core change (REQ-D-07). First instantiated profile = the project owner’s sensor (REQ-AD-02); its numeric content is private.
The typed
Profile(SDDC-COM-PROFILE.Profile) is the in-code mirror of this schema; the JSON-Schema file (sensors/profile.schema.json) is delivered as configuration (REQ-DEL-03), id/ version tracked with the profile version.
<5.3.7> ICD-IF-DIAG & ICD-IF-HMI — status, diagnostics and CLI (MMI)#
Provided service (ICD-IF-DIAG). Machine-readable run outcome surfaced to E4 (SDD
C-COM-ORC.RunResult):
Item id |
Description |
Representation |
Range |
|---|---|---|---|
exit status |
process completion |
int |
|
structured diagnostics |
errors/warnings |
JSON log records |
per logging conf |
processing report |
parameters, ADF/profile ids, per-product QA summary |
JSON + human-readable |
present on every run (REQ-O-02) |
per-product QA |
flags/metrics carried in |
in product |
— |
Provided service (ICD-IF-HMI). Non-interactive entry points (no GUI, REQ-HF-01):
Item id |
Description |
Representation |
|---|---|---|
CLI (project) |
|
argv |
Pipeline driver |
|
argv |
CLI (CPM) |
|
argv |
Python API |
|
function call |
inputs |
triggering payload (<5.3.5>) + config/profile files |
files/URIs |
outputs |
exit status + diagnostics ( |
files/streams |
<6> Validation requirements#
Approach (Annex E <6>a). Each uniquely identified interface requirement of <5.2> carries an inline
Verify: method (T/A/I/R). Validation reuses the V&V process of SRS <6> / RD-8: schema and structure
checks by inspection/review against this ICD and the PSFD; automated tests (pytest, CPM
run_validating / validate(validation_mode)) in public CI for everything not needing private data, a
container runtime, a Dask gateway or S3; and local tests on real data/ADFs for the field-level
content of the private interfaces. Interfaces needing those services or private data run non-blocking in
CI and are validated locally (REQ-PORT-03). The detailed interface→test-case trace is maintained in RD-9.
Validation matrix (Annex E <6>b — requirement → method → means).
Interface req. |
Method |
Means / milestone |
|---|---|---|
ICD-IF-L0-01 |
I, T |
Decode a sample |
ICD-IF-L0-02 |
I, T |
Inspect selection metadata; resolve profile/ADF from a sample |
ICD-IF-L0-03 |
A, I |
Static analysis / inspection: no write path to L0 ( |
ICD-IF-ADF-01 |
I, T |
Load gain/offset, dark, flat-field, |
ICD-IF-ADF-02 |
I, T |
Inspect id/version/validity attrs; select valid ADF for a given |
ICD-IF-ADF-03 |
I, A |
Repo/CI scan: no ADF content committed; runtime-URI resolution |
ICD-IF-ADF-04 |
A, I |
Static analysis / inspection: ADFs opened read-only |
ICD-IF-OUT-01 |
T, I |
Write |
ICD-IF-OUT-02 |
T |
Chunked partial read; write POSIX (CI) + S3 (when available) |
ICD-IF-OUT-03 |
I, T |
Inspect provenance/STAC attrs vs <5.3.3>D; assert no private coefficients |
ICD-IF-SW-01 |
R, I |
Review PU class attrs + |
ICD-IF-SW-02 |
R, T |
Round-trip |
ICD-IF-SW-03 |
I |
Inspect pinned |
ICD-IF-SW-04 |
T |
Unit-test pure cores without CPM runtime (<5.3.4>C, |
ICD-IF-SW-05 |
I |
Inspection: no hardware interface (N/A closure) |
ICD-IF-TRIG-01 |
I, T |
Inspect payload vs <5.3.5>; trigger a run from a sample payload |
ICD-IF-TRIG-02 |
T |
Trigger with local + remote (when available) URIs |
ICD-IF-TRIG-03 |
T |
Trigger + verify on the CI shell runner (local-FS variant) |
ICD-IF-TRIG-04 |
T |
Sub-chain run via |
ICD-IF-PROF-01 |
T, I |
Load valid + invalid profile vs JSON-Schema; reject-with-diagnostic test |
ICD-IF-PROF-02 |
I, T |
Inspect id/version; select profile per run via payload |
ICD-IF-DIAG-01 |
T |
Assert exit status + diagnostics/QA on success and forced failure |
ICD-IF-HMI-01 |
I, T |
Invoke via CLI/API; confirm no GUI dependency |
Requirements not validated against the baseline. ICD-IF-SW-05 (no hardware interface) is an N/A
closure verified by inspection. Field-level items still flagged [TBC@impl] (the sensor-private L0
on-wire codec, the viewing-model coefficient form, the legacy ADF container, QA bit 7) are confirmed
against real data during implementation (SDP WP-5), not at CDR; the CPM-surface items previously flagged
[TBC@CDR] are resolved at this issue against the installed eopf == 2.8.1 (clause <2> note).
<7> Traceability#
Per Annex E <7>a the forward and backward interface traceability is reported below; the authoritative,
tool-maintained matrix (incl. trace to design C-*/IF-* and test cases) is RD-9 (Annex E <7>b — DJF
reference).
<7.1> Forward — upper-level interface req. (IRD REQ-IF-*) → ICD interfaces#
IRD |
Realised by |
|---|---|
REQ-IF-CAP-01 |
ICD-IF-TRIG-01/04, ICD-IF-SW-01 |
REQ-IF-CAP-02 |
ICD-IF-OUT-02, ICD-IF-SW-02 |
REQ-IF-CAP-03 |
ICD-IF-OUT-03 |
REQ-IF-CAP-04 |
ICD-IF-TRIG-01, ICD-IF-SW-01 |
REQ-IF-CAP-05 |
ICD-IF-DIAG-01 |
REQ-IF-IN-L0-01..03 |
ICD-IF-L0-01/02/03 |
REQ-IF-IN-ADF-01..04 |
ICD-IF-ADF-01/02/03/04 |
REQ-IF-OUT-01..04 |
ICD-IF-OUT-01/02/03, ICD-IF-SW-02 |
REQ-IF-SW-01..04 |
ICD-IF-SW-01/02/03/04 |
REQ-IF-COM-01..03 |
ICD-IF-TRIG-01/02/03 |
REQ-IF-HW-01 |
ICD-IF-SW-05 |
REQ-IF-HMI-01 |
ICD-IF-HMI-01 |
REQ-IF-SEC-01..03 |
ICD-IF-ADF-02/03/04, ICD-IF-L0-03, ICD-IF-OUT-03 |
REQ-IF-AD-01..04 |
ICD-IF-PROF-01/02 |
<7.2> Backward — ICD interfaces → upper-level (IRD REQ-IF-* / SRS REQ-*)#
ICD interface |
Parent IRD |
Parent SRS |
|---|---|---|
ICD-IF-L0-* |
REQ-IF-IN-L0-01..03 |
REQ-F-L0-01..05, REQ-I-03 |
ICD-IF-ADF-* |
REQ-IF-IN-ADF-01..04, REQ-IF-SEC-01/02 |
REQ-F-RAD-01..05, REQ-F-TOA-01, REQ-F-ENH-02, REQ-F-ATM-01, REQ-DAT-02, REQ-S-04, REQ-M-02 |
ICD-IF-OUT-* |
REQ-IF-OUT-01..04, REQ-IF-CAP-03 |
REQ-F-PRD-01/02, REQ-F-QA-02, REQ-I-04, REQ-DAT-01, REQ-D-09 |
ICD-IF-SW-* |
REQ-IF-SW-01..04, REQ-IF-HW-01 |
REQ-D-01/03/06, REQ-F-ORC-01, REQ-R-02/03 |
ICD-IF-TRIG-* |
REQ-IF-COM-01..03, REQ-IF-CAP-01 |
REQ-I-05/06, REQ-O-01, REQ-F-ORC-01, REQ-PORT-03 |
ICD-IF-PROF-* |
REQ-IF-AD-01..04 |
REQ-AD-01/02, REQ-DAT-03, REQ-D-07 |
ICD-IF-DIAG-* |
REQ-IF-CAP-05 |
REQ-O-03, REQ-F-DEP-01 |
ICD-IF-HMI-* |
REQ-IF-HMI-01 |
REQ-HF-01, REQ-I-02 |
The traceability of clause <7> is consolidated and kept current in RD-9 (
compliance/traceability/); this satisfies Annex E <7>b. The downward trace to the design components (C-*) and internal interfaces (IF-*) is in SDD <6> (RD-5).
End of ICD (final / CDR issue). Authored per ECSS-E-ST-40C Rev.1 Annex E. This issue finalises the
field-level interface design at CDR, superseding the PDR preliminary issue. CPM-surface items are
resolved against the installed eopf == 2.8.1; only genuinely sensor-private encodings remain
[TBC@impl] (confirmed during implementation, SDP WP-5). The EOPF PSFD is the normative
product-structure reference. Upstream interface requirements are in the IRD (RD-3) and SRS (RD-4); the
detailed design is in the SDD (RD-5); the maintained traceability matrix is RD-9.