Software Design Document (SDD) — Detailed (CDR)#
Field |
Value |
|---|---|
Document |
SDD — Software Design Document (DETAILED / element-level) |
DRD ref |
ECSS-E-ST-40C Rev.1, Annex F |
Container |
Design Definition File (DDF) — |
Project |
|
Software criticality |
Category C (ECSS-Q-ST-80C Rev.2 / ECSS-E-ST-40C Annex R) |
Baselined at |
CDR (detailed per-component design + internal-interface data; supersedes the PDR architectural issue) |
Status |
Draft for CDR |
Detailed issue (CDR). This is the detailed issue of the SDD, the design constituent of the DDF (Annex F, F.1.2). It supersedes the PDR architectural issue: clauses <1>–<4> and <5.1>–<5.3> retain (and where needed refine) the baselined architecture; clause <5.4> is expanded from architectural granularity to the element-level detailed design required by Annex F <5.4.2>–<5.4.11> (component identifier/type/purpose/function/subordinates/dependencies/interfaces/resources/references/ data, with data-element dtype/dimension/range/initial value); clause <5.5> gives the full internal-interface data design (Annex F <5.5>c/e), the file-level data structures previously postponed (<5.5>d); clause <6> closes the requirements→design traceability and the critical-component measures for the DJF. Per the SDP (RD-1) §5.2.3/§5.5.2 the full SDD is delivered at CDR; implementation (SDP WP-5) starts only after CDR. Element-level code internals that are legitimately fixed during implementation (the NDA-bound L0 bit-codec body, the rigorous collinearity geolocation kernel, the down-selected atmospheric RT engine and scene classifier, exact filter/matcher tunings) are specified here at interface + algorithm level and explicitly marked
[impl]where the body is finalised post-CDR. Sensor-private numbers (calibration coefficients, accuracy budgets) stay in the profile/ADFs and are not reproduced (data policy, SRS <5.8>). This SDD does not re-specify requirements (SRS, RD-4) nor algorithm mathematics (DPM RD-6 / ATBD RD-7); it specifies how the software is structured and built to satisfy them, binding each design element to itsREQ-*,DPM-M-*/DPM-*andALG-*.
<1> Introduction#
Purpose. This SDD describes the software design of msi-processor, a generic high-resolution
pushbroom multispectral imager (MSI) ground-segment processor that transforms downlinked RAW
Level-0 (L0c) data into calibrated, orthorectified, atmospherically corrected products up to
Level-2 (L2A). At this (CDR) issue it gives both the architectural design (static, dynamic and
behavioural views; patterns; standards; requirement allocation) and the detailed design (per-component
core signatures and data structures, the EOProcessingUnit wrapper contracts, the CPM computing-model
JSON, error handling and the element-level internal-interface data).
Objective. The SDD is the bridge from the baselined requirements (SRS, RD-4) and processing model
(DPM RD-6 / ATBD RD-7) to the implementation (SDP WP-5) and the unit/integration test design (V&V plan,
RD-8). Every design component traces upward to one or more REQ-* software requirements (clause <6>) and
to the DPM module(s) / ATBD algorithm(s) it realises, and is the unit of work for implementation and unit
test.
Content. Clause <2> lists applicable/reference documents; clause <3> adds design-specific terms. Clause <4> gives the design overview — static architecture (<4.1>), dynamic architecture and computational model (<4.2>), behaviour (<4.3>), interfaces context (<4.4>), long-lifetime design (<4.5>), memory & PU budget (<4.6>), and design standards/conventions/trade-offs (<4.7>). Clause <5> gives the software design proper — general (<5.1>), overall architecture (<5.2>), component design general (<5.3>), per-component detailed design (<5.4>) and internal-interface design (<5.5>). Clause <6> gives the requirements→design traceability and the measures for critical components.
Reason for preparation. msi-processor is an integration and ECSS productisation effort: the
processing algorithms already exist as prior work (RD-9 / SRF RD-10) and the runtime platform is the EOPF
CPM (eopf == 2.8.1). This detailed SDD records, at CDR, the element-level design that turns that heritage
into a sensor-agnostic, configuration-driven, verifiable CPM product, so that implementation can begin
against a fixed, traceable design.
<2> Applicable and reference documents#
Applicable documents (AD)#
Id |
Document |
Reference |
|---|---|---|
AD-1 |
ECSS Space engineering — Software |
ECSS-E-ST-40C Rev.1 (30 April 2025) |
AD-2 |
ECSS Space product assurance — Software product assurance |
ECSS-Q-ST-80C Rev.2 |
AD-3 |
EOPF CPM — Product Structure & Format Definition (PSFD) / common data model |
EOPF CPM docs ( |
Reference documents (RD)#
Id |
Document |
Reference |
|---|---|---|
RD-1 |
|
|
RD-2 |
|
|
RD-3 |
|
|
RD-4 |
|
|
RD-5 |
|
|
RD-6 |
|
|
RD-7 |
|
|
RD-8 |
|
|
RD-9 |
Prior work — multispectral pushbroom preprocessing pipeline ( |
RD-10 |
RD-10 |
|
|
RD-11 |
|
|
RD-12 |
EOPF CPM API documentation ( |
EOPF CPM ( |
RD-13 |
Cloud-native data conventions |
Zarr v2/v3, CF metadata, STAC, GeoZarr |
<3> Terms, definitions and abbreviated terms#
The SSS <3>, IRD <3>, SRS <3>, DPM <3> and ATBD <3> glossaries apply in full. Only design-specific terms not defined there are added here (per Annex F <3>).
Term / abbr. |
Definition |
|---|---|
Processing Unit (PU) |
An EOPF CPM |
Core |
The pure, framework-independent algorithmic function set behind a PU (no CPM dependency); the unit of numerical test ( |
Wrapper / adapter |
The thin |
Computing model (JSON) |
The per-PU CPM declarative model ( |
Triggering payload |
The CPM job order ( |
Profile |
Per-sensor configuration set (schema-validated) specialising the generic chain; data only, no algorithm code |
ADF |
Auxiliary Data File (private instrument calibration / auxiliary input), referenced by URI as a CPM |
|
CPM |
Block / chunk / tile |
A bounded (line- or grid-) block processed independently to bound memory (Zarr/Dask unit); a halo is the overlap added to a tile for spatially-coupled stages |
QAFlag |
The per-pixel quality bit registry ( |
|
Marks an element-level code internal whose body is legitimately finalised during implementation (post-CDR); its interface and algorithm are fixed here |
|
A field whose exact encoding is sensor-private or pinned-CPM-specific and confirmed at CDR finalisation (per ICD <3>) |
DDF / DJF |
Design Definition File / Design Justification File (ECSS-E-ST-40C) |
Component aspect |
The Annex F <5.4> description axes (identifier, type, purpose, function, subordinates, dependencies, interfaces, resources, references, data) |
<4> Software design overview#
This clause introduces the system context and the design before the formal design in clause <5>.
<4.1> Software static architecture#
msi-processor is a batch, non-interactive, single-process (optionally Dask-distributed) Python
library + CLI. Its static architecture is a layered, three-package decomposition over the EOPF
CPM, where the processing chain is a sequence of EOProcessingUnits and every product is an
EOProduct persisted as cloud-native Zarr (REQ-D-01, REQ-F-PRD-01).
The three top-level components and their relationship:
┌─────────────────────────────────────────────────────────────┐
│ msi_processor (item) │
│ │
trigger/payload │ msi_processor.sensors msi_processor.common │
───────────────► │ (C-SENSORS) (C-COMMON) │
(triggering JSON) │ profile schema + per-sensor product/Zarr I/O, ADF access, │
│ profile data + ADF bindings profile loader, provenance, │
│ │ (config, data only) QA-flag registry, chunking, │
│ │ URI/store map, orchestration │
│ ▼ ▲ ▲ ▲ │ │
│ ┌──────────────────────────────────────────────────────┐ │
│ │ msi_processor.computing (C-COMPUTING) │ │
│ │ l0_decode → radiometric → enhancement → toa ───► │ │
│ │ coregistration → georeference → atmospheric ──► │ │
│ │ [pansharpen] (opt, post-L2A) ; qa (cross-cutting) │ │
│ │ each stage = pure Core + thin EOProcessingUnit Wrap │ │
│ └──────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────┘
▲ ADFs (private, by URI) ▼ EOProduct (Zarr) out
msi_processor.computing(C-COMPUTING) — the processing chain. One subpackage per stage; each contains a pure Core (algorithm, no CPM/IO) and a thin Wrapper (theEOProcessingUnit). This is where the heritage algorithms (RD-9) live, realising the DPM modulesDPM-M-*(RD-6) and the ATBD algorithmsALG-*(RD-7).msi_processor.sensors(C-SENSORS) — the adaptation layer: the versioned profile schema and per-sensor profile data + ADF bindings. It contains no instrument constants in code and no algorithm code (REQ-AD-01, REQ-D-04); the first instantiated profile is the owner’s sensor (REQ-AD-02).msi_processor.common(C-COMMON) — the platform/services layer shared by all PUs:EOProduct/EOZarrStorebuild and I/O, ADF resolution, profile load/validation, provenance assembly, QA-flag registry/propagation, chunking/Dask wiring, URI/store mapping, run configuration, chain orchestration + computing-model handling, and the CLI.
Dependency rule (acyclic, downward only). computing depends on common and consumes a resolved
sensors profile; common depends only on EOPF CPM and the reused stack (RD-10); a Core depends on
neither the CPM nor common (it takes plain arrays/parameters). sensors is data + schema and
depends on nothing. There is no upward or cyclic dependency.
Main relationship between major components. The chain is data-driven: each PU consumes the upstream
EOProduct (+ ADFs + profile-derived parameters), produces the next-level EOProduct, and hands it on.
The level breakpoints (L1A/L1B/L1C/L2A, DPM-BKP-*) are the inter-component contract (internal
interface, <5.5>). QA (qa) is cross-cutting: it is invoked by every measurement PU to attach metrics
and accumulate the per-pixel flag layer (REQ-F-QA-01/02).
a. Architecture of the software item. As above: a 3-layer package decomposition (computing /
sensors / common) realising a CPM PU pipeline with a pure-core + thin-wrapper pattern.
b. System states/modes in which the software operates. The processor implements the three states/modes of SRS <4.1>/<8> and REQ-O-04: configured/idle (profile, inputs and ADFs resolved), processing (one or more PUs executing), terminating in completed or error/aborted (fail-stop, REQ-F-DEP-01). There is no resident, cyclic or real-time mode (SSS <4.2>). The behavioural model is in <4.3>/<5.2>e.
c. Separated mission and configuration data (Annex F <4.1>c). All sensor-, mission- and site-dependent data is externalised from the code into the profile and ADFs (REQ-AD-01/03/04, REQ-D-04), classified per the Annex F NOTE categories:
Data category (Annex F <4.1>c) |
In |
Location / owner |
Changeable without code change |
|---|---|---|---|
Mission-analysis data, varies per mission |
Orbit/attitude & viewing-model references, acquisition geometry sources |
|
Yes (by reference) |
Reference data specific to a family of products |
Profile schema defaults, band list / spectral-response references, output CRS/grid/tiling |
|
Yes (profile edit) |
Reference data that never changes |
Physical constants (π, solar-spectrum integration constants) used by Cores |
code constants in |
n/a (universal) |
Data depending on the specific mission (e.g. sensor calibration) |
Dark/DSNU, PRNU/flat-field, bad-pixel map, radiometric gain/offset, viewing model |
private ADFs, referenced by URI ( |
Yes (ADF swap, REQ-M-02) |
Data varying with the higher-level system design |
Output store target + chunking, breakpoints, optional-stage toggles, atmospheric source |
run config / triggering JSON ( |
Yes (config) |
This separation is the mechanism by which the same software runs across sensors and contexts (REQ-D-07, REQ-AD-01..04) and by which no private calibration is ever embedded in the code (REQ-S-01/05).
<4.2> Software dynamic architecture (computational model)#
msi-processor has no hard real-time constraint (REQ-R-05): there are no cyclic/sporadic threads,
no scheduling-by-deadline, and no RMS/DMS/EDF analysis is applicable. The computational model required by
Annex F <4.2>/<5.2>c is expressed as a data-flow DAG of passive components with two interchangeable
execution strategies:
Component types. All components are passive (called, not self-scheduling). The active element is the chain runner (C-COM-ORC, realising
DPM-M-PRDorchestration), which executes the PU DAG. PUs are stateless across invocations; state lives in theEOProducts flowing between them.Scheduling model. Two modes selected by configuration: (1) sequential — the runner executes PUs in topological order in a single process (the CI / local-workstation path, REQ-PORT-03); (2) distributed — the per-PU Core operates on Zarr/Dask blocks and the Dask scheduler builds and runs the task graph across workers (REQ-F-ORC-02, REQ-R-04). The two share identical Cores; only the block mapping (C-COM-CHUNK) changes.
Means of communication. Components communicate exclusively through
EOProductobjects (in-memoryxarray-backedMappingDataType) and, at level breakpoints, through Zarr stores read/written viaEOZarrStore(REQ-F-PRD-01). There are no mailboxes, shared mutable globals or RPC; the only inter-process channel is the Dask task graph over chunked arrays.Means of synchronisation. None at application level beyond the DAG topology (a PU starts when its inputs are materialised). Concurrency control is delegated to the Dask scheduler; Cores are pure and free of shared state, so they are inherently safe to parallelise per block.
Distribution (Annex F <5.2>c). Optional, via Dask only; nodes are Dask workers, the unit of distribution is the block/tile. No custom inter-node protocol is defined.
Computing-model / triggering approach. The static DAG is wired declaratively by the CPM triggering
payload (ICD <5.3.5>, RD-12): a workflow[] of WorkFlowUnitDescriptions naming, for the requested
chain or sub-chain, the active PUs, the io (input product(s), ADF set, output target), the per-PU
parameters and the breakpoints. Each PU additionally carries its own computing-model JSON
(models/<name>_<version>.json, <5.4>) declaring its mandatory inputs, ADFs, outputs, parameters and
modes; the CPM validates a unit’s wiring against this model. The runner (C-COM-ORC) parses the payload,
resolves URIs (C-COM-IO), validates the profile (C-COM-PROFILE), instantiates the required
EOProcessingUnits and executes them honouring breakpoints. This is the single mechanism realising “run
a level, a sub-chain, or the full L0c→L2A chain” (REQ-F-ORC-01, REQ-I-05, DPM-BKP-*).
<4.3> Software behaviour#
A single run is a finite state machine (mirroring SRS <8>):
stateDiagram-v2
[*] --> ConfiguredIdle: parse triggering JSON,\nresolve URIs, validate profile+ADF
ConfiguredIdle --> Processing: inputs+ADFs+profile OK
ConfiguredIdle --> ErrorAborted: invalid profile/ADF/input (REQ-DAT-03, REQ-S-04)
Processing --> Processing: next PU (per DAG / breakpoints)
Processing --> Completed: last PU OK -> publish EOProduct(s)
Processing --> ErrorAborted: any PU fails (fail-stop, REQ-F-DEP-01)
Completed --> [*]: status 0 + report
ErrorAborted --> [*]: status !=0, no partial product published
Behaviour is deterministic and reproducible for a fixed (input, ADF, profile, processor) quadruple (REQ-F-DEP-02, REQ-REL-01); stochastic kernels (RANSAC) use a profile-fixed seed (ATBD <6>). Error handling and fault tolerance are detailed in <5.2>g and, per-component, in <5.4>.
<4.4> Interfaces context#
The external interfaces are specified at requirements level in the IRD (RD-3, REQ-IF-*) and defined
concretely in the ICD (RD-5, ICD-IF-*); per Annex F <4.4>a this SDD refers to the ICD and does
not re-derive them. The context (IRD <4.1> external entities E1–E6) is:
flowchart LR
E1[E1 L0 ingestion / downlink\nL0c product] -->|input, read-only| MSI
E2[E2 Calibration facility / ADF provider\nprivate ADFs] -->|input, by URI| MSI
E3[E3 Sensor-profile / config provider] -->|input, profile| MSI
E4[E4 Orchestration / trigger\ntriggering JSON] -->|invoke| MSI
MSI[msi-processor] -->|output, Zarr EOProduct| E5[E5 Product store / archive / dissemination]
MSI -. executes within .-> E6[E6 EOPF CPM + storage backend]
Internal interfaces (between the design components of clause <5>) are designed in detail in <5.5>;
external interfaces are bound by REQ-I-01..07 and controlled in the ICD (RD-5, ICD-IF-*).
<4.5> Long lifetime software#
Design choices that minimise dependency on the operating system and hardware to support a long planned lifetime and portability (Annex F <4.5>; REQ-PORT-01/02, REQ-D-09):
Pure Python 3.11, no native build, no GPU (REQ-R-01): the only platform assumption is a POSIX x86-64 Linux host; the Cores use portable array libraries (numpy/xarray).
Open data format: outputs are Zarr with CF/STAC/GeoZarr metadata (RD-13, REQ-D-09) — readable independently of this software and of any vendor runtime.
Location-transparent I/O: all inputs/outputs are referenced by URI and resolved through the EOPF store abstraction (C-COM-IO), so local-FS, POSIX and S3 backends are interchangeable without code change (REQ-PORT-02, REQ-I-06).
Externalised volatile data: instrument constants, CRS/grid, and per-stage parameters live in the profile/ADFs, so sensor and mission evolution does not touch the code (REQ-AD-01, <4.1>c).
Single pinned platform dependency (
eopf == 2.8.1) with a documented, V&V-gated bump procedure (REQ-M-03), isolating the one component most likely to force change. The CPM coupling is confined to the Wrappers and theC-COM-*services (see <5.4>); the Cores are CPM-free.Pure-core isolation keeps the algorithm bodies independent of the CPM, so a future platform change re-touches only the thin wrappers, not the numerics (REQ-D-03, REQ-M-04).
<4.6> Memory and PU budget#
Per Annex F <4.6>, the allocation of memory and processing time to components. The governing design rule
is bounded, block-proportional memory: peak per-worker memory is set by the configured block/tile
size (C-COM-CHUNK), not by product size (REQ-F-ORC-02, REQ-P-05, REQ-R-04), and wall-time scales with the
number of tiles × workers (REQ-P-04). Absolute numeric budgets (MEM_BUDGET, THRU_SCENE) are
per-profile parameters held in the private calibration/auxiliary store (SRS <5.1>, DPM-PRM-GEN-02)
and are verified locally; they are not reproduced here. The qualitative allocation and the per-stage
blocking strategy (used by the detailed design of <5.4>):
PU (component) |
DPM module |
Memory intensity |
Compute intensity |
Dominant cost |
Blocking strategy (C-COM-CHUNK) |
|---|---|---|---|---|---|
C-PU-L0 l0_decode |
DPM-M-L0 |
low–med |
low |
full-frame assembly |
per-band/per-detector line streaming |
C-PU-RAD radiometric |
DPM-M-RAD |
low |
low |
element-wise per band |
line-chunked; per-detector vectors broadcast |
C-PU-ENH enhancement |
DPM-M-ENH |
med |
med–high |
MTFC/PSF-deconvolution + denoise kernels |
tiled with halo (mandatory) |
C-PU-TOA toa |
DPM-M-TOA |
low |
low |
element-wise scaling |
line-chunked |
C-PU-COR coregistration |
DPM-M-COR |
med |
high |
feature detect/match (SIFT/FLANN/RANSAC) |
per-band-pair on reference-band overview; full-band warp |
C-PU-GEO georeference |
DPM-M-GEO |
high |
high |
DEM ortho + resampling to grid |
tiled resampling, windowed DEM reads |
C-PU-PAN pansharpen (opt, post-L2A) |
DPM-M-PAN |
med–high |
med |
MS↔PAN fusion at PAN resolution on L2A BOA (post-AC, terminal derivative) |
tiled; default-off |
C-PU-ATM atmospheric |
DPM-M-ATM |
med |
high |
RT/retrieval per pixel + classification |
grid-chunked; LUT-based RT |
C-PU-QA qa |
DPM-M-QA |
low |
low |
metric reductions |
streaming reductions |
C-COMMON services |
DPM-M-PRD |
low |
low |
Zarr I/O, provenance |
lazy/chunked store I/O |
georeference and atmospheric are the design’s memory/compute hot-spots and the priority targets for
chunking/Dask validation (REQ-P-04/05). The per-component numeric budget table is delivered with the
per-profile data and verified locally (REQ-Q-03).
<4.7> Design standards, conventions and procedures#
Per Annex F <4.7>a the adopted software methods are summarised here and refer to the SDP (RD-1) §5.3 for the toolchain and process detail. The Annex F <4.7>b items:
Architectural design method. EOPF CPM processing-unit architecture: the chain is a DAG of
EOProcessingUnits overEOProduct/Zarr, decomposed by processing stage (one per DPM module), each stage realised as pure Core + thin Wrapper, specialised by an externalised sensor-profile layer (SDP §5.3).Detailed design method. Structured, per-component description against the Annex F <5.4> aspect set, with element-level data structures (dataclasses + numpy dtypes), Python type-annotated core signatures and
run()contracts, and the CPM computing-model JSON. This issue (CDR) provides that detailed design in <5.4>/<5.5>; bodies marked[impl]are completed in WP-5.Code documentation standards. Module/class/function docstrings (NumPy/Sphinx style); the published design subset rendered to
docs/sdd/via Sphinx (SDP §5.4/§5.5).Naming conventions. PEP 8 for code; EOPF data-model conventions for product variables, bands, dimensions and command/payload fields (REQ-I-07); hierarchical component naming
package.module.Componentmirroring the package tree (Annex F <5.4.2>c). PU class attributesPROCESSOR_NAME = "msi_<stage>".Programming standards. Python 3.11, PEP 8 enforced by
black/ruff/flake8/isort; typing bymypy; security bybandit/trivy; complexity bounded byxenon; quality gateSonarQube; tests bypytest(SDP §5.4; REQ-D-02, REQ-Q-01).Intended list of reuse components. EOPF CPM (
EOProcessingUnit/EOProduct/EOZarrStore), numpy, xarray, zarr, Dask, scikit-image, OpenCV, GDAL/rasterio, pywt, scikit-learn, plus the prior-work algorithm heritage (RD-9). The authoritative reuse declaration and licences are in the SRF (RD-10); product I/O is implemented only through the CPM abstractions (REQ-D-06).Main design trade-offs.
Pure Core + thin Wrapper (chosen) vs algorithm-in-PU. Cost: a small adapter per stage and a stable Core↔Wrapper contract (IF-CORE-01, <5.5>). Benefit: Cores are unit-testable without the CPM runtime (CI shell runner, REQ-PORT-03, REQ-D-03), numerically verifiable in isolation, and portable across a future platform change (REQ-M-04).
Sensor-agnostic profile layer (chosen) vs per-sensor code forks. Benefit: a new sensor is a new profile + private ADFs, no core change (REQ-D-07, REQ-AD-01); no instrument constants in code (REQ-D-04, REQ-S-05). Cost: an up-front profile schema + validation (C-COM-PROFILE).
One PU per stage with optional stages toggleable (chosen) vs fused mega-stages. Benefit: level breakpoints, independent verification and re-run granularity (REQ-F-ORC-01, REQ-REL-02, REQ-M-04); pansharpen is optional/default-off, while enhancement is mandatory (its MTF-compensation / PSF-deconvolution sub-step is a required Level-1 image-quality restoration) with the denoise sub-step profile-configurable (REQ-F-ENH-03). Cost: more inter-PU
EOProducthand-offs (mitigated by lazy Zarr).Chunked + optional Dask (chosen) vs whole-product in memory. Benefit: bounded memory and horizontal scaling (REQ-F-ORC-02, REQ-P-05). Cost: tiling/halo handling in spatial PUs.
EOProduct/Zarr-exclusive I/O (chosen) vs custom formats. Benefit: cloud-native, CPM-native, open (REQ-D-09, REQ-D-06). Cost: bound to the pinned
eopf(REQ-M-03).Atmospheric correction as new development (no heritage) vs reuse. The only non-reused Core; designed interface-first against the DPM (RD-6)/ATBD (RD-7) so the rest of the chain is unaffected; the RT engine and classifier are
[impl]/DPM down-select.
<5> Software design#
<5.1> General#
This clause describes the software architectural design (<5.2>, <5.3>) and the detailed design (<5.4> per-component element level, <5.5> internal-interface data). The architecture is described identifying the software components, their hierarchical relationships, dependencies and interfaces (Annex F <5.1>b). Flight-software in-flight-modification design is not applicable (ground software, REQ-D-08; Annex F <5.1>c). The structure of <5.2>–<5.5> is used (Annex F <5.1>d). The DJF is referenced in <6>b.
<5.2> Overall architecture#
<5.2>a/b Static architecture (summary)#
The software item decomposes into the three top-level components of <4.1> and, within computing, nine
processing-stage components plus the cross-cutting common services. The static hierarchy:
msi_processor (software item)
├── computing C-COMPUTING (processing chain)
│ ├── l0_decode C-PU-L0 {core, unit}
│ ├── radiometric C-PU-RAD {core, unit}
│ ├── enhancement C-PU-ENH {core, unit} (MTFC mandatory)
│ ├── toa C-PU-TOA {core, unit}
│ ├── coregistration C-PU-COR {core, unit}
│ ├── georeference C-PU-GEO {core, unit}
│ ├── pansharpen C-PU-PAN {core, unit} (opt, post-L2A derivative)
│ ├── atmospheric C-PU-ATM {core, unit}
│ └── qa C-PU-QA {core, unit} (cross-cutting)
├── sensors C-SENSORS (profile schema + per-sensor data + ADF bindings)
└── common C-COMMON
├── product C-COM-PRODUCT (EOProduct build + EOZarrStore I/O)
├── io C-COM-IO (URI / store mapping: local FS, POSIX, S3)
├── adf C-COM-ADF (ADF resolution + read, private, by URI)
├── profile C-COM-PROFILE (profile schema load + validation)
├── provenance C-COM-PROV (provenance metadata assembly)
├── qaflags C-COM-QAFLAG (QA/mask flag bit registry + propagation)
├── chunking C-COM-CHUNK (tile/chunk planning + Dask wiring)
├── config C-COM-CONFIG (run configuration / parameter resolution)
├── orchestration C-COM-ORC (chain runner + triggering/computing-model handling)
├── errors (C-COM-ORC) (typed exception hierarchy — <5.4.1>)
└── cli C-COM-CLI (`msi-processor` batch entry point)
Each computing stage subpackage holds exactly two subordinate components — a core (pure
algorithm, package.stage.core) and a unit (the EOProcessingUnit, package.stage.unit) —
realising the design pattern of REQ-D-03.
<5.2>c/d Dynamic architecture (computational model)#
As <4.2>: a passive-component data-flow DAG executed by C-COM-ORC, sequential or Dask-distributed,
communicating through EOProduct/Zarr, with the CPM triggering payload + per-PU computing-model JSON as
the declarative wiring. There is no real-time scheduling/analytical model (REQ-R-05); the Annex F <5.2>d
items (scheduling type/model, analytical model, task priorities, timing) are not applicable and are
recorded as such.
<5.2>e Software behaviour#
As <4.3>: the ConfiguredIdle → Processing → {Completed | ErrorAborted} automaton, fail-stop on any PU
error, deterministic for a fixed input/ADF/profile/processor quadruple (REQ-F-DEP-02).
<5.2>f Consistency with the design method#
The static (package/PU), dynamic (DAG/Dask) and behavioural (state machine) views above are all expressed in the adopted CPM PU + pure-core/thin-wrapper method (<4.7>), satisfying Annex F <5.2>f. The detailed design of <5.4> instantiates this method uniformly across all components.
<5.2>g Error handling and fault tolerance principles#
The principles below are specialised, per component, in the Error/exception handling aspect of <5.4>;
the typed exception hierarchy is defined in <5.4.1>.
Detection. Inputs/ADFs/profile validated before processing (REQ-F-L0-03, REQ-DAT-03, REQ-S-04); per-PU acceptance checks (e.g. co-registration match count REQ-F-COR-03); numerical range/saturation/no-data checks (REQ-F-RAD-04, REQ-D-05).
Containment region. The PU boundary is the fault-containment region: a Core raises a typed
MsiProcessorErrorsubclass, the Wrapper converts it to a flagged, reported failure; faults do not propagate silently across the DAG.Reporting & logging. Structured logs + a machine-readable processing report (
ICD-IF-DIAG) carry success/failure, parameters and provenance (REQ-O-02/03, REQ-HF-02); per-pixel QA flags carry localised defects (REQ-F-QA-02,C-COM-QAFLAG).Recovery policy. Fail-stop (REQ-F-DEP-01; CPM
triggering__error_policy: FAIL_FAST): on any PU failure the run exits non-zero and no partial or misleading product is published; the chain is re-runnable at breakpoint granularity (REQ-REL-02, REQ-F-ORC-01,DPM-BKP-*). There is no in-run retry/redundancy (ground, reprocessable).Residual-hazard alignment. The residual hazard class is product-data integrity (REQ-SAF-01); the QA flags + provenance + fail-stop triad is its design mitigation.
<5.3> Software components design — General#
Per Annex F <5.3>a/b: the components, their relationships, purpose, development type (new vs reused), and the requirements allocation (each component uniquely identified). Components written for reuse expose their function and interfaces externally (the Cores; <5.4>). Handling of reused components is governed by the SRF (RD-10), per Annex N.
Id |
Component (package path) |
Type |
Purpose (one line) |
Dev type |
Allocated |
DPM/ATBD |
|---|---|---|---|---|---|---|
C-COMPUTING |
|
package |
Processing chain container |
new |
REQ-D-01, REQ-F-ORC-01 |
— |
C-PU-L0 |
|
PU (core+unit) |
Decode/reformat |
reuse-adapt (RD-9 |
REQ-F-L0-01..05 |
DPM-M-L0 / ALG-L0-* |
C-PU-RAD |
|
PU (core+unit) |
Dark/DSNU, NUC/PRNU, BPR, saturation/no-data |
reuse-adapt (RD-9 |
REQ-F-RAD-01..05 |
DPM-M-RAD / ALG-RAD-* |
C-PU-ENH |
|
PU (core+unit) |
MTF compensation (MTFC via PSF deconvolution, mandatory) + configurable denoise, radiometry-preserving |
reuse-adapt (RD-9 |
REQ-F-ENH-01..03 |
DPM-M-ENH / ALG-ENH-* |
C-PU-TOA |
|
PU (core+unit) |
DN→TOA radiance (+opt reflectance), emit |
reuse-adapt (RD-9 |
REQ-F-TOA-01..03 |
DPM-M-TOA / ALG-TOA-* |
C-PU-COR |
|
PU (core+unit) |
Inter-band co-registration to reference band |
reuse-adapt (RD-9 |
REQ-F-COR-01..03 |
DPM-M-COR / ALG-COR-* |
C-PU-GEO |
|
PU (core+unit) |
Viewing-model geoloc + GCP + DEM ortho → |
reuse-adapt (RD-9 |
REQ-F-GEO-01..04 |
DPM-M-GEO / ALG-GEO-* |
C-PU-PAN |
|
PU (core+unit) (opt, post-L2A) |
MS↔PAN fusion to high-res MS on L2A BOA (post-AC); terminal optional derivative, default-off |
reuse-adapt (RD-9 |
REQ-F-PAN-01/02 |
DPM-M-PAN / ALG-PAN-* |
C-PU-ATM |
|
PU (core+unit) |
AOT/WV, TOA→BOA, scene class + masks → |
new (DPM RD-6/ATBD RD-7) |
REQ-F-ATM-01..04 |
DPM-M-ATM / ALG-ATM-* |
C-PU-QA |
|
PU/library (core+unit) |
QA metrics + per-pixel flag propagation |
reuse-adapt (RD-9 |
REQ-F-QA-01/02 |
DPM-M-QA / ALG-QA-* |
C-SENSORS |
|
data + schema |
Profile schema + per-sensor profile data + ADF bindings |
new |
REQ-AD-01..04, REQ-DAT-03 |
DPM <7.4> |
C-COM-PRODUCT |
|
library |
Build |
new (over CPM) |
REQ-F-PRD-01, REQ-DAT-01, REQ-D-09 |
DPM-M-PRD |
C-COM-IO |
|
library |
URI resolution / store mapping (local/POSIX/S3) |
new (over CPM) |
REQ-I-06, REQ-PORT-02/03 |
— |
C-COM-ADF |
|
library |
Resolve + read private ADFs by URI, validity check |
new |
REQ-F-RAD-01, REQ-DAT-02, REQ-S-04 |
DPM <7.2> |
C-COM-PROFILE |
|
library |
Load + schema-validate the sensor profile |
new |
REQ-AD-01/02, REQ-DAT-03 |
ICD <5.3.6> |
C-COM-PROV |
|
library |
Assemble provenance metadata |
new |
REQ-F-PRD-02, REQ-S-05 |
DPM-M-PRD |
C-COM-QAFLAG |
|
library |
QA/mask flag bit registry + propagation |
new |
REQ-F-QA-02 |
ICD <5.3.3>E |
C-COM-CHUNK |
|
library |
Tile/chunk planning + optional Dask wiring |
new (over Dask) |
REQ-F-ORC-02, REQ-R-04, REQ-P-05 |
DPM-PRM-GEN-02 |
C-COM-CONFIG |
|
library |
Resolve run config / parameters |
new |
REQ-AD-04, REQ-O-01 |
ICD <5.3.5> |
C-COM-ORC |
|
executable |
Chain runner; parse/execute triggering payload |
new (over CPM) |
REQ-F-ORC-01, REQ-F-DEP-01, REQ-I-05 |
DPM-M-PRD |
C-COM-CLI |
|
executable |
|
new |
REQ-I-02, REQ-O-01, REQ-HF-01 |
ICD <5.3.7> |
Relationships. C-COM-ORC drives the C-PU-* chain; every C-PU-* Wrapper uses C-COM-PRODUCT,
C-COM-ADF, C-COM-PROFILE, C-COM-PROV, C-COM-QAFLAG, C-COM-CHUNK; every C-PU-* Core uses
none of common (pure). C-SENSORS is consumed (as data) by C-COM-PROFILE. C-COM-CLI is the
entry point to C-COM-ORC. The backward (component→requirement) trace is the inverse of the allocation
column above and is consolidated in <6> and RD-11.
<5.4> Software components design — Detailed design of each component#
Per Annex F <5.4.1>a / <5.4.2>–<5.4.11> each component is described against the aspect set, at
element-level for this CDR issue. To avoid repetition, the common PU aspects and the shared
data structures / exception hierarchy are stated once in <5.4.1>, then specialised per stage in
<5.4.2>–<5.4.10>; the common/sensors components are in <5.4.11>–<5.4.12>.
Reading the per-component blocks. Each stage block gives: Identifier/type; Purpose & trace; Pure-core — signatures & data structures (the CPM-free algorithm, IF-CORE-01, grounded in RD-9 and the cited
ALG-*); EOProcessingUnit wrapper —run()I/O (inputs/adfs/outputs/parameters/modes, ICD <5.3.4>A); Computing-model JSON (the per-PU CPM declaration); Error/exception handling. Python signatures are the designed public contract; private helper bodies and tuning are[impl].
<5.4.2> C-PU-L0 — l0_decode (DPM-M-L0; ALG-L0-DEC, ALG-L0-LOSS)#
Identifier/type. msi_processor.computing.l0_decode; PU (core+unit); PROCESSOR_LEVEL="L1A".
Purpose & trace. Decode the L0c stream to per-band/per-detector focal-plane arrays, detect/handle
line loss, assemble the L1A EOProduct; the ground_decode module implements the consumer-side
canonical-L0 decompression (reassemble + CCSDS-122 decode, bit-exact — the real-chain L1A-side
operation). Trace: REQ-F-L0-01..06; DPM-M-L0; ALG-L0-DEC/LOSS;
ICD-IF-L0-*.
Pure-core — signatures & data structures (l0_decode.core):
@dataclass(frozen=True)
class LineLoss: # per-band loss record -> processing report + QA
band: str; start_line: int; n_lost: int
def decode(raw: "RawL0", codec: "CodecSpec") -> dict[str, np.ndarray]:
"""ALG-L0-DEC. Ground-decode the documented canonical L0 (CCSDS space packets
carrying CCSDS-122 lossless payloads) bit-exactly via `ground_decode` (REQ-F-L0-06);
map other L0c source packets to {band: DN[line, detector]} in focal-plane
geometry, applying the profile focal-plane layout (line order / flips, per-band line_factor).
Body is sensor/NDA-specific and profile-bound -> [impl]; heritage Decoder.decode is a stub."""
def detect_and_truncate_loss(
bands: dict[str, np.ndarray],
line_factor: Mapping[str, int], # DPM-PRM-L0-01; e.g. PAN band -> 2
) -> tuple[dict[str, np.ndarray], list[LineLoss]]:
"""ALG-L0-LOSS (heritage level_0.lost_package). For each band, first index l with
DN[l]!=0 and DN[l+1]==0 marks a loss start l*=min over bands; truncate DN[:-(l*·s_b)]
and record LineLoss. Pure numpy; deterministic."""
def check_legality(bands, telemetry, profile) -> None:
"""REQ-F-L0-03 precondition check; raises InputValidationError on a malformed/mismatched input."""
def initial_qa(bands, losses) -> dict[str, np.ndarray]:
"""Initialise QA bitmask (uint16); set LOST_PACKET on truncated/zero-filled lines, NO_DATA on fill."""
Data structures: RawL0/CodecSpec are profile-bound descriptors [impl] (decode internals are NDA,
ATBD <5.1>); the output of the Core is a BandStack of geom="focal_plane" plus the telemetry meta.
EOProcessingUnit wrapper — run() I/O (l0_decode.unit.L0DecodeUnit):
Aspect |
Value |
|---|---|
|
|
|
none (no calibration at L0) |
|
|
|
|
|
|
Computing-model JSON (models/msi_l0_decode_1.0.0.json; CPM schema, exact keys bound to 2.8.1
[TBC@CDR]):
{
"name": "msi_l0_decode", "version": "1.0.0", "level": "L1A",
"inputs": [{"name": "l0", "mandatory": true}],
"adfs": [],
"outputs": [{"name": "l1a", "mandatory": true}],
"parameters": {
"profile_id": {"type": "string"},
"profile_version": {"type": "string"},
"bit_depth": {"type": "integer", "default": 12},
"line_factor": {"type": "object", "default": {}}
},
"modes": ["nominal"]
}
Error/exception handling. Malformed/mismatched input or failed profile/ADF resolution ⇒
InputValidationError before any radiometric step (REQ-F-L0-03), fail-stop; L0 never written
(REQ-F-L0-05). Loss is not an error: truncate + LOST_PACKET flag + report (REQ-F-L0-02). Open point
(ATBD <5.1>): non-zero corruption / CRC handling is [impl] beyond the zero-line rule.
<5.4.3> C-PU-RAD — radiometric (DPM-M-RAD; ALG-RAD-NUC/DARK/BPR/SAT)#
Identifier/type. msi_processor.computing.radiometric; PU; intermediate level (optional breakpoint
DPM-BKP-RAD). Purpose & trace. Dark/DSNU subtraction, NUC/PRNU equalisation, bad-pixel replacement,
saturation/no-data flagging; optional NUC derivation (calibration mode). Trace: REQ-F-RAD-01..05;
DPM-M-RAD; ALG-RAD-*; ICD-IF-ADF-01.
Pure-core — signatures & data structures (radiometric.core):
@dataclass(frozen=True)
class RadiometricParams:
bit_depth: int = 12
g_min: float | None = None # DPM-PRM-RAD-02 bad-pixel gain bounds
g_max: float | None = None
saturation: int | None = None # else 2**bit_depth - 1
fill_value: int | None = None
remove_dark_fft: bool = False # DPM-PRM-RAD-03
def estimate_nuc(dark: np.ndarray, flat: np.ndarray,
cut_dark: int = 0, cut_flat: int = 0) -> tuple[np.ndarray, np.ndarray]:
"""ALG-RAD-NUC (heritage NUC.compute_nuc). Column-mean dark/flat -> per-detector
g = (mean(F)-mean(D))/(F-D), o = mean(F) - g·F. Returns (gain[detector], offset[detector])."""
def apply_nuc(dn: np.ndarray, gain: np.ndarray, offset: np.ndarray,
dark_offset: np.ndarray | float) -> np.ndarray:
"""ALG-RAD-DARK (heritage apply_nuc_and_bpr). X = dn·g + o − d (float32)."""
def detect_bad_pixels(gain: np.ndarray, params: RadiometricParams,
bpm: np.ndarray | None) -> np.ndarray:
"""ALG-RAD-BPR detection. bad = (g>=g_max)|(g<=g_min) | bpm. Returns bool[detector]."""
def replace_bad_pixels(corrected: np.ndarray, bad: np.ndarray) -> np.ndarray:
"""ALG-RAD-BPR replacement: across-track neighbour interpolation
(½(left+right); copy nearest valid at clusters/edges). Flags DEFECTIVE upstream."""
def flag_saturation(corrected: np.ndarray, params: RadiometricParams
) -> tuple[np.ndarray, np.ndarray]:
"""ALG-RAD-SAT. Clip to [0, 2^bit_depth−1]; return (clipped, qa) with SATURATED/NO_DATA set."""
def remove_dark_fft(dn: np.ndarray, dark: np.ndarray) -> np.ndarray: # optional, DPM-PRM-RAD-03
"""heritage dark_noise_removal: Re{IFFT2(FFT2(dn) − FFT2(dark))}, clipped."""
The Core consumes/produces a BandStack (geom="focal_plane"). Per-detector gain/offset/dark are
1-D float32 vectors broadcast across lines; element-level shapes/ranges in <5.5>.
EOProcessingUnit wrapper — run() I/O (radiometric.unit.RadiometricUnit):
Aspect |
Value |
|---|---|
|
|
|
|
|
|
|
|
|
|
Computing-model JSON (models/msi_radiometric_1.0.0.json):
{
"name": "msi_radiometric", "version": "1.0.0", "level": "L1A",
"inputs": [{"name": "l1a", "mandatory": true}],
"adfs": [{"name": "dark", "mandatory": true},
{"name": "nuc", "mandatory": false},
{"name": "flatfield", "mandatory": false},
{"name": "badpixel", "mandatory": false}],
"outputs": [{"name": "rad", "mandatory": true},
{"name": "nuc", "mandatory": false}],
"parameters": {
"bit_depth": {"type": "integer", "default": 12},
"nuc_mode": {"type": "string", "enum": ["read", "derive"], "default": "read"},
"g_min": {"type": "number"}, "g_max": {"type": "number"},
"remove_dark_fft": {"type": "boolean", "default": false}
},
"modes": ["nominal", "calibration"]
}
Error/exception handling. Missing/validity-mismatched ADF ⇒ AdfResolutionError (REQ-S-04),
fail-stop. NUC singularity (F==D detector) and out-of-range gain are handled as bad pixels (flagged
DEFECTIVE), not exceptions. All outputs clipped to the valid range (REQ-F-RAD-04, REQ-D-05). Open point
(ATBD <5.2>): non-linear response term is [impl].
<5.4.4> C-PU-ENH — enhancement (mandatory) (DPM-M-ENH; ALG-ENH-*)#
Identifier/type. msi_processor.computing.enhancement; PU (mandatory); intermediate (DPM-BKP-ENH).
Purpose & trace. Mandatory Level-1 image-quality restoration: MTF compensation (MTFC) via PSF
deconvolution (mandatory sub-step — recovers the high-spatial-frequency content attenuated by the
instrument MTF: optics + detector + platform motion) plus a profile-configurable denoise sub-step,
radiometry-preserving. The stage always runs because MTFC is mandatory; MTFC materially affects
radiometric/spatial product quality. Change note (CR): enhancement promoted to mandatory;
“sharpening” = MTFC / PSF deconvolution. Change note (CR-3): the PSF/MTF kernel is now the
mandatory per-band psf ADF (DPM-ADF-PSF) — per-band 2-D kernels (focal-plane geometry),
float32, normalised to unit DC gain (sum=1) so radiometry is preserved, referenced by ADF URI
(values not reproduced here) and opened read-only — not a parameter; the optional dark ADF
(fft_dark only) is retained. Trace: REQ-F-ENH-01..03; DPM-M-ENH;
ALG-ENH-BWLP/WAVE/PCA/MA/GAUSS/FFTDARK/DECONV.
Pure-core — signatures & data structures (enhancement.core):
DenoiseMethod = Literal["butterworth", "wavelet", "pca", "moving_average", "gaussian", "fft_dark"]
def denoise(image: np.ndarray, method: DenoiseMethod, params: Mapping[str, object],
dark: np.ndarray | None = None) -> np.ndarray:
"""Dispatch to the selected denoiser (heritage Denoiser.*). Returns clipped float32."""
# concrete kernels (each pure; bodies reuse skimage/scipy/pywt/sklearn — RD-10):
def butterworth_lowpass(image, cutoff_ratio=0.2, order=10.0,
squared=False, npad=0) -> np.ndarray: ... # ALG-ENH-BWLP
def wavelet_visushrink(image, wavelet="db3", levels=20,
sigma_scale=1/3, mode="soft") -> np.ndarray: ...# ALG-ENH-WAVE
def pca_denoise(image, n_components: int) -> np.ndarray: ... # ALG-ENH-PCA
def moving_average(image, n: int = 60) -> np.ndarray: ... # ALG-ENH-MA ((2N+1) sliding)
def gaussian_smooth(image, ksize=5, sigma=None) -> np.ndarray: ... # ALG-ENH-GAUSS (σ=std if None)
def fft_dark_subtract(image, dark) -> np.ndarray: ... # ALG-ENH-FFTDARK
def mtf_compensate(image: np.ndarray, psf_kernel: np.ndarray) -> np.ndarray:
"""ALG-ENH-DECONV — MTF compensation (MTFC) via PSF deconvolution (MANDATORY sub-step).
Restores the high-spatial-frequency content attenuated by the instrument MTF
(optics + detector + platform motion); materially affects radiometric/spatial quality.
Heritage sharpening.deconvolution_kernel (cv2.filter2D), clipped. PSF-derived per-band
kernel (broader for the PAN band); `psf_kernel` is sourced at run() from the mandatory
`psf` ADF (DPM-ADF-PSF), not a parameter."""
EOProcessingUnit wrapper — run() I/O (enhancement.unit.EnhancementUnit):
Aspect |
Value |
|---|---|
|
|
|
|
|
|
|
|
|
|
Computing-model JSON (models/msi_enhancement_1.0.0.json): inputs:[{rad,true}], adfs:[{psf,true},{dark,false}],
outputs:[{enh,true}], parameters carrying the nested mtfc (PSF deconvolution, always applied; kernel
from the psf ADF) and denoise (configurable) objects, modes:["nominal"].
Error/exception handling. Mandatory stage — MTFC (PSF deconvolution) is always applied; only the
denoise sub-step is profile-configurable (REQ-F-ENH-03). Outputs always clipped to the valid range;
radiometric impact reported via QA metrics (REQ-F-QA-01), never silently applied. A missing/invalid psf
ADF (DPM-ADF-PSF) or a denoise method-not-in-allowed-set ⇒ InputValidationError at profile validation
(C-COM-PROFILE), not at run.
<5.4.5> C-PU-TOA — toa (DPM-M-TOA; ALG-TOA-RAD/REF)#
Identifier/type. msi_processor.computing.toa; PU; PROCESSOR_LEVEL="L1B". Purpose & trace.
DN→TOA radiance (mandatory) and TOA reflectance (optional); emit L1B. Trace: REQ-F-TOA-01..03;
DPM-M-TOA; ALG-TOA-RAD/REF; ICD-IF-OUT-*.
Pure-core — signatures & data structures (toa.core):
def dn_to_radiance(dn: np.ndarray, gain: np.ndarray | float,
offset: np.ndarray | float) -> np.ndarray:
"""ALG-TOA-RAD (heritage TOA.dn_to_radiance). L = (DN − offset)·gain (float32).
NOTE: the heritage `radiance -= radiance.min()` is DROPPED (ATBD <5.3> open point 3)."""
def radiance_to_reflectance(radiance: np.ndarray, esun: float,
sun_zenith_rad: float, earth_sun_dist_au: float) -> np.ndarray:
"""ALG-TOA-REF. ρ = π·L·d² / (E·cos θ_s). Clipped to [0,1]."""
def earth_sun_distance(doy: int) -> float:
"""d ≈ 1 − 0.01672·cos(0.9856°·(DOY−4)); or solar ephemeris [impl]."""
def solar_geometry(acq_time: "datetime", lon: float, lat: float) -> tuple[float, float]:
"""Proper solar ephemeris -> (sun_zenith_rad, sun_azimuth_rad).
Replaces the heritage sub-point conflation (ATBD <5.3> open point 3) -> [impl] ephemeris choice."""
EOProcessingUnit wrapper — run() I/O (toa.unit.ToaUnit):
Aspect |
Value |
|---|---|
|
|
|
|
|
|
|
|
|
|
Computing-model JSON (models/msi_toa_1.0.0.json): inputs:[{enh}],
adfs:[{radiometric,true},{spectral,false}], outputs:[l1b],
parameters:{emit_reflectance:{boolean,default:false}}, modes:["nominal"].
Error/exception handling. Validity-mismatched radiometric/spectral ADF ⇒ AdfResolutionError,
fail-stop. Non-physical (negative) radiance clipped + flagged. ESUN is profile/ADF data — the heritage
hard-coded ESUN table is not carried over (REQ-AD-01, ATBD <5.3>).
<5.4.6> C-PU-COR — coregistration (DPM-M-COR; ALG-COR-FEAT/HOM/WARP)#
Identifier/type. msi_processor.computing.coregistration; PU; intermediate (DPM-BKP-COR).
Purpose & trace. Feature-based inter-band alignment to a reference band with fail-stop on failure.
Trace: REQ-F-COR-01..03; DPM-M-COR; ALG-COR-*.
Pure-core — signatures & data structures (coregistration.core):
@dataclass(frozen=True)
class CoregParams:
reference_band: str # DPM-PRM-COR-01 (heritage "b2")
clahe_clip: float = 2.0; clahe_grid: tuple[int, int] = (8, 8) # DPM-PRM-COR-02
match_fraction: float = 0.10 # top-fraction matches kept
min_keypoints: int = 20; min_keypoints_pan: int = 40
ransac_tau: float = 5.0 # reprojection threshold (px)
max_residual: float | None = None # acceptance vs BAND_COREG (DPM-PRM-COR-04, private)
seed: int = 0 # deterministic RANSAC (REQ-F-DEP-02)
@dataclass(frozen=True)
class CoregResidual:
band: str; n_inliers: int; rms_residual_px: float; accepted: bool
def estimate_homography(band: np.ndarray, reference: np.ndarray, params: CoregParams
) -> tuple[np.ndarray, CoregResidual]:
"""ALG-COR-FEAT+HOM: 8-bit normalise → CLAHE → SIFT → FLANN(top-fraction) → RANSAC H(3×3).
Raises CoregistrationError on insufficient keypoints/matches. Reuses cv2; tuning [impl]."""
def warp_to_reference(band: np.ndarray, H: np.ndarray, shape: tuple[int, int]) -> np.ndarray:
"""ALG-COR-WARP: I_reg = warpPerspective(band, H, shape)."""
def coregister(bands: Mapping[str, np.ndarray], params: CoregParams
) -> tuple[dict[str, np.ndarray], list[CoregResidual]]:
"""Per non-reference band: estimate H, check residual ≤ max_residual, warp; crop to common
extent. Raises CoregistrationError if any required band is not accepted (REQ-F-COR-03)."""
EOProcessingUnit wrapper — run() I/O (coregistration.unit.CoregistrationUnit):
Aspect |
Value |
|---|---|
|
|
|
none |
|
|
|
|
|
|
Computing-model JSON (models/msi_coregistration_1.0.0.json): inputs:[l1b], adfs:[],
outputs:[cor], parameters:{reference_band, match_fraction, ransac_tau, min_keypoints, …},
modes:["nominal"].
Error/exception handling. Insufficient keypoints/matches or residual outside acceptance ⇒
CoregistrationError, the affected band flagged COREG_FAIL, and fail-stop (REQ-F-COR-03,
REQ-F-DEP-01) — no misregistered product is emitted. RANSAC uses seed for reproducibility (REQ-F-DEP-02).
<5.4.7> C-PU-GEO — georeference (DPM-M-GEO; ALG-GEO-ORBIT/GSD/GCP/ORTHO/RESAMP)#
Identifier/type. msi_processor.computing.georeference; PU; PROCESSOR_LEVEL="L1C". Purpose &
trace. Viewing-model geolocation + GSD, optional GCP refinement, DEM orthorectification, resample to
the profile CRS/grid; emit L1C. Trace: REQ-F-GEO-01..04; DPM-M-GEO; ALG-GEO-; ICD-IF-OUT-.
Pure-core — signatures & data structures (georeference.core):
@dataclass(frozen=True)
class PlatformState: # ALG-GEO-ORBIT
lat: float; lon: float; altitude_m: float; ground_velocity_ms: float
@dataclass(frozen=True)
class Geotransform: # GDAL affine [ulx, xres, 0, uly, 0, -yres]
ulx: float; xres: float; uly: float; yres: float; crs_wkt: str
def compute_gsd(altitude_m: float, pixel_pitch_m: float, focal_length_m: float) -> float:
"""ALG-GEO-GSD (heritage). GSD = altitude·pitch/focal."""
def orbit_state(viewing_model: "ViewingModel", acq_time: "datetime") -> PlatformState:
"""ALG-GEO-ORBIT: propagate orbit (SGP4/ephemeris) to acquisition; sub-point + altitude.
Engine choice (pyorbital/skyfield) [impl]."""
def geolocate(shape: tuple[int, int], state: PlatformState,
viewing_model: "ViewingModel", dem: np.ndarray) -> np.ndarray:
"""ALG-GEO-ORTHO. Rigorous collinearity (line-of-sight ∩ ellipsoid+DEM) per pixel.
CDR-target body -> [impl]; PDR heritage = reference-image homography fallback."""
def refine_with_gcp(image: np.ndarray, reference: np.ndarray, params: CoregParams) -> np.ndarray:
"""ALG-GEO-GCP: reuse coregistration.core machinery against a geolocated reference."""
def resample_to_grid(image: np.ndarray, src_geo, dst: Geotransform,
resampling: str) -> tuple[np.ndarray, Geotransform]:
"""ALG-GEO-RESAMP: warp to the profile CRS/grid/resolution (GDAL/rasterio + PROJ/osr)."""
EOProcessingUnit wrapper — run() I/O (georeference.unit.GeoreferenceUnit):
Aspect |
Value |
|---|---|
|
|
|
|
|
|
|
|
|
|
Computing-model JSON (models/msi_georeference_1.0.0.json): inputs:[cor],
adfs:[{viewing_model,true},{dem,true},{gcp,false}], outputs:[l1c],
parameters:{crs, grid, resolution, resampling, use_gcp}, modes:["nominal"].
Error/exception handling. Missing DEM/viewing-model coverage for the footprint/epoch ⇒
GeolocationError, fail-stop. Geolocation error verified locally vs GEO_CE90 (REQ-F-GEO-03, REQ-Q-03).
Open point (ATBD <5.7> open point 2): the rigorous collinearity body replaces the heritage homography at
implementation [impl].
<5.4.8> C-PU-PAN — pansharpen (optional) (DPM-M-PAN; ALG-PAN-ALIGN/FUSE)#
Identifier/type. msi_processor.computing.pansharpen; PU (opt, terminal post-L2A derivative).
Purpose & trace. MS↔PAN fusion to PAN resolution as a terminal, optional, default-off post-L2A
derivative product running after atmospheric correction (C-PU-ATM): its input is the L2A BOA
(surface) reflectance product, and it emits a separate pan-sharpened L2A derivative with spectral-fidelity
QA. Trace: REQ-F-PAN-01/02; DPM-M-PAN; ALG-PAN-*.
Change note (CR-4). Pan-sharpening is moved out of the L1B→L1C transition and re-placed as an optional terminal step after atmospheric correction. Rationale: pan-sharpening trades spectral/radiometric fidelity for spatial sharpness, so it is a visual/derivative product, not a science-grade input to quantitative retrieval/indices; and atmospheric correction must precede fusion so it operates on physically-meaningful BOA (surface) reflectance rather than TOA values (empirical guidance: Lin et al. 2015, WorldView-2 AC×pan-sharpen study).
L1Ctherefore stays band-co-registered orthorectified TOA reflectance with no pan-sharpening; the AC→pan-sharpen order is settled.
Pure-core — signatures & data structures (pansharpen.core):
FusionMethod = Literal["simple_mean", "brovey", "gs", "ihs", "atrous"] # heritage = simple_mean
def align_ms_to_pan(ms_stack: Mapping[str, np.ndarray], pan: np.ndarray,
params: CoregParams) -> dict[str, np.ndarray]:
"""ALG-PAN-ALIGN: reuse coregistration.core (CLAHE→SIFT→FLANN→RANSAC→warp) onto the PAN grid."""
def fuse(ms_aligned: Mapping[str, np.ndarray], pan: np.ndarray,
method: FusionMethod = "simple_mean") -> dict[str, np.ndarray]:
"""ALG-PAN-FUSE. simple_mean: ½(MS_b + PAN), clipped. Other methods [impl] (ATBD <5.9> open point 6)."""
EOProcessingUnit wrapper — run() I/O (pansharpen.unit.PansharpenUnit):
Aspect |
Value |
|---|---|
|
|
|
none |
|
|
|
|
|
|
Computing-model JSON (models/msi_pansharpen_1.0.0.json): inputs:[l2a], adfs:[],
outputs:[pan], parameters:{enabled, method, pan_band}, modes:["nominal"].
Open point (PAN reflectance handling) — [impl]/profile. The AC→pan-sharpen order is settled, but
the PAN-band reflectance to fuse against BOA-MS is not: rigorous AC is band-specific and the broadband
PAN band is too broad for a well-defined atmospheric correction. “After AC” therefore requires either
(a) producing a BOA-PAN approximation, or (b) fusing BOA-MS with TOA-PAN — a TOA/BOA domain
mismatch that weakens component-substitution methods (Brovey/GS/PCA). The choice between (a) and (b) is an
open [impl]/profile decision and is not resolved here.
Error/exception handling. Optional/default-off. Alignment failure ⇒ flag + skip fusion (the L2A
product remains valid at MS resolution) rather than fail-stop. Spectral fidelity reported vs the
per-profile budget (REQ-F-PAN-02); if unmet, a spectral-preserving method is selected by profile.
<5.4.9> C-PU-ATM — atmospheric (new) (DPM-M-ATM; ALG-ATM-PAR/RT/SCM)#
Identifier/type. msi_processor.computing.atmospheric; PU; PROCESSOR_LEVEL="L2A". Purpose &
trace. AOT/WV ingest or retrieval, TOA→BOA inversion, scene classification + cloud/shadow masks; emit
L2A. Trace: REQ-F-ATM-01..04; DPM-M-ATM; ALG-ATM-*. No prior-work heritage — interface-first new
development; RT engine and classifier are down-selected in the DPM/[impl].
Pure-core — signatures & data structures (atmospheric.core):
@dataclass(frozen=True)
class AtmParams:
aot: np.ndarray | float; water_vapour: np.ndarray | float; ozone: float | None = None
def get_atmospheric_parameters(toa_refl: Mapping[str, np.ndarray], mode: Literal["ingest","retrieve"],
aux: "AtmAux", params: Mapping[str, object]) -> AtmParams:
"""ALG-ATM-PAR: ingest auxiliary AOT/WV or retrieve from imagery (DDV / band-ratio). [impl]/DPM."""
def toa_to_boa(toa_refl: Mapping[str, np.ndarray], atm: AtmParams, geometry: "SceneGeometry",
dem: np.ndarray, rt_lut: "RTLut") -> dict[str, np.ndarray]:
"""ALG-ATM-RT: invert 6S surface-reflectance eqn per pixel via RT-LUT interpolation
(ATBD <5.8>). Engine (Py6S/Sen2Cor-LUT/ACOLITE) down-selected -> [impl]. Returns BOA in [0,1]."""
def classify_scene(boa: Mapping[str, np.ndarray], params: Mapping[str, object]
) -> tuple[np.ndarray, np.ndarray, np.ndarray]:
"""ALG-ATM-SCM: returns (scene_class, cloud_mask, cloud_shadow_mask). Classifier -> [impl]."""
EOProcessingUnit wrapper — run() I/O (atmospheric.unit.AtmosphericUnit):
Aspect |
Value |
|---|---|
|
|
|
|
|
|
|
|
|
|
Computing-model JSON (models/msi_atmospheric_1.0.0.json): inputs:[l1c],
adfs:[{atmospheric,false},{dem,true}], outputs:[l2a],
parameters:{mode:{enum:[retrieve,ingest]}, model, classification_opts}, modes:["nominal"].
Error/exception handling. Missing atmospheric/DEM coverage for the footprint/epoch ⇒
AtmosphericError, fail-stop. Cloud / cloud-shadow pixels flagged CLOUD/CLOUD_SHADOW (REQ-F-ATM-03).
Open point (ATBD <5.8> open point 1): parameter-retrieval method, RT engine and classifier are to be
defined and validated in the DPM before/at CDR; this SDD fixes the interface and the candidate algorithm,
the bodies are [impl].
<5.4.10> C-PU-QA — qa (cross-cutting) (DPM-M-QA; ALG-QA-*)#
Identifier/type. msi_processor.computing.qa; PU/library invoked by every measurement PU. Purpose
& trace. Compute per-band metrics and merge/propagate the per-pixel flag layer. Trace: REQ-F-QA-01/02;
DPM-M-QA; ALG-QA-*.
Pure-core — signatures & data structures (qa.core):
def compute_metrics(test: np.ndarray, reference: np.ndarray | None) -> MetricSet:
"""ALG-QA-* (heritage metrics_ips). SNR=20log10(mean/std); referential RMSE/MSE/PSNR vs
reference (aligned to common extent); variance. Returns MetricSet (∞ PSNR when MSE=0)."""
def align_extent(a: np.ndarray, b: np.ndarray) -> tuple[np.ndarray, np.ndarray]:
"""Crop both to the common extent before referential comparison (heritage run_validation)."""
def merge_flags(*masks: np.ndarray) -> np.ndarray:
"""QA bit OR-accumulation (monotone, REQ-F-QA-02). Same as C-COM-QAFLAG.merge."""
EOProcessingUnit wrapper — run() I/O (qa.unit.QaUnit). Usable standalone (inputs:{"test", "reference"?} → outputs:{"metrics"}) and as a library called inline by the other PUs (step 5 of the
wrapper template). modes:["nominal"]. Computing-model JSON models/msi_qa_1.0.0.json:
inputs:[{test,true},{reference,false}], outputs:[metrics], parameters:{metric_set, reference_sel}.
Error/exception handling. Metrics are observational and never abort the chain; a metric outside the configured tolerance is a warning in the report (REQ-O-02). Metrics carry no private data (REQ-HF-02).
<5.4.11> common services (C-COM-*) — detailed design#
Library/executable components; the data aspect dominates (Annex F <5.4.11>). The public contracts:
# C-COM-PRODUCT (msi_processor.common.product) — REQ-F-PRD-01, REQ-DAT-01, REQ-D-09
def build_eoproduct(measurements: Mapping[str, BandImage], conditions: Mapping, quality: Mapping,
attrs: Mapping) -> EOProduct: ... # DataTree per ICD <5.3.3>A
def write_product(product: EOProduct, uri: str, store_params: Mapping, mode: str) -> None: ...
def read_product(uri: str, store_type: str = "zarr") -> EOProduct: ... # via EOZarrStore; raises ProductWriteError
def extract_band_stack(product: EOProduct, bands: list[str]) -> BandStack: ...
# C-COM-IO (…common.io) — REQ-I-06, REQ-PORT-02/03
def resolve_uri(uri: str) -> "StoreHandle": ... # local FS | posix | s3:: (no container/Dask/S3 needed for local)
def open_store(uri: str, mode: str, store_params: Mapping | None = None) -> EOZarrStore: ...
# C-COM-ADF (…common.adf) — REQ-F-RAD-01, REQ-DAT-02, REQ-S-04
def resolve_adf(binding: "AdfBinding", acq_time, sensor_id, profile_id) -> AuxiliaryDataFile: ...
def load_adf(adf: AuxiliaryDataFile) -> "AdfData": ... # read-only
def check_validity(adf: AuxiliaryDataFile, acq_time, sensor_id) -> None: ... # raises AdfResolutionError
# C-COM-PROFILE (…common.profile) — REQ-AD-01/02, REQ-DAT-03
@dataclass(frozen=True)
class Profile: ... # typed mirror of ICD <5.3.6> schema
def load_profile(uri: str) -> Profile: ...
def validate_profile(raw: Mapping, schema_version: str) -> Profile: ... # JSON-Schema; raises ProfileValidationError
# C-COM-PROV (…common.provenance) — REQ-F-PRD-02, REQ-S-05
def build_provenance(run_ctx: "RunContext") -> dict: ... # ids/versions/params/timestamp; NO coefficients
# C-COM-QAFLAG (…common.qaflags) — REQ-F-QA-02
class QAFlag(enum.IntFlag): ... # the registry of <5.4.1>
def set_flag(mask: np.ndarray, condition: np.ndarray, flag: QAFlag) -> np.ndarray: ...
def merge(a: np.ndarray, b: np.ndarray) -> np.ndarray: ... # bitwise OR (monotone)
# C-COM-CHUNK (…common.chunking) — REQ-F-ORC-02, REQ-R-04, REQ-P-05
@dataclass(frozen=True)
class ChunkPlan: chunks: tuple[int, ...]; halo: int
def plan_chunks(shape, chunk_size, halo: int = 0) -> ChunkPlan: ...
def map_over_blocks(func, stack: BandStack, plan: ChunkPlan, use_dask: bool) -> BandStack: ...
# C-COM-CONFIG (…common.config) — REQ-AD-04, REQ-O-01
@dataclass(frozen=True)
class RunContext: profile: Profile; adf_ids: dict; params: dict; input_ids: list; processor_version: str
def resolve_config(payload: Mapping, profile: Profile) -> RunContext: ...
# C-COM-ORC (…common.orchestration) — REQ-F-ORC-01, REQ-F-DEP-01, REQ-I-05
@dataclass(frozen=True)
class RunResult: status: int; report: dict; outputs: dict[str, str]
def run_chain(payload: Mapping) -> RunResult: ... # parse → resolve → validate → execute PUs → publish
# FAIL_FAST: any MsiProcessorError → status!=0, no publish
# C-COM-CLI (…common.cli) — REQ-I-02, REQ-O-01, REQ-HF-01
def main(argv: list[str]) -> int: ... # `msi-processor <payload.{yaml,json}>`; exit code = RunResult.status
Per-component element-level data (dtypes/ranges/initial values) for the product DataTree, the ADF content and the profile schema are governed by the ICD (<5.3.3>/<5.3.2>/<5.3.6>) and reproduced as the internal data structures of <5.5>; they are not duplicated field-by-field here (Annex F <5.4.10> reference rule).
C-COM-PRODUCT. Type: library over CPM
EOProduct/EOGroup/EOZarrStore. Data: the output DataTree (measurements/conditions/quality+ root provenance/STAC attrs), chunking and CRS encoding per ICD <5.3.3>A/C/D. Errors:ProductWriteErroron a store failure (fail-stop).C-COM-IO. Type: library. Function: URI→store handle for local FS / POSIX / S3; guarantees a local-FS path with no container/Dask/S3 for CI (REQ-PORT-03). Errors: unresolvable URI ⇒
InputValidationError.C-COM-ADF. Type: library. Function: select the validity-/version-matched ADF, open read-only, load content. Errors:
AdfResolutionErroron missing/mismatched ADF (REQ-S-04).C-COM-PROFILE. Type: library. Function: load + JSON-Schema-validate the versioned profile to a typed
Profile; an invalid/incomplete profile ⇒ProfileValidationErrorbefore processing (REQ-DAT-03). Data: the profile schema is the in-model id space (paths = item ids, ICD <5.1>b).C-COM-QAFLAG. Type: library. Data: the fixed
QAFlagbitfield (ICD <5.3.3>E); the merge is bitwise-OR (monotone accumulation).C-COM-CHUNK. Type: library over Dask. Function: compute the block grid + halo per stage (<4.6>) and map the Core over blocks sequentially or via Dask; the halo guarantees spatially-coupled stages (ENH/COR/GEO/PAN) see sufficient context.
C-COM-CONFIG / C-COM-ORC / C-COM-CLI. Type: config library + executables. Function: turn the triggering payload (ICD <5.3.5>) into a
RunContext, topologically order and execute the active PUs honouringbreakpoints, enforce FAIL_FAST, publish on success, return exit status + report (ICD-IF-DIAG). Interfaces (Annex F <5.4.8>, executable): control in =main(argv)/run_chain, terminate = exit code; data in = payload + config/profile files, data out = exit status + report +EOProduct(s).
<5.4.12> C-SENSORS — sensor adaptation layer (data + schema)#
Identifier/type. msi_processor.sensors; logical: data + schema package; physical: a JSON-Schema
file (sensors/profile.schema.json), per-sensor profile JSON instances (sensors/<profile_id>.json) and
their ADF binding tables. Non-executable — no algorithm code (REQ-D-04). Purpose & trace.
Externalise all sensor-specific data so the generic chain is sensor-agnostic; a new sensor = a new profile
(+ private ADFs), no core change. Trace: REQ-AD-01..04, REQ-DAT-03, REQ-D-07; ICD-IF-PROF-*; DPM <7.4>.
Data (Annex F <5.4.11>). The profile object structure is the ICD <5.3.6> schema (id/version/sensor_id,
bands[], focal_plane, adf_bindings, per-stage parameter blocks, output, optional_stages,
breakpoints); element-level field ids, types, ranges and legality are controlled in the ICD and validated
by C-COM-PROFILE. The first instantiated profile is the project owner’s sensor (REQ-AD-02); its numeric
content is private (data policy). Subordinates. the schema file (parent) ‘composes’ the per-sensor
profile instances and the ADF-binding tables.
<5.5> Internal interface design#
Per Annex F <5.5>a/b the internal interfaces among the identified components, organised as the interfaces
map; per Annex F <5.5>c/e each interface lists its data elements (name, type, dimension, range, initial
value). The previously-postponed file-level data structures (Annex F <5.5>d — the on-disk EOProduct/Zarr
layout and the profile-file schema) are now controlled in the ICD (RD-5 <5.3.2/3/6>) and referenced here.
Interface inventory (component ↔ component).
If id |
From → To |
Mechanism |
Data (element level) |
|---|---|---|---|
IF-CHAIN-01 |
C-COM-ORC → C-PU-*.unit |
|
|
IF-PROD-01 |
C-PU-L0 → C-PU-RAD |
in-memory |
|
IF-PROD-02 |
C-PU-RAD → C-PU-ENH → C-PU-TOA |
|
corrected DN float32→uint16; |
IF-PROD-03 |
C-PU-TOA → C-PU-COR ( |
|
|
IF-PROD-04 |
C-PU-COR → C-PU-GEO ( |
|
co-registered → orthorectified TOA reflectance (no pan-sharpening), |
IF-PROD-05 |
C-PU-GEO → C-PU-ATM ( |
|
BOA reflectance, |
IF-PROD-06 |
C-PU-ATM → C-PU-PAN (opt, terminal) |
|
BOA-MS fused with PAN to PAN resolution, spectral-fidelity QA; default-off (PAN-reflectance handling |
IF-CORE-01 |
C-PU-.unit → C-PU-.core |
pure function call |
|
IF-SVC-01 |
C-PU-*.unit → C-COM-PRODUCT |
function call |
|
IF-SVC-02 |
C-PU-*.unit → C-COM-ADF |
function call (URI) |
|
IF-SVC-03 |
C-PU-*.unit → C-COM-PROFILE |
function call |
|
IF-SVC-04 |
C-PU-*.unit → C-COM-QAFLAG |
function call |
|
IF-SVC-05 |
C-PU-*.unit → C-COM-PROV |
function call |
provenance |
IF-SVC-06 |
C-COM-ORC/PRODUCT → C-COM-IO |
function call |
URI→ |
IF-SVC-07 |
C-PU-*.unit → C-COM-CHUNK |
function call |
|
IF-TRIG-01 |
E4 → C-COM-CLI/ORC |
triggering JSON (ICD <5.3.5>) |
|
Element-level data structures of the dominant interfaces (Annex F <5.5>c/e):
Inter-PU
EOProduct(IF-PROD-01..05) — the on-disk/in-memory DataTree, dtypes, dims, chunking, CRS encoding, QA bitfield, provenance fields, ranges and fill values are defined element-by-element in ICD <5.3.3> tables A–E and are the normative file-level structure (Annex F <5.5>d). Measurement variables:uint16packed (scale_factor/add_offset) orfloat32reflectance[0.0,1.0],_FillValue= profile sentinel, initial value n/a (computed); QAquality/mask/<band>uint16bitmask, initial value0; provenance root attrs (ids/versions/params/timestamp), no coefficients.BandStack/BandImage(IF-CORE-01) — the pure-core payload of <5.4.1>:datafloat32(working) /uint16(DN), dims(line|y, detector|x), range[0,2^B−1]or[0,1];qauint16initial0;name,geom. This is deliberately CPM-free so Cores are testable off-platform (REQ-D-03, REQ-PORT-03).Profile (IF-SVC-03) — the typed
Profilemirrors the ICD <5.3.6> schema; the JSON-Schema file is the element-level definition (field ids = dotted paths, ICD <5.1>b).
Interface-map notes. The level-breakpoint EOProduct (IF-PROD-03/04/05, DPM-BKP-*) is the
dominant internal interface and the point of optional Zarr persistence (resume semantics, DPM <9>). The
Core↔Wrapper interface (IF-CORE-01) is uniform across PUs, which is what lets a new stage or a new
profile slot in without touching the others (REQ-M-04, REQ-D-07). The cross-cutting service interfaces
(IF-SVC-*) are uniform across PUs (the six wrapper steps of <5.4.1>).
<6> Requirements to design components traceability#
Per Annex F <6>a this clause gives the forward (requirement→component) and backward
(component→requirement) traceability. The authoritative, tool-maintained matrix — including the
downward trace to unit/integration test cases and the upward link to DPM DPM-M-* and ATBD ALG-* — is
RD-11; per Annex F <6>b this clause is its design-level summary and the DJF reference.
<6.1> Forward trace — SRS REQ-* → design component (+ DPM/ATBD)#
SRS requirement group |
Design component(s) |
DPM / ATBD |
|---|---|---|
REQ-F-L0-01..05 |
C-PU-L0 (+ C-COM-PRODUCT, C-COM-PROFILE, C-COM-ADF, C-COM-QAFLAG) |
DPM-M-L0 / ALG-L0-* |
REQ-F-RAD-01..05 |
C-PU-RAD (+ C-COM-ADF, C-COM-QAFLAG) |
DPM-M-RAD / ALG-RAD-* |
REQ-F-ENH-01..03 |
C-PU-ENH (+ C-PU-QA) |
DPM-M-ENH / ALG-ENH-* |
REQ-F-TOA-01..03 |
C-PU-TOA (+ C-COM-ADF, C-COM-PRODUCT, C-COM-PROV) |
DPM-M-TOA / ALG-TOA-* |
REQ-F-COR-01..03 |
C-PU-COR (+ C-COM-QAFLAG, fail-stop via C-COM-ORC) |
DPM-M-COR / ALG-COR-* |
REQ-F-GEO-01..04 |
C-PU-GEO (+ C-COM-ADF, C-COM-PRODUCT) |
DPM-M-GEO / ALG-GEO-* |
REQ-F-PAN-01/02 |
C-PU-PAN (+ C-PU-QA) |
DPM-M-PAN / ALG-PAN-* |
REQ-F-ATM-01..04 |
C-PU-ATM (+ C-COM-ADF, C-COM-PRODUCT) |
DPM-M-ATM / ALG-ATM-* |
REQ-F-QA-01/02 |
C-PU-QA, C-COM-QAFLAG |
DPM-M-QA / ALG-QA-* |
REQ-F-PRD-01/02 |
C-COM-PRODUCT, C-COM-PROV |
DPM-M-PRD |
REQ-F-ORC-01/02 |
C-COM-ORC, C-COM-CHUNK |
DPM-M-PRD / DPM-BKP-* |
REQ-F-DEP-01/02 |
C-COM-ORC (fail-stop), all Cores (determinism, seeds; REQ-D-05) |
DPM-M-PRD / ATBD <6> |
REQ-P-01..05 |
C-PU-RAD/TOA (RAD_ACC), C-PU-COR/GEO (GEO_CE90/BAND_COREG), C-PU-ATM (BOA_ACC), C-COM-CHUNK (THRU/MEM) |
DPM <7.4> |
REQ-I-01..07 |
C-COM-CLI, C-COM-ORC, C-COM-IO, C-COM-PRODUCT (ICD-bound) |
ICD-IF-* |
REQ-O-01..04 |
C-COM-CLI, C-COM-ORC (logs/report/status/modes) |
ICD-IF-DIAG |
REQ-R-01..05 |
architecture-wide (pure Python/CPU; C-COM-CHUNK sizing; REQ-R-05 N/A) |
— |
REQ-D-01..09 |
the architecture itself (PU pattern, profile layer, reuse, numerics, Zarr) + the typed exception/seed design (<5.4.1>) |
— |
REQ-S-01..05 |
C-COM-ADF (private by URI), C-COM-PROV (no calib in output), C-COM-IO |
ICD-IF-ADF/OUT |
REQ-PORT-01..03 |
C-COM-IO, Core/Wrapper split, C-COM-CHUNK (local-FS path) |
ICD-IF-TRIG-03 |
REQ-AD-01..05 |
C-SENSORS, C-COM-PROFILE, C-COM-CONFIG |
ICD-IF-PROF |
REQ-DAT-01..03 |
C-COM-PRODUCT, C-COM-ADF, C-COM-PROFILE |
ICD <5.3> |
REQ-Q/REL/M/SAF/DEL/HF-* |
toolchain + C-COM-PROFILE/PROV/QAFLAG + C-SENSORS + <4.5>/<4.7> |
— |
<6.2> Backward trace — component → SRS REQ-*#
The backward trace is the inverse of the <5.3> allocation table (each component’s Allocated REQ-*
column) and of <6.1>; it is consolidated and kept current in RD-11. Every design component traces to at
least one REQ-* (no orphan components), every REQ-* is allocated to at least one component (no
unimplemented requirement), and every component additionally links to its DPM module / ATBD algorithm
(the DPM/ATBD column of <5.3> and <6.1>). The closure to unit/integration test cases is held in RD-11
and exercised per the V&V plan (RD-8) and the SRS/ICD validation matrices.
<6.3> Measures for critical software components (Annex F <6>c)#
msi-processor is Category C (degraded/incorrect data only; reprocessable; SDP §5.6). The residual
hazard class is product-data integrity (REQ-SAF-01). The design measures, per ECSS-Q-ST-80 6.2.2.4
(minimising critical components), now realised at element level:
No hard-coded instrument constants in the core — all externalised to the profile/ADFs (REQ-D-04, <4.1>c, <5.4.1>/<5.4.12>); the Cores take only arrays + a typed parameter object, so a calibration error is a data fix, not a code fix.
Pure-core isolation behind a tested interface — the
core/unitsplit and the CPM-freeBandStackcontract (IF-CORE-01) make numerical kernels unit-verifiable in isolation and off-platform (REQ-D-03), with per-stage tolerances documented in the DPM/ATBD (REQ-D-05).Bounded cyclomatic complexity (
xenonthresholds) percore/unitmodule; the thin-wrapper template keeps the CPM-facing code trivial (REQ-D-04, REQ-Q-04).Typed fault containment — the
MsiProcessorErrorhierarchy (<5.4.1>) makes every failure a typed, flagged, reported event at the PU boundary; no silent cross-DAG propagation.Determinism/reproducibility for a fixed input/ADF/profile/processor quadruple, including profile-fixed RANSAC seeds (REQ-F-DEP-02, REQ-D-05).
Fail-stop + QA flags + provenance as the integrity triad (REQ-F-DEP-01, REQ-F-QA-02, REQ-F-PRD-02); no partial product is ever published as complete.
No software-criticality-driven special design measures beyond the above (e.g. redundancy, watchdog, N-version) are levied at Category C; this is recorded to close Annex F <6>c. The detailed justification is carried in the DJF.
End of SDD (detailed / CDR issue). Authored per ECSS-E-ST-40C Rev.1 Annex F. This issue baselines the
element-level per-component design (<5.4>) and internal-interface data (<5.5>) at CDR, superseding the PDR
architectural issue. Requirements are in the SRS (RD-4); algorithm basis in the DPM (RD-6) / ATBD (RD-7);
concrete interfaces in the ICD (RD-5); reuse heritage in the SRF (RD-10); the maintained traceability
matrix is RD-11. Code internals marked [impl] are completed in implementation (SDP WP-5), which starts
only after CDR.