Software Development Plan (SDP)#


Document	Software Development Plan (SDP)
DRD	ECSS-E-ST-40C Rev.1, Annex O
Container	Management File (MGT)
Project	`msi-processor` — generic high-resolution MSI data processor
Configuration item	`gitlab.eopf.copernicus.eu/ipf/msi-processor`
Software criticality	Category C (ECSS-Q-ST-80C Rev.2 / ECSS-E-ST-40C Annex R)
Baselined at	SRR
Status	Draft for SRR

This SDP is the top-level management artefact of the ECSS software life cycle for the msi-processor project. It follows the ECSS-E-ST-40C Rev.1 Annex O DRD section structure and records the management and development approach, the life-cycle model, the review milestones, and — through the tailoring in §5.5 / §5.6 — the documentation tree the project commits to. The footprint is tailored to a Category C, small-team ground-segment processor.

<1> Introduction#

The msi-processor is an operational ground-segment data processor that transforms downlinked raw (Level-0) multispectral imager (MSI) data into calibrated and geophysically usable products up to Level 2. It is designed as a generic high-resolution pushbroom MSI processor: the processing chain (decoding → radiometric correction → geometric correction → atmospheric correction) is sensor-agnostic and driven by a per-sensor configuration/profile, the first instantiated profile being the project owner’s own sensor.

The processor is built on the ESA Earth Observation Processing Framework (EOPF): each processing stage is an EOPF Core Python Modules (CPM) EOProcessingUnit, products are handled as EOPF EOProduct objects, and outputs are written as cloud-native Zarr. The processing algorithms are not developed from scratch — their mathematical basis already exists in prior work (see SRF) — so this project is fundamentally an integration and ECSS-compliant productisation effort rather than new-algorithm research.

The purpose of this SDP is to describe the established management and development approach for the software items of msi-processor, in accordance with ECSS-E-ST-40C Rev.1. It is prepared at project start to establish the SRR baseline and is maintained throughout the life cycle.

<2> Applicable and reference documents#

Applicable documents

Ref	Document
AD-1	ECSS-E-ST-40C Rev.1 (30 April 2025) — Space engineering — Software
AD-2	ECSS-Q-ST-80C Rev.2 (30 April 2025) — Space product assurance — Software product assurance
AD-3	ECSS-M-ST-10C Rev.1 — Space project management — Project planning and implementation
AD-4	ECSS-M-ST-40C — Configuration and information management

Reference documents

Ref	Document
RD-1	EOPF Software Development Environment (SDE) — User Manual & Guidelines
RD-2	EOPF Core Python Modules (CPM) documentation (`eopf == 2.8.1`)
RD-3	Prior work — multispectral pushbroom preprocessing pipeline (algorithm heritage; see SRF)
RD-4	ECSS-E-ST-40C Rev.1 Annex R — Tailoring based on software criticality
RD-5	ECSS-E-ST-40C Rev.1 Annex Q — Document organization and contents at each review

<3> Terms, definitions and abbreviated terms#

Terms and definitions follow AD-1, AD-2 and the EOPF SDE glossary. Abbreviations used in this document and not defined in the applicable/reference documentation:

Abbreviation	Definition
ADF	Auxiliary Data File
ATBD	Algorithm Theoretical Basis Document
CPM	(EOPF) Core Python Modules
DPM	Data Processing Model
DRD / DRL	Document Requirements Definition / Document Requirements List
EOPF	Earth Observation Processing Framework
MSI	Multispectral Imager
NUC	Non-Uniformity Correction
PU	(EOPF CPM) Processing Unit
SDE	(EOPF) Software Development Environment
TOA	Top Of Atmosphere

(ECSS review acronyms SRR/PDR/CDR/QR/AR and DRD acronyms SDP/SRS/SDD/ICD/… per AD-1.)

<4> Software project management approach#

<4.1> Management objectives and priorities#

Management objectives, in priority order:

Standards conformance — develop the processor under the ECSS-E-ST-40C Rev.1 life cycle (documentation-first, milestone-driven) so that progress, rationale and traceability are auditable from the Git history.
Correctness & verifiability — every processing stage is traceable to a requirement and verified against deterministic unit tests and, where data permits, reference products.
Reuse over reinvention — integrate the existing algorithm heritage and the EOPF CPM framework rather than redeveloping (see SRF).
Reproducibility & portability — pinned environment (eopf == 2.8.1), Zarr outputs, CI-enforced quality gates.

<4.2> Master schedule#

The project schedule is milestone-driven and tracked as GitLab group milestones on ipf (SRR, PDR, CDR, QR, AR). The master schedule references those milestones; document baselines per milestone are defined in §5.2.3. (Calendar dates are managed in the GitLab milestones, not duplicated here.)

<4.3> Assumptions, dependencies and constraints#

Assumptions: the algorithm mathematical basis is available from prior work; the owner’s raw data and the corresponding instrument calibration (gain/offset, dark, flat-field) are available for local verification.
Dependencies: EOPF SDE GitLab platform and CI runner; EOPF CPM (eopf == 2.8.1) build image; availability of the sensor calibration auxiliary data.
Constraints:
- eopf is pinned to 2.8.1 to match the SDE cpm-build-environment image; it is not upgraded (a version change would desynchronise the build environment and break CI).
- Code is public; raw input data and calibration are private — they are never committed and never used in public CI. Numerical verification on real data is performed locally.
- The CI runner is a shell executor (no container runtime, no Dask gateway, no S3 in CI); jobs requiring those tools are non-blocking (allow_failure) until a Kubernetes runner is available.

<4.4> Work breakdown structure#

Top-level work packages (WP), in execution order:

WP	Title	Milestone
WP-1	Project setup (repository, CI, runner, milestones)	(pre-SRR, done)
WP-2	SRR documentation (SDP, SRevP, SPAP, Risk Register, SSS, IRD)	SRR
WP-3	PDR documentation (SRS, SVerP, SValP, ICD, DPM, ATBD, preliminary SDD)	PDR
WP-4	CDR documentation (detailed SDD+DJF, SUITP, SRF, traceability matrix)	CDR
WP-5	Implementation — processing units L0→L2 (per design)	post-CDR
WP-6	Verification & validation (SVR, SUITR), release (SRelD/SRN), SUM	QR / AR

Detailed activities are managed as GitLab issues assigned to the corresponding milestone.

<4.5> Risk management#

The software engineering function contributes to project risk management per ECSS-M-ST-80. Risks are recorded and maintained in the Risk Register (compliance/drd/risk-register.md), reviewed at each milestone.

<4.6> Monitoring and controlling mechanisms#

Work is monitored through: GitLab milestones and issues (progress), merge requests (change control and review), and the CI pipeline status (quality gates). Each documentation/work item is a merge request linked to its milestone, giving an auditable progress trail.

<4.7> Staffing plan#

Single-developer project (project owner acting as supplier-side software, PA and verification roles). Independence of verification is achieved by automated tooling and explicit review checklists rather than separate staff; this is consistent with the Category C tailoring (§5.6).

<4.8> Software procurement process#

No software is procured. All third-party software is open-source reuse (EOPF CPM and its dependencies, scientific Python stack); the reuse declaration and licences are recorded in the SRF. Therefore the procured-software list is empty.

<4.9> Supplier management#

Not applicable beyond §4.8 — there are no subcontracted software suppliers.

<5> Software development approach#

<5.1> Strategy to the software development#

The overall strategy is documentation-first systems engineering followed by incremental implementation: the requirements, interfaces, data processing model, algorithm basis and design are baselined through SRR → PDR → CDR before code is written; implementation (WP-5) starts only after CDR. Each processing stage is then implemented incrementally as a CPM EOProcessingUnit (pure, unit-testable core + thin PU wrapper), integrated and verified against its requirement before the next stage. Git history therefore mirrors the engineering order.

<5.2> Software project development life cycle#

<5.2.1> Software development life cycle identification#

The life-cycle paradigm is incremental, synchronised by the formal ECSS reviews. Software versioning follows semantic versioning with Git tags; the configuration item is the main branch of the Git repository (see SCF). The SDP covers the implementation of all relevant software processes: system-related, requirements & architecture, design & implementation, validation, verification, delivery & acceptance, operation, maintenance, and management.

<5.2.2> Relationship with the system development cycle#

The processor is a ground-segment software product. Its life cycle is phased to the standard EO ground-segment process model; the milestones below are the synchronisation points.

<5.2.3> Reviews and milestones identification and associated documentation#

Reviews are implemented as GitLab group milestones (group ipf). Each review baselines the documents listed (see §5.5 for the full tailored DRL and Annex Q for content-per-review):

Review	Scope / purpose	Baselined documents
SRR	System & management baseline	SDP, SRevP, SPAP, Risk Register, SSS, IRD
PDR	Software requirements + V&V plans + preliminary design	SRS, SVerP, SValP, ICD (start), DPM, ATBD, preliminary SDD
CDR	Detailed design + test/reuse	SDD (+DJF), SUITP, SRF, ICD (final), Traceability matrix
QR	Qualification	SVR, SUITR, SRelD/SRN, SUM (start), CIDL/SCF
AR	Acceptance	SUM (final), SRelD, SMP

Level of formalism: reviews are conducted as documented, checklist-based merge-request reviews (proportionate to a single-developer Category C project; see SRevP). Implementation begins after CDR.

<5.3> Software engineering standards and techniques#

Methodologies/standards: ECSS-E-ST-40C Rev.1 (engineering) and ECSS-Q-ST-80C Rev.2 (PA), tailored for Category C (§5.6). Coding follows PEP 8 enforced by black/ruff/flake8.
Requirements analysis method: textual, uniquely-identified requirements (REQ-*) with forward/backward traceability (see SRS and Traceability matrix).
Design method: EOPF CPM processing-unit architecture — pure-function core + thin EOProcessingUnit wrapper, with mandatory inputs/ADFs/outputs/parameters declared in CPM computing-model JSON; sensor-specific behaviour externalised to a sensor-profile layer.
Auto-code generation: none.
Delivery format: Python package + container image + Zarr product specification (see SRelD).

<5.4> Software development and software testing environment#

Language: Python 3.11. Dependency/build: uv / flit; environment pinned via pyproject.toml (eopf == 2.8.1).
Build image: registry.eopf.copernicus.eu/sde/cpm-build-environment:latest (provides CPM).
CI/CD: GitLab CI on the EOPF SDE; self-hosted shell runner on the Studio VM.
Quality/static analysis tools: flake8, black, isort (style); mypy (typing); bandit + trivy (security); xenon (complexity); SonarQube (quality gate); pytest + coverage (test).
Documentation: Sphinx → GitLab Pages.
Note: tools requiring a container runtime / Dask gateway / S3 are not available on the shell runner and are configured non-blocking until a Kubernetes runner lands.

<5.5> Software documentation plan#

<5.5.1> General#

All project documentation follows the ECSS-E-ST-40C Rev.1 DRDs (engineering) and ECSS-Q-ST-80C Rev.2 (PA), tailored for Category C. Formal DRD deliverables are authored in compliance/ (authoritative) and the EOPF SDE-rendered subset is published from docs/ (Sphinx/Pages). The two are kept consistent.

<5.5.2> Software documentation identification — tailored DRL#

For each document: file location, document name, the review at which it is delivered, and the tailoring decision (PRODUCE / REUSE / TAILORED-OUT). Tailoring is per Annex R for Category C.

ECSS-E-ST-40C (engineering)

DRD	Annex	Decision	Delivered	File
SSS — Software System Specification	B	PRODUCE (light)	SRR	`compliance/drd/sss-software-system-specification.md`
IRD — Interface Requirements Document	C	PRODUCE	SRR	`compliance/drd/ird-interface-requirements.md`
SRS — Software Requirements Specification	D	PRODUCE	PDR	`compliance/drd/srs-software-requirements.md`
ICD — Interface Control Document	E	PRODUCE	PDR/CDR	`compliance/drd/icd-interface-control.md` + `docs/icd.md`
SDD — Software Design Document	F	PRODUCE	CDR	`compliance/drd/sdd-software-design.md` + `docs/sdd/`
DPM — Data Processing Model	(EOPF)	PRODUCE	PDR	`docs/dpm/`
ATBD — Algorithm Theoretical Basis	(EO)	PRODUCE	PDR	`docs/atbd/`
SRelD — Software Release Document	G	PRODUCE (light)	QR/AR	`docs/srn.md` + GitLab Releases
SUM — Software User Manual	H	PRODUCE	QR/AR	`docs/sum/`
SVerP — Software Verification Plan	I	PRODUCE (merged)	PDR	`compliance/drd/vv-plan.md`
SValP — Software Validation Plan	J	PRODUCE (merged)	PDR	`compliance/drd/vv-plan.md`
SUITP — Unit/Integration Test Plan	K	PRODUCE (merged)	CDR	`compliance/drd/vv-plan.md`
SVS — Software Validation Specification	L	REUSE	—	test suite + V&V plan
SVR — Software Verification Report	M	PRODUCE	QR	`compliance/drd/vv-report.md`
SRF — Software Reuse File	N	PRODUCE	CDR	`compliance/drd/srf-software-reuse-file.md` + `docs/srf.md`
SDP — Software Development Plan	O	PRODUCE	SRR	this document
SRevP — Software Review Plan	P	PRODUCE (light)	SRR	`compliance/drd/srevp-software-review-plan.md`
SMP — Software Maintenance Plan	T	TAILORED-OUT (light)	AR	maintenance = GitLab issues + SemVer; noted in SRS

ECSS-Q-ST-80C (product assurance) and cross-discipline

Artefact	Decision	Delivered	File
SPAP — SW Product Assurance Plan	PRODUCE (light)	SRR	`compliance/drd/spa-plan.md`
Risk Register (M-ST-80)	PRODUCE	SRR	`compliance/drd/risk-register.md`
Traceability matrix (E-40 §5.8)	PRODUCE	CDR	`compliance/traceability/traceability-matrix.md`
CIDL — Configuration Item Data List	PRODUCE (light)	QR	`docs/cidl.md`
SCF — Software Configuration File	PRODUCE (light)	QR	`docs/scf.md`

<5.5.3> Deliverable items#

Deliverable items: the documentation set above; the software package (msi_processor); the container image; and the Zarr product specification. Internal vs deliverable status, sender/ receiver, perimeter, maturity and known limitations of each delivery are recorded in the SRelD.

<5.5.4> Software documentation standards#

Documents are authored in Markdown (CommonMark/MyST), rendered with Sphinx. Each formal DRD document states its DRD reference and follows the corresponding Annex section structure. Any tailoring of a DRD’s content is recorded in that document’s header.

<5.6> This Standard’s tailoring traceability#

The applicable tailoring of ECSS-E-ST-40C clause 5 is derived from Annex R for software criticality Category C. Rationale for the classification: a failure of msi-processor produces degraded or incorrect data products only — it has no safety, mission-loss or space-segment consequence and is recoverable by reprocessing — which corresponds to a major severity, i.e. Category C.

The document-level tailoring (DRL) is given in §5.5.2. The detailed clause-by-clause coverage matrix of ECSS-E-ST-40C clause 5 (each requirement: applied / tailored / N/A with rationale) is maintained in compliance/gap-analysis.md and referenced here. TAILORED-OUT items with rationale: SMP (light — GitLab issues + SemVer); SPAMR (no formal PA milestone reports — subsumed by SVR and the milestone reviews).

End of SDP. Authored per ECSS-E-ST-40C Rev.1 Annex O.