msi_processor.computing.atmospheric package

msi_processor.computing.atmospheric package#

Atmospheric-correction processing unit (C-PU-ATM; DPM-M-ATM).

TOA -> BOA (surface) reflectance inversion (6S/Vermote, ATBD <5.8>) plus scene classification and cloud / cloud-shadow masking; emits the L2A product.

Submodules#

msi_processor.computing.atmospheric.core module#

Pure atmospheric-correction core (C-PU-ATM; ALG-ATM-PAR/RT/SCM).

CPM-free, I/O-free functions implementing the atmospheric-correction algorithms of ATBD <5.8>. This stage has no prior-work heritage; it is new development specified interface-first (SDD <5.4.9>) with a candidate algorithm whose RT engine and parameter-retrieval/classifier bodies are down-selected in the DPM and left as private [impl] stubs.

Algorithm split (operational baseline vs [impl])#

Two boundaries follow the same pattern as the geo-referencing core (an operational baseline now, the rigorous body deferred to the DPM/CDR):

  • ALG-ATM-RT (TOA -> BOA inversion). The 6S surface-reflectance inversion itself is exact algebra once the per-band radiative-transfer terms \(\{\rho_{\mathrm{atm}}, T, S\}\) are known; it is realised here (invert_boa(), toa_to_boa()) and fully unit-tested. The RT engine that *builds/interpolates* the LUT for the per-pixel geometry / AOT / water vapour / surface altitude (Py6S / 6SV / Sen2Cor-LUT / ACOLITE – down-selected in the DPM) is the deferred body resolve_rt_lut() ([impl]). The unit consumes an already-resolved RTLut for the scene operating point.

  • ALG-ATM-PAR (parameter ingest/retrieval). Ingesting AOT / water vapour / ozone from auxiliary meteorology is realised here (get_atmospheric_parameters() with mode="ingest"). Retrieving them from the imagery (dark-dense-vegetation AOT inversion, differential band-ratio water vapour) is the deferred body (mode="retrieve" -> retrieve_atmospheric_parameters(), [impl]).

  • ALG-ATM-SCM (scene classification & masks). A spectral-threshold scene classifier (Sen2Cor-style candidate, ATBD <5.8>) is realised here (classify_scene()) producing a scene-class layer plus cloud / cloud-shadow masks. A sophisticated (e.g. ML) classifier is the deferred refinement (classify_scene_ml(), [impl]).

Theory (ATBD <5.8>). Over a Lambertian surface the TOA reflectance is \(\rho_{\mathrm{TOA}} = \rho_{\mathrm{atm}} + T(\theta_s)T(\theta_v)\rho_s / (1 - S\rho_s)\); inverting for the surface (BOA) reflectance gives \(\rho_{\mathrm{BOA}} = (\rho_{\mathrm{TOA}} - \rho_{\mathrm{atm}}) / (T + S(\rho_{\mathrm{TOA}} - \rho_{\mathrm{atm}}))\) with \(T = T(\theta_s)T(\theta_v)\).

Trace: REQ-F-ATM-01..04; DPM-M-ATM; ALG-ATM-PAR/RT/SCM.

class msi_processor.computing.atmospheric.core.AtmAux(aot, water_vapour, ozone=None)#

Bases: object

Auxiliary atmospheric inputs ingested from meteorology ADFs (ingest mode).

Mirrors the atmospheric ADF content for ALG-ATM-PAR mode="ingest".

Attributes:
ozone
aot: Union[float, ndarray[tuple[Any, ...], dtype[float64]]]#
ozone: Optional[float] = None#
water_vapour: Union[float, ndarray[tuple[Any, ...], dtype[float64]]]#
class msi_processor.computing.atmospheric.core.AtmParams(aot, water_vapour, ozone=None)#

Bases: object

Atmospheric state used by the RT inversion (ALG-ATM-PAR output).

Attributes:
aot:

Aerosol optical thickness (550 nm), scalar or per-pixel field.

water_vapour:

Columnar water vapour (g/cm^2), scalar or per-pixel field.

ozone:

Columnar ozone (Dobson units), optional scalar.

aot: Union[float, ndarray[tuple[Any, ...], dtype[float64]]]#
ozone: Optional[float] = None#
water_vapour: Union[float, ndarray[tuple[Any, ...], dtype[float64]]]#
class msi_processor.computing.atmospheric.core.RTLut(path_reflectance, transmittance, spherical_albedo=<factory>)#

Bases: object

Per-band radiative-transfer terms for the scene operating point (ALG-ATM-RT).

Each mapping is keyed by band id. The terms are the Vermote/6S decomposition:

  • path_reflectance – intrinsic (path) reflectance \(\rho_{\mathrm{atm}}\);

  • transmittance – total two-way transmittance \(T = T(\theta_s)T(\theta_v)\);

  • spherical_albedo – atmospheric spherical albedo \(S\).

Producing this table from an RT engine / pre-computed LUT for a given AtmParams + SceneGeometry + DEM altitude is the [impl] resolve_rt_lut().

path_reflectance: Mapping[str, float]#
spherical_albedo: Mapping[str, float]#
transmittance: Mapping[str, float]#
msi_processor.computing.atmospheric.core.ScalarOrArray#

A radiative quantity that may be a per-scene scalar or a per-pixel field.

alias of Union[float, ndarray[tuple[Any, …], dtype[float64]]]

class msi_processor.computing.atmospheric.core.SceneClass#

Bases: object

Scene-classification label codes (Sen2Cor-style subset, ALG-ATM-SCM).

Stored as a uint8 layer in quality/scene_classification of the L2A product. NO_DATA is reserved for fill; the cloud / cloud-shadow classes are mirrored into the per-band QA flag layer (QAFlag).

BARE_SOIL = 4#
CLOUD = 1#
CLOUD_SHADOW = 2#
NO_DATA = 0#
UNCLASSIFIED = 6#
VEGETATION = 3#
WATER = 5#
class msi_processor.computing.atmospheric.core.SceneGeometry(sun_zenith, view_zenith=0.0, relative_azimuth=0.0)#

Bases: object

Per-scene illumination/view geometry operating point (degrees).

The LUT is resolved at this operating point; rigorous per-pixel variation is handled inside the [impl] resolve_rt_lut().

relative_azimuth: float = 0.0#
sun_zenith: float#
view_zenith: float = 0.0#
msi_processor.computing.atmospheric.core.classify_scene(boa, params)#

ALG-ATM-SCM – spectral-threshold scene classification + masks.

A Sen2Cor-style spectral-threshold classifier (ATBD <5.8> candidate). It derives, from the BOA stack, a scene-class layer (SceneClass codes) plus boolean cloud and cloud-shadow masks. Spectral roles (blue, green, red, nir) are resolved via params["band_roles"] or common band names; vegetation/water use NDVI/NDWI, clouds use a brightness threshold and shadows a low-NIR/low-brightness threshold. A more sophisticated (ML) classifier is the deferred classify_scene_ml() ([impl]).

Thresholds (overridable via params): cloud_brightness (0.5), shadow_nir (0.08), shadow_brightness (0.12), ndvi_veg (0.4), ndwi_water (0.2).

Return type:

tuple[ndarray[tuple[Any, ...], dtype[uint8]], ndarray[tuple[Any, ...], dtype[bool]], ndarray[tuple[Any, ...], dtype[bool]]]

Returns:
tuple

(scene_class[uint8], cloud_mask[bool], cloud_shadow_mask[bool]).

Raises:
InputValidationError

If the BOA stack is empty.

msi_processor.computing.atmospheric.core.classify_scene_ml(boa, params)#

ALG-ATM-SCM refinement – learned scene classifier, [impl].

A trained (e.g. CNN / random-forest) scene classifier is the CDR refinement of the spectral-threshold baseline (classify_scene()). The model and its training data are down-selected in the DPM; this fail-stops until then.

Return type:

tuple[ndarray[tuple[Any, ...], dtype[uint8]], ndarray[tuple[Any, ...], dtype[bool]], ndarray[tuple[Any, ...], dtype[bool]]]

msi_processor.computing.atmospheric.core.get_atmospheric_parameters(toa_refl, mode, aux, params)#

ALG-ATM-PAR – obtain the atmospheric state (ingest or retrieve).

Return type:

AtmParams

Parameters:
toa_refl:

TOA reflectance band stack (only used by the retrieval path).

mode:

"ingest" reads AOT / water vapour / ozone from aux (auxiliary meteorology); "retrieve" inverts them from the imagery and is the deferred [impl] body (DDV AOT, band-ratio water vapour, ATBD <5.8>).

aux:

Ingested auxiliary atmospheric inputs (mandatory for mode="ingest").

params:

Profile parameters (e.g. default fallbacks); reserved for the retrieval path.

Returns:
AtmParams

The atmospheric state for the RT inversion.

Raises:
InputValidationError

On an unknown mode or a missing aux in ingest mode.

AtmosphericError

From the [impl] retrieval path (until down-selected in the DPM).

msi_processor.computing.atmospheric.core.invert_boa(toa, path_reflectance, transmittance, spherical_albedo)#

ALG-ATM-RT kernel – invert one band’s TOA reflectance to BOA.

Applies the 6S surface-reflectance inversion (ATBD <5.8>): :rtype: ndarray[tuple[Any, ...], dtype[float32]]

\[\rho_{\mathrm{BOA}} = \frac{\rho_{\mathrm{TOA}} - \rho_{\mathrm{atm}}} {T + S\,(\rho_{\mathrm{TOA}} - \rho_{\mathrm{atm}})}\]

with T the total two-way transmittance and S the spherical albedo. The result is clipped to the physical [0, 1] reflectance range.

Raises:
AtmosphericError

If the total transmittance is non-positive (degenerate / invalid LUT term).

msi_processor.computing.atmospheric.core.resolve_rt_lut(atm, geometry, dem, raw_lut)#

Interpolate the per-band RT terms for the scene operating point – [impl].

Multi-dimensional interpolation of a pre-computed radiative-transfer LUT (or a direct RT-engine call) over geometry, AOT, water vapour and surface altitude, yielding RTLut. The engine (Py6S / 6SV / Sen2Cor-LUT / ACOLITE) is down-selected in the DPM and licence-screened in the SRF; the inversion that consumes the result (toa_to_boa()) is operational. Fail-stops until the engine is selected (ATBD <5.8> open point 1).

Return type:

RTLut

msi_processor.computing.atmospheric.core.retrieve_atmospheric_parameters(toa_refl, params)#

ALG-ATM-PAR retrieval body – [impl] (DPM down-selection / CDR target).

Image-based retrieval of AOT (dark-dense-vegetation inversion) and water vapour (differential band-ratio absorption) per ATBD <5.8>. The concrete method is down-selected and validated in the DPM before CDR; until then this fail-stops so the chain never emits an L2A product from un-retrieved parameters.

Return type:

AtmParams

msi_processor.computing.atmospheric.core.toa_to_boa(toa_refl, atm, geometry, dem, rt_lut)#

ALG-ATM-RT – invert the TOA reflectance stack to BOA per band.

For each band the per-band RT terms are looked up from rt_lut (resolved at the scene operating point) and the 6S inversion (invert_boa()) is applied. The atm / geometry / dem arguments document the operating point the LUT was resolved at; per-pixel LUT interpolation lives in the [impl] resolve_rt_lut().

Return type:

dict[str, ndarray[tuple[Any, ...], dtype[float32]]]

Returns:
dict[str, ndarray]

BOA reflectance per band, float32 in [0, 1].

Raises:
AtmosphericError

If a band has no RT terms in the LUT (missing coverage).

msi_processor.computing.atmospheric.unit module#

Thin EOProcessingUnit wrapper for atmospheric correction (C-PU-ATM).

Adapts the pure core to the EOPF CPM runtime following the wrapper template of SDD <5.4.1>/<5.4.9>: read parameters, extract the TOA-reflectance bands, load/validate ADFs, orchestrate the pure-core functions (ALG-ATM-PAR -> ALG-ATM-RT -> ALG-ATM-SCM), propagate QA, and build the L2A EOProduct (BOA reflectance + scene classification + cloud/shadow masks). No algorithm lives here.

Input convention. The upstream stage is the georeference unit; its l1c product carries the orthorectified TOA reflectance under measurements/reflectance/<band> and QA under quality/mask/<band> (IF-PROD-03). Atmospheric correction inverts the TOA reflectance to BOA (surface) reflectance on the same grid; L2A is the surface-reflectance product (DPM <8.7>).

ADF data convention (this increment). ADF content is read from the AuxiliaryDataFile.data_ptr mapping:

  • atmospheric -> {"aot": float|ndarray, "water_vapour": float|ndarray, "ozone": float | None, "rt_lut": {"path_reflectance": {band: float}, "transmittance": {band: float}, "spherical_albedo": {band: float}}} (optional; required when mode="ingest": ingested AOT/water-vapour + the resolved per-band RT terms for the scene operating point, ICD <5.3.2>A). Building the rt_lut from a radiative-transfer engine is the [impl] resolve_rt_lut().

  • dem -> {"elevation": ndarray2d, ...} (mandatory; surface altitude for the RT operating point; presence/coverage enforced here, REQ-F-ATM-02).

Implementation status (ATBD <5.8> open point 1). Image-based parameter retrieval (mode="retrieve") and the RT engine that builds the LUT are [impl] (DPM down-selection / CDR target). The PDR-operational path realised here is parameter ingest (mode="ingest") with a pre-resolved RT-LUT from the atmospheric ADF, the exact 6S BOA inversion, and the spectral-threshold scene classifier.

Fail-stop (REQ-F-ATM-04). Missing DEM coverage, a missing atmospheric ADF in ingest mode, or the [impl] retrieval/RT-engine paths raise AtmosphericError; it propagates to the chain runner so no L2A product is emitted from incomplete atmospheric inputs. Cloud / cloud-shadow pixels are flagged CLOUD / CLOUD_SHADOW in the QA layer (REQ-F-ATM-03) but never abort the chain.

Mandatory inputs/ADFs are declared by the CPM computing-model JSON (models/msi_atmospheric_1.0.0.json), not by overriding the list methods.

Trace: REQ-F-ATM-01..04; DPM-M-ATM; ALG-ATM-PAR/RT/SCM; ICD IF-PROD-03, REQ-IF-OUT-02.

class msi_processor.computing.atmospheric.unit.AtmosphericUnit(identifier='')#

Bases: EOProcessingUnit

Atmospheric-correction processing unit (C-PU-ATM; SDD <5.4.9>).

Attributes:
identifier

Identifier of the processing step

Methods

run:

Invert the TOA-reflectance stack to BOA (surface) reflectance, classify the scene and emit the L2A product with cloud / cloud-shadow QA. Image-based parameter retrieval and the RT-LUT-building engine are [impl] (ATBD <5.8> open point 1); the operational path is ingest mode with a pre-resolved RT-LUT.
PROCESSOR_LEVEL = 'L2A'#
PROCESSOR_MODEL = True#
PROCESSOR_NAME = 'msi_atmospheric'#
PROCESSOR_VERSION = '1.0.0'#
run(inputs, adfs=None, mode=None, **kwargs)#

Run the atmospheric correction.

Return type:

Mapping[str, Union[EOProduct, EOContainer, DataTree, Iterable[Union[EOProduct, EOContainer, DataTree]]]]

Parameters:
inputs:

{"l1c": EOProduct} with TOA reflectance under measurements/reflectance/<band> and optional QA under quality/mask/<band>.

adfs:

atmospheric (required for param_mode="ingest": ingested AOT/water-vapour + a resolved RT-LUT) and dem (mandatory; surface altitude).

mode:

"nominal" (the only supported processing mode).

**kwargs:

param_mode ("ingest" | "retrieve", default "ingest"), sun_zenith / view_zenith / relative_azimuth (scene geometry operating point), scene-classification thresholds and band_roles (see classify_scene()), and an optional name for the output product.

Returns:
Mapping[str, DataType]

{"l2a": EOProduct} with BOA measurements/reflectance/<band>, quality/scene_classification and quality/mask/<band>.

Raises:
AtmosphericError

On missing DEM coverage, a missing atmospheric ADF in ingest mode, or the [impl] retrieval / RT-engine paths – fail-stop (REQ-F-ATM-04).

Help & feedback

Documentation

Edit this page in GitLab’s WebUI to fix an error or add an improvement in a merge request.
Create an issue to suggest improvements to this page.

Product

Create an issue if there is something you do not like about this feature or if you would like to propose a functionality by submitting a feature request.