From f045f266c11fd124baaac7e3848edb123698e3b8 Mon Sep 17 00:00:00 2001 From: bramjanssen Date: Mon, 2 Feb 2026 14:07:57 +0100 Subject: [PATCH 1/9] icg: updated definitions based on v4.1 --- interoperability/definitions.qmd | 70 +++++++++++++++++++++++--------- 1 file changed, 50 insertions(+), 20 deletions(-) diff --git a/interoperability/definitions.qmd b/interoperability/definitions.qmd index f6a78c1..19ff6d0 100644 --- a/interoperability/definitions.qmd +++ b/interoperability/definitions.qmd @@ -47,35 +47,33 @@ analytics procedures to perform an application-specific processing task. For the sake of simplicity in the context of APEx, we refer to these simply as (EO) “algorithms”. -### Algorithm Service Implementation +### Application Algorithm Definition -The algorithm definition refers to a representation of the algorithm modules and interfaces that can be exposed by an -API and/or processing platform. Typically, it includes a general description of the algorithm along with detailed +The application algorithm definition refers to a representation of the algorithm modules and interfaces that can be exposed +by an API and/or processing platform. Typically, it includes a general description of the algorithm along with detailed information on its parameters, expected output, scientific method, and an overview of the different steps executed within the algorithm. Examples of algorithm definitions include openEO’s User Defined Processes (UDP) [@udp] and OGC Application Package [@ap], using the Common Workflow Language (CWL) [@cwl]. -### Algorithm Catalogue +To increase uptake and interoperability, APEx aims to enable the execution of application algorithm definitions via +standardised web service APIs. This transitions algorithms from being rather arbitrary pieces of software with potentially +complex requirements, in terms of execution environment, usage, inputs, …, into on-demand services that can readily be +invoked by stakeholders. This transformation primarily involves making the application algorithm definition available as +a service on an algorithm hosting platform. The transition process into a hosted on-demand algorithm service is supported +by the APEx Algorithm Support. -The [APEx Algorithm Catalogue](https://algorithm-catalogue.apex.esa.int/) is a central register of algorithm +An important boundary condition for hosted algorithms is that they can be executed at a predictable cost for a given set +of inputs. This predictability allows a service user to accurately estimate and determine the cost associated with the +execution of the final deployed service instance. + +### Algorithm Services Catalogue + +The [APEx Algorithm Services Catalogue](https://algorithm-catalogue.apex.esa.int/) is a central register of algorithm definitions and the corresponding algorithm service instances that can be executed on APEx-compliant algorithm hosting platforms. Curated by APEx, the catalogue relies on automated checks to ensure that advertised algorithms service instances are available and functional. Whenever a malfunction is detected, this is reported to ESA and the EO project consortium, allowing them to determine a proper course of action. -### Hosted Algorithm - -To increase uptake and interoperability, APEx aims to enable the execution of algorithms via standardised web service -APIs. This transitions algorithms from being rather arbitrary pieces of software with potentially complex requirements, -in terms of execution environment, usage, inputs, …, into on-demand services that can readily be invoked by -stakeholders. This transformation primarily involves converting a given algorithm into an APEx-compliant algorithm -definition and making it available as a service on an algorithm hosting platform. The transition process into a hosted -on-demand algorithm service is supported by the [APEx Algorithm Support](../propagation/index.qmd). - -An important boundary condition for hosted algorithms is that they can be executed at a predictable cost for a given set -of inputs. This predictability allows a service user to accurately estimate and determine the cost associated with the -execution of the final deployed service instance. - ### Algorithm Hosting Platform An EO algorithm hosting platform enables the execution of a standardised algorithm, represented by an algorithm @@ -96,6 +94,38 @@ affects properties such as cost, performance, stability, and the amount of compu algorithm. Compliance with these requirements does not necessarily imply a high overall quality level across all aspects, ensuring that EO projects retain a sufficient degree of freedom in selecting their preferred platform. +### Last Mile Applications + +A ‘last mile’ application, as seen from the EO perspective, bridges the final gap between the user at the end of the +value chain, and the web service APIs that offer EO-derived information. This could, for instance, be the integration +of parcel statistics into the field management software of a farmer. In the ESA context, Green Transition Information +Factories can be considered examples of last mile applications. + +Typically, each user group and domain will have its own set of purpose-built tools. The IT integrator or vendor building +these tools will retrieve EO algorithm results via web service APIs based on open standards. These results can then be +further processed as needed, depending on the application. Note that these applications are not necessarily EO-centric +or even geospatial but could be using an EO algorithm as a small part or in the background. + +The APEx Algorithm Services Catalogue acts as a discovery tool for last mile application builders. It shows various +technical parameters, the cost and potential limitations for specific use cases. This speeds up the discovery and selection +process. Application builders will be required to create an account on the hosting platforms and ensure the necessary +funds are available, if they want to test or use the API. + +APEx currently offers two main options for projects to develop their on-demand services: + +* OpenEO-based services [@openeo], implemented in line with openEO’s User Defined Processes (UDP) [@udp] +* OGC API - Processes-based services [@apiprocess], implemented in line with the OGC Application Package Best +Practice [@ap] + +These APEx-compliant technologies allow algorithms to be hosted on an APEx-compliant algorithm hosting platform and make +them available for execution through web services. These technologies promote seamless reuse and integration of existing +EO algorithms. Additionally, by leveraging web services and cloud-based approaches, these technologies simplify the execution +of EO algorithms, shielding users from underlying complexities, such as data access, data processing optimisation, and other +technical challenges. + +APEx remains committed to future innovation and is open to integrating additional specifications, provided they align with +FAIR principles and facilitate algorithm execution through web services. + ## Actors ### EO Project & Algorithm PI @@ -104,9 +134,9 @@ An EO project refers to the consortium that is responsible for building the EO a Algorithm PI (principal investigator). In certain ESA EOP procurement and ITTs, there is now a requirement included to comply with APEx [Interoperability and Compliance Guidelines](../interoperability/index.md). When compliance is required, the consortium can utilise various services offered by APEx. Specifically, the -[APEx Algorithm Support](../propagation/index.qmd) aims tosupport the enhancement of algorithms on a technical and +[APEx Algorithm Support](../propagation/index.qmd) aims to support the enhancement of algorithms on a technical and software level and facilitate the transition to hosted algorithms that can be included in the -[APEx Algorithm Catalogue](https://algorithm-catalogue.apex.esa.int/). +[APEx Algorithm Services Catalogue](https://algorithm-catalogue.apex.esa.int/). ESA EO projects that do not have an explicit compliance requirement are also eligible to receive support. The APEx support can boost project impact, so projects are encouraged to inquire with their ESA technical officer about the From d4bb9fe2e6d2c9ed17d0e71c0e0934d006e7157c Mon Sep 17 00:00:00 2001 From: bramjanssen Date: Mon, 2 Feb 2026 14:17:00 +0100 Subject: [PATCH 2/9] icg: updated provider guidelines based on v4.1 --- interoperability/algohosting.md | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/interoperability/algohosting.md b/interoperability/algohosting.md index 53b783d..209e408 100644 --- a/interoperability/algohosting.md +++ b/interoperability/algohosting.md @@ -49,6 +49,14 @@ complexity. This ensures that the algorithm can be hosted on one of the APEx-compliant algorithm hosting platforms. The APEx documentation will provide clear guidance and samples demonstrating these two options. + More information is available at the following pages in the APEx documentation: + @@ -92,7 +100,7 @@ complexity. -Table: Interoperability requirements for algorithm providers +Table: Interoperability requirements (mandatory) for algorithm providers ::: @@ -181,10 +189,9 @@ Table: Interoperability requirements for algorithm providers -Table: Interoperability recommendations for algorithm providers +Table: Interoperability recommendations (optional) for algorithm providers ::: - ## Best Practices The following sections provide best practice guidelines for developing APEx-compliant algorithms. While these guidelines From 3afe8b218368c014eb1198af3e3da292fb7c3a92 Mon Sep 17 00:00:00 2001 From: bramjanssen Date: Mon, 2 Feb 2026 17:04:44 +0100 Subject: [PATCH 3/9] icg: partial integration of data provider requirements --- guides/file_formats.qmd | 57 +++++++++++++++++++++++++++++---- interoperability/datahosting.md | 41 +++++++++++++++++++++++- 2 files changed, 90 insertions(+), 8 deletions(-) diff --git a/guides/file_formats.qmd b/guides/file_formats.qmd index e3ff525..69ddf2c 100644 --- a/guides/file_formats.qmd +++ b/guides/file_formats.qmd @@ -33,14 +33,19 @@ challenging to use. An exception to this are the 'RGB' style products, where three bands are used to represent a single image. In this case, creating a Cloud Optimised GeoTIFF with three bands is an option. -For associating time information, create one GeoTIFF per timestamp, and one STAC item per timestamp. The GeoTIFF format has -not built-in support for conveying time information, but STAC metadata is supporting this very well. +For associating time information, create one GeoTIFF per timestamp, and one STAC item per timestamp. The GeoTIFF format +has not built-in support for conveying time information, but STAC metadata is supporting this very well. ### Visualisation in APEx Geospatial Explorer -To optimise visualisation in the APEx Geospatial Explorer, additional guidelines have been established. Adhering to these -guidelines will ensure that the data is effectively optimised for visualisation on a map. Please refer to -[this page](../interoperability/geospatial_explorer.qmd#cloud-optimized-geotiff-cog) for more information. +To optimise visualisation in the APEx Geospatial Explorer, it is recommended to use the GoogleMapsCompatible tiling scheme- +typically 256x256 pixel tiles aligned to a global grid. The default Coordinate Reference System (CRS) used in the Geospatial +Explorer is Web Mercator projection (EPSG:3857) and therefore all datasets in this projection will be supported. On the +fly reprojection and / or configuration of a Geospatial Explorer instance to alternative CRS’s is feasible, although we +advise contact the APEx team for specific advice when using alternative projections. The BitsPerSample field must accurately +reflect the data format. Overviews are essential for performance and should be generated using downsampling by factors of +two until the image dimensions are the size of a tile or smaller. These overviews should also be tiled and placed after +the main image data to conform with the COG specification. ## (Geo-)Zarr @@ -61,6 +66,44 @@ At the time of writing, there are, however these important caveats: ## NetCDF -NetCDF is a self-describing format with some properties similar to Zarr, but less optimised for cloud access. It can be useful -for exchanging data cubes as single files through traditional methods. However, it is less recommended for convenient +NetCDF is a self-describing format with some properties similar to Zarr, but less optimised for cloud access. It can be +useful for exchanging data cubes as single files through traditional methods. However, it is less recommended for convenient sharing of large datasets, for which either COG or Zarr provide better options. + +## Statistical Datasets (FlatGeobuf, GeoJSON) + +Statistical datasets can be used to store precomputed statistics for dataset variables based on spatial units, such as +administrative areas. An example is to collect land cover statistics on using boundaries from nomenclature of territorial +units for statistics (NUTS), as shown in the [APEx Geospatial Explorer](https://explorer.apex.esa.int/) (Statistics). The +guidelines in this section are focused on supporting the integration of statistical data for visualisation in the APEx +Geospatial Explorer. + +The statistical datasets are expected to be vector layers that are provided in a format that can be parsed to a feature +collection following the GeoJSON [@geojson] specification. Currently tested and supported formats are GeoJSON [@geojson] +and FlatGeobuf [@flatgeobuf]. FlatGeobuf should be used where the statistical data is a large size as this allows for +streaming of the relevant features without having to download the full dataset, increasing performance. + +The metadata header of the file should contain the following properties to define which fields on the features in the +dataset should be used for the following purposes. + +- identifierKey: The name of the field that stores the unique identifier for each feature. +- nameKey: The name of the field that stores the human-readable name for display. +- levelKey: The name of the field that stores the administrative level number. +- childrenKey: The name of the field that has a comma-separated list of child feature IDs as declared in identifierKey. +Can be the empty string if this is the bottom level. +- attributeKeys: A comma-separated list of field numbers that store the statistical data. +- units: The units as displayed in the UI. This is for UI purposes only and has no effect on the data. +- visualization_hint: A string of histogram, categorised, or continuous used as a hint to the UI to choose a suitable +presentation for the data. + +For example, properties in the file metadata that is defined as follows: + +- identifierKey: NUTS_ID +- nameKey: NUTS_NAME +- levelKey: LEVL_CODE +- childrenKey: children +- attributeKeys: Trees, Shrubland, Grassland +- visualization_hint: categorised + +would use the fields NUTS_ID, NUTS_NAME, … in the data to determine the navigation and display of statistics in the +Geospatial Explorer. For further guidance, please contact the APEx team through the [APEx User Forum](http://forum.apex.esa.int/). diff --git a/interoperability/datahosting.md b/interoperability/datahosting.md index 5e82624..2bdfb6b 100644 --- a/interoperability/datahosting.md +++ b/interoperability/datahosting.md @@ -44,7 +44,7 @@ fostering wider adoption and enabling advanced use cases in downstream applicati DATA-REQ-03 EO project results should be accompanied with metadata in a STAC [@stac] format, including applicable STAC extensions. - The specific STAC profiles will align with the recommendations provided by the ESA Project Results Repository (PRR). More details regarding which profiles to apply will be added as the project progresses. + The specific STAC profiles will align with the recommendations will align with the recommendations provided in the [Metadata Recommendations](#metadata-recommendations) section. @@ -56,3 +56,42 @@ Table: Interoperability requirements for data providers For more details regarding the recommended file formats and their usage within APEx, please refer to the [APEx File Format Recommendations](../guides/file_formats.qmd). ::: + + +## Metadata Recommendations + +### Format Specific Recommendations + +When sharing geospatial datasets in cloud-optimised formats, such as Cloud Optimised GeoTIFF (COG), NetCDF, and Zarr, it +is essential to embed as much relevant metadata as possible directly within the files. Although these formats are designed +for efficient cloud access, their interoperability potential is enhanced when the files carry rich, standardised metadata +aligned with their respective specifications. Doing so not only improves data reuse by third-party tools but also enables +more reliable automatic inference of STAC metadata during cataloguing or dataset publication. + +APEx recommends that the following details be incorporated into the file metadata: + +- The projection system used to present the data within the file +- he Nodata value applied +- The unit of measurement for values represented in the dataset +- A definition of the colour map or legend utilised for the dataset visualisation in case of categorical data. +- Band or variable names and descriptions + +For more details and examples on adding this additional metadata to your results, please consult the specific tools +(e.g. gdal, rasterio, …) for generating the results. + +### STAC Metadata Recommendations + +The STAC specification provides a comprehensive and interoperable framework for describing geospatial datasets. Within +APEx, STAC serves as the foundation to enhance the discoverability, interoperability, and integration of data across a +range of platforms, data catalogues, including the ESA Project Results Repository, and tools such as the APEx Geospatial +Explorer. + +To enhance interoperability, data providers are advised to consistently use a recommended set of STAC-related extensions +and best practices. These recommendations come from community input and collaboration with other initiatives, like +EarthCODE and EOEPCA, to ensure consistency across projects and promote the adoption of best practices. + +@tbl-metadata offers a summary of the suggested metadata. For further details, please refer to the resources listed below. + +- [STAC Best Practices](https://github.com/radiantearth/stac-best-practices/blob/main/README.md) +- [EOEPCA+ Datacube Access Best Practices](https://github.com/EOEPCA/datacube-access/blob/main/best_practices/stac_best_practices.md) +- [ESA PRR Collection Specifications](https://eoresults.esa.int/prr_collection_specifications.html) From 532d8c49ff8f29683a6f3136ed12a8fe1d13c15c Mon Sep 17 00:00:00 2001 From: bramjanssen Date: Wed, 4 Feb 2026 12:05:28 +0100 Subject: [PATCH 4/9] icg: updated data hosting guidelines --- interoperability/datahosting.md | 217 +++++++++++++++++++++++++++++++- 1 file changed, 216 insertions(+), 1 deletion(-) diff --git a/interoperability/datahosting.md b/interoperability/datahosting.md index 2bdfb6b..fb9daa7 100644 --- a/interoperability/datahosting.md +++ b/interoperability/datahosting.md @@ -49,7 +49,7 @@ fostering wider adoption and enabling advanced use cases in downstream applicati -Table: Interoperability requirements for data providers +Interoperability requirements for data providers ::: :::{.callout-tip title="File Format Recommendations"} @@ -95,3 +95,218 @@ EarthCODE and EOEPCA, to ensure consistency across projects and promote the adop - [STAC Best Practices](https://github.com/radiantearth/stac-best-practices/blob/main/README.md) - [EOEPCA+ Datacube Access Best Practices](https://github.com/EOEPCA/datacube-access/blob/main/best_practices/stac_best_practices.md) - [ESA PRR Collection Specifications](https://eoresults.esa.int/prr_collection_specifications.html) + +:::{#tbl-metadata} + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
IDScenarioLevelRequirementDescription
General
METADATA-REC-01Collection / ItemThe STAC collection and items should use STAC 1.1 or higher.
METADATA-REC-02CollectionThe STAC collection must follow the [ESA PRR collection specifications](https://eoresults.esa.int/prr_collection_specifications.html).This guarantees the collection is compatible for upload and registration in the ESA Project Results Repository.
METADATA-REC-03Collection / ItemCollections must be homogeneous: each item has the same assets and uses the same asset keys.Consistent and homogeneous definition of assets simplifies client-side handling and supports datacube generation.
METADATA-REC-04ItemEach item must have at least one asset where the role is set to *data*.This allows for accurate identification of assets containing the data.
METADATA-REC-05Item + Each asset must include: +
    +
  • type
    • +
  • role
    • +
  • title
    • +
  • file:size
    • +
+ Optionally properties: +
    +
  • file:checksum
    • +
+ 		These properties help tools and platforms accurately import the dataset. Furthermore, the *file* properties allow + the ESA PRR to perform extra quality checks.
METADATA-REC-06Data VisualisationItem +

In the case of categorical datasets, + classification extension is recommended to identify + the different classes used in the asset.

+

For additional visualization support, it is recommend setting the *title* and *color_hint* properties to allow + external tools, such as the [APEx Geospatial Explorer](../instantiation/geospatial_explorer.md) to properly visualise + the data.

+ 		The classification extension supports the proper interpretation of categorical data that is included in the + collection, item or asset.
METADATA-REC-07Data VisualisationItem / Asset +

To support the visualization of the dataset, the + render extension is recommended.

+

The render extension allows the definition of the following properties:

+
    +
  • rescale: 2 dimensions array of delimited Min,Max range per band. If not provided, the data will not be rescaled.
    • +
  • Colormap: that must be applied for a raster band
    • +
+ 		The render extension supplies rendering tools, like the [APEx Geospatial Explorer](../instantiation/geospatial_explorer.md), + with key data to auto-configure visualization settings, including rescaling and colour maps.
Datacube Formats (netCDF, ZARR, ...)
METADATA-REC-08Data ProcessingCollection / Item +

The datacube extension (v2.x) should be used to + properly describe the datacube:

+
    +
  • For a single variable: only use *cube:dimensions*
    • +
  • For multiple variables: *cube:variables* and *cube:dimensions*. Each variable should be a separate datacube, + no attempt should be made to combine variables automatically.
    • +
+ 		+

The extension enables correct data parsing into a datacube by the platform or tool.

+

A variable can be bands in EO data or meteorological variables like rain or temperature in meteorological data + sets.

+
Raster Formats (COG)
METADATA-REC-09Data Processing,Data VisualisationItem +

The projection extension must be used to identify + the CRS, raster bounds and shape.

+

At minimum the following must be defined:

+
    +
  • *proj:code*
    • +
  • *proj:bbox*
    • +
  • *proj:shape*
    • +
+ 		+

The projection extension ensures that any tools accessing the data can accurately determine key raster + properties without the performance overhead of inspecting the raster file.

+

If the goal is to visualise your data through the [APEx Geospatial Explorer](../instantiation/geospatial_explorer.md), + please consider the projections that are currently supported, as defined in [EXPLORER-REQ-07](./geospatial_explorer.qmd).

+
METADATA-REC-10Data Processing,Data VisualisationItemTo incorporate a time dimension, the item must define a *datetime*, *start_datetime* and *end_datetime* at the + item level. Both properties contain a single value using ISO8601 time intervals. +

These properties enable tools to accurately identify the data's temporal dimension, simplifying search and + filtering within the STAC collection.

+

For temporal dimensions, it is recommended to maintain the original level of granularity; data should not be + aggregated from daily records to a single label unless specifically instructed by the user or noted in the metadata. + When combining data and overlap exists, the user must indicate the methodology unless indicated in the metadata.

+
METADATA-REC-11Data Processing,Data VisualisationItem +

The *bands* + array must be used to identify band information in the raster, keep the order as identified in the array.

+
    +
  • Use the *name* property, if provided. Alternatively, use *eo:common_name*. As a last resort, use the array + indices.
    • +
  • Ensure homogeneous data types across bands, choosing the most precise one.
    • +
+
METADATA-REC-12ItemFor other dimensions, the datacube extension must be provided.
METADATA-REC-13Data Processing,Data VisualisationItem +

The raster extension must be used to accurately specify + the following attributes associated with the raster file:

+
    +
  • *raster:spatial_resolution* (if multiple resolutions are provided in the item, otherwise specify it in the + Item)
    • +
  • *raster:scale*
    • +
  • *raster:offset*
    • +
  • *raster:nodata value*
    • +
+ 		The raster extension offers valuable information about the dataset, eliminating the need to directly access or + analyse the data itself. For instance, when visualising, details like scale and offset can help convert raw values + into real-world figures.
Statistical Data (FlatGeobuff, GeoJSON)
METADATA-REC-14Data VisualisationItemIt is recommended that STAC records for statistics include a *datetime* property that matches the time stamp of + the source data that the statistics are derived from. The description should also make reference to the boundary + datasets (e.g. NUTS) that the statistics represent.This information offers clear insights into the statistical data and its applications and also assists in integrating + this data into the [APEx Geospatial Explorer](../instantiation/geospatial_explorer.md). +
+ +Metadata Recommendations +::: From 1fe4a091bd067ff6814c68f590c39266a9ea69d9 Mon Sep 17 00:00:00 2001 From: bramjanssen Date: Wed, 4 Feb 2026 12:12:08 +0100 Subject: [PATCH 5/9] icg: updated formatting of data provider table --- css/components/_tables.scss | 13 +++++++++++++ interoperability/datahosting.md | 2 +- 2 files changed, 14 insertions(+), 1 deletion(-) diff --git a/css/components/_tables.scss b/css/components/_tables.scss index 5c7c50d..cdecbe4 100644 --- a/css/components/_tables.scss +++ b/css/components/_tables.scss @@ -39,3 +39,16 @@ table.requirements { width: 40%; } } + +table.requirements-extended { + td, th { + padding: .5rem; + vertical-align: top; + } + td:first-child{ + min-width: 150px; + } + td:nth-child(4), td:nth-child(5){ + width: 35%; + } +} diff --git a/interoperability/datahosting.md b/interoperability/datahosting.md index fb9daa7..cf4cf61 100644 --- a/interoperability/datahosting.md +++ b/interoperability/datahosting.md @@ -97,7 +97,7 @@ EarthCODE and EOEPCA, to ensure consistency across projects and promote the adop - [ESA PRR Collection Specifications](https://eoresults.esa.int/prr_collection_specifications.html) :::{#tbl-metadata} - +
From 49a00ebc4db05a23541e1c8bbd06ff4205945673 Mon Sep 17 00:00:00 2001 From: bramjanssen Date: Wed, 4 Feb 2026 12:15:42 +0100 Subject: [PATCH 6/9] icg: updated algorithm provider guidelines --- interoperability/algohostingenv.md | 11 +++-------- 1 file changed, 3 insertions(+), 8 deletions(-) diff --git a/interoperability/algohostingenv.md b/interoperability/algohostingenv.md index c9f1ab7..dff2443 100644 --- a/interoperability/algohostingenv.md +++ b/interoperability/algohostingenv.md @@ -65,26 +65,21 @@ their compatibility with the APEx standards. - - - - - - + - + - + From f5aabe1c99b93711a601afb90437ce4c9575b9f8 Mon Sep 17 00:00:00 2001 From: bramjanssen Date: Wed, 4 Feb 2026 13:55:07 +0100 Subject: [PATCH 7/9] icg: updated explorer guidelines --- guides/file_formats.qmd | 46 +++++++ .../images/worldcover_bar_chart_example.png | Bin .../images/worldsoils_table_example.png | Bin interoperability/geospatial_explorer.qmd | 130 +++++++----------- 4 files changed, 94 insertions(+), 82 deletions(-) rename {interoperability => guides}/images/worldcover_bar_chart_example.png (100%) rename {interoperability => guides}/images/worldsoils_table_example.png (100%) diff --git a/guides/file_formats.qmd b/guides/file_formats.qmd index 69ddf2c..eff477f 100644 --- a/guides/file_formats.qmd +++ b/guides/file_formats.qmd @@ -107,3 +107,49 @@ For example, properties in the file metadata that is defined as follows: would use the fields NUTS_ID, NUTS_NAME, … in the data to determine the navigation and display of statistics in the Geospatial Explorer. For further guidance, please contact the APEx team through the [APEx User Forum](http://forum.apex.esa.int/). + +Datasets that have classifications (such as land use) should have key:value entires consisting of 'name':'value' and an +entry with a key of 'classifications' with a value consisting of a string based comma separated list containing all the +keys for the classifications and a 'total' key with the sum of all other values. This will allow for correctly rendering +bar charts and pie charts. + +```{json} +{ + Bare / sparse vegetation: 3349.349614217657, + Built-up: 18474.280639104116 + Cropland: 155067.6934300016 + Grassland: 140178.79417018566 + Herbaceous wetland: 1612.828666906516 + Mangroves: 479.46053523623897 + Moss and lichen: 499.40601429089236 + Permanent water bodies: 8969.837211370474 + Shrubland: 7342.96093361589 + Snow and ice: 495.7695064816955 + Tree cover: 301783.0035618253 + Unknown: 1.7258467103820294 + total: 638255.1101299465 + classifications: "Tree cover,Shrubland,Grassland,Cropland,Built-up,Bare / sparse vegetation,Snow and ice, + Permanent water bodies,Herbaceous wetland,Mangroves,Moss and lichen,Unknown" +} +``` + +![worldcover_bar_chart_example](./images/worldcover_bar_chart_example.png){width=75%} + +Datasets that do not have classifications (such as a raster showing soil organic carbon) should contain a selection of +the following entries: + +- mean +- min +- max + +These values will be rendered as a table. + +```{json} +{ + mean: 437.94353402030356 + min: 60 + max: 4410 +} +``` + +![worldsoils_table_example](./images/worldsoils_table_example.png){width=75%} diff --git a/interoperability/images/worldcover_bar_chart_example.png b/guides/images/worldcover_bar_chart_example.png similarity index 100% rename from interoperability/images/worldcover_bar_chart_example.png rename to guides/images/worldcover_bar_chart_example.png diff --git a/interoperability/images/worldsoils_table_example.png b/guides/images/worldsoils_table_example.png similarity index 100% rename from interoperability/images/worldsoils_table_example.png rename to guides/images/worldsoils_table_example.png diff --git a/interoperability/geospatial_explorer.qmd b/interoperability/geospatial_explorer.qmd index 4ab4769..f3a19af 100644 --- a/interoperability/geospatial_explorer.qmd +++ b/interoperability/geospatial_explorer.qmd @@ -34,7 +34,7 @@ and best practices shall be followed to avoid any breaking changes between versi - + @@ -45,6 +45,7 @@ and best practices shall be followed to avoid any breaking changes between versi Raster sources defined within the config shall be provided as either:
  • A well formed URL to a hosted Cloud Optimised GeoTiff (COG) [@cog].
    • +
  • A well formed URL pointing to STAC item containing a publicly hosted Cloud Optimised Geotiff as an asset.
  • WMS [@wms] or WMTS [@wmts] endpoint following the relevant OGC specifications.
  • A template URL following the XYZ(*) format serving PNG or JPEG tiles.
@@ -96,12 +97,13 @@ and best practices shall be followed to avoid any breaking changes between versi @@ -109,10 +111,24 @@ and best practices shall be followed to avoid any breaking changes between versi - + FlatGeobuf [@flatgeobuf]. Details on the structure of these datasets are outlined in the [File Formats](../guides/file_formats.qmd#statistical-datasets-flatgeobuf-geojson) section. + + + + + + + + + + +
ID
HOST-REQ-05The algorithm hosting platform shall provide an SLA that guarantees support beyond the project lifetime.This ensures the long-term sustainability and reliability of the algorithm hosting platform, providing assurance to users that they can rely on continued support even after the project's completion.
HOST-REQ-06 The operator of the algorithm hosting platform shall announce major changes to the SLA, including decommissioning of the platform, to APEx and the NoR, preferably with a lead time of 1 year. Such communication is important to ensure that stakeholders, including APEx and the NoR, are given adequate notice of major changes that could impact the availability or functionality of the algorithm hosting platform. This approach allows for proper planning, adjustment, and mitigation of potential disruptions, ensuring continuity of services for users.
HOST-REQ-07HOST-REQ-06 The algorithm hosting platform shall support standardized methods for machine-to-machine authentication. For instance, the OIDC client credentials workflow allows APEx to securely authenticate.
HOST-REQ-08HOST-REQ-07 The operator of the algorithm hosting platform shall support the APEx consortium in obtaining a single account either freely, or via ESA Network of Resources, that allows to test all published services. For convenient service testing and minimal administrative overhead, a single account and NoR request should give access to multiple services.
HOST-REQ-09HOST-REQ-08 The algorithm hosting platform shall expose process metadata publicly without requiring authentication, unless the nature of the algorithm requires its description to be hidden. APEx tools request service metadata for informative purposes, also from browser-based applications that do not have platform tokens available.
EXPLORER-REQ-01Configuration of the Geospatial Explorer shall adhere to the provided JSON schema.Configuration of the Geospatial Explorer shall adhere to the provided JSON schema, as described in the [Configuration Schema](#configuration-schema) section. The application is configured through a hosted JSON object which is fetched when the client-side application loads. For the application to correctly render, a valid configuration JSON that follows the outlined schema must be provided pre-instantiation. During early development the schema will be subject to change and provided with limited validation.
EXPLORER-REQ-08 - Legends should be configured by providing: +

Legends support is provided in the Geospatial Explorer via the following mechanisms:

    -
  • An inline configuration using either the 'Swatch' or 'Gradient' outlined in the schema.
    • -
  • A URL to a browser supported image resource (PNG, JPEG, SVG).
    • -
  • A WMS [@wms] getLegendGraphic request (If supported by a WMS source).
    • +
  • Dynamically using “swatch” or “gradient” options in the Geospatial Explorer config, which displays a legend using category, gradient or colormap settings;
    • +
  • By calling a WMS [@wms] getLegendGraphic request (if supported by a WMS source)
    • +
  • By specifying in config the URL to a browser supported image resource (PNG, JPEG, SVG).
+

Legend settings can be configured using the configuration builder tool.

 This guideline ensures that the legend is fully compatible with the visualisation library used in the Geospatial Explorer.
EXPLORER-REQ-09 Statistical datasets should be configured by providing public URLs to either: GeoJSON [@geojson] or - FlatGeobuf [@flatgeobuf]The statistics feature is complex due to the architecture and requirements of the Geospatial Explorer. One or - more files containing vector features that describe an area and the relevant statistics for the area can be added - to the configuration of an explorer instance.The statistics feature is complex due to the architecture and requirements of the Geospatial Explorer. Statistical datasets contain one or more files that include vector features that describe an area and the relevant statistics for the area can be added to the configuration of an explorer instance.
EXPLORER-REQ-10 +

To facilitate the integration of STAC datasets within the Geospatial Explorer, it is advisable to follow the recommendations presented in our [metadata recommendations](./datahosting.md#metadata-recommendations). For optimal results within the Geospatial Explorer, the use of the following STAC extensions is recommended:

+
    +
  • [classification extension](https://github.com/stac-extensions/classification)
    • +
  • [render extension](https://github.com/stac-extensions/render)
    • +
+ 		The usage of the suggested extensions will allow the Geospatial Explorer to automatically configure the visualization properties of the selected dataset without the need for additional manual configuration.
EXPLORER-REQ-11“Secondary” constraint layers (e.g. elevation; land cover) can be used to filter the data in a “primary” layer (e.g. solar energy potential). Both the primary and secondary layers need to be Cloud Optimised GeoTIFFs. All primary and secondary layers must use the same grid (i.e. CRS, origin and resolution) as each other.
@@ -124,76 +140,26 @@ While there is no dedicated specification for XYZ, it is widely used (e.g., [Ope ## Format Specification & Guidelines -### Cloud Optimized GeoTiff (COG) - -When generating Cloud Optimised GeoTiffs, it is recommended to use the *GoogleMapsCompatible* tiling scheme—typically -256x256 pixel tiles aligned to a global grid—and to store the image in the Web Mercator projection (EPSG:3857). The -*BitsPerSample* field must accurately reflect the data format. Overviews are essential for performance and should be -generated using downsampling by factors of two until the image dimensions are the size of a tile or smaller. These -overviews should also be tiled and placed after the main image data to conform with the COG specification. An example -command line invocation using GDAL would be: - -```{bash} -gdal_translate -of COG -co TILING_SCHEME=GoogleMapsCompatible -``` - -### Statistics (Vector Layers) - -The statistics feature expects vector layers that are provided in a format that can be parsed to a feature collection -following the [GeoJSON specification](https://datatracker.ietf.org/doc/html/rfc7946). Currently tested and supported -formats are GeoJSON[@geojson] and FlatGeobuf[@flatgeobuf]. -FlatGeobuf[@flatgeobuf] should be used where the statistical data is a large size as this allows for streaming of the -relevant features without having to download the full dataset, increasing performance. - -Statistics should be contained within the properties entry of each feature. Each feature must contain the following properties: - -* `id` - A short unique id string. -* `name` - A description label for the feature to be shown to users. -* `level` - An integer that describes the features geographical hierarchy. This should be contiguous with parent and child -features. -* `children` - A string based comma separated list containing the `id` of all child features. - -Datasets that have classifications (such as land use) should have key:value entires consisting of 'name':'value' and an -entry with a key of 'classifications' with a value consisting of a string based comma separated list containing all the -keys for the classifications and a 'total' key with the sum of all other values. This will allow for correctly rendering -bar charts and pie charts. - -``` -{ - Bare / sparse vegetation: 3349.349614217657, - Built-up: 18474.280639104116 - Cropland: 155067.6934300016 - Grassland: 140178.79417018566 - Herbaceous wetland: 1612.828666906516 - Mangroves: 479.46053523623897 - Moss and lichen: 499.40601429089236 - Permanent water bodies: 8969.837211370474 - Shrubland: 7342.96093361589 - Snow and ice: 495.7695064816955 - Tree cover: 301783.0035618253 - Unknown: 1.7258467103820294 - total: 638255.1101299465 - classifications: "Tree cover,Shrubland,Grassland,Cropland,Built-up,Bare / sparse vegetation,Snow and ice, - Permanent water bodies,Herbaceous wetland,Mangroves,Moss and lichen,Unknown" -} -``` - -![worldcover_bar_chart_example](./images/worldcover_bar_chart_example.png){width=75%} - -Datasets that do not have classifications (such as a raster showing soil organic carbon) should contain a selection of the following entries: - -* mean -* min -* max - -These values will be rendered as a table. - -```{json} -{ - mean: 437.94353402030356 - min: 60 - max: 4410 -} -``` - -![worldsoils_table_example](./images/worldsoils_table_example.png){width=75%} +To support the smooth integration of datasets in the Geospatial Explorer, it is essential that all datasets adhere to the +general [file format recommendations](../guides/file_formats.qmd). Following these guidelines ensures compatibility and +ease of integration within an APEx Geospatial Explorer. + +When referencing data through a STAC catalogue, it is further advised to comply with the STAC [metadata recommendations](./datahosting.md#metadata-recommendations). +Adhering to these metadata standards will improve dataset discoverability, interoperability, and enable seamless integration +with the Geospatial Explorer. + + +## Configuration Schema + +The service configuration will be based on a schema that provides administrators with the expected structure and contents +of the configuration. Taking this approach enables: + +* Automated and dynamic instantiation of the service with differing functionality. +* Configuration validation. +* Definition of a “contract” for easier documentation of features and their configuration. + +For more detailed information on this configuration schema, refer to the APEx documentation portal: +[APEx Configuration Schema Documentation](../guides/geospatial_explorer/tutorials/geospatial_explorer_config_guide.qmd). +Numerous example configurations can be found in the +[APEx Geospatial Explorer Configurations](https://github.com/ESA-APEx/apex_geospatial_explorer_configs) repository on +GitHub. From 732173b0d54d33a7153c111eb9c44a877d53acdb Mon Sep 17 00:00:00 2001 From: bramjanssen Date: Wed, 4 Feb 2026 13:58:54 +0100 Subject: [PATCH 8/9] code_server: fixed broken links --- instantiation/app_code_server.md | 50 +++++++++++++++++--------------- 1 file changed, 26 insertions(+), 24 deletions(-) diff --git a/instantiation/app_code_server.md b/instantiation/app_code_server.md index b723385..f5bf3f5 100644 --- a/instantiation/app_code_server.md +++ b/instantiation/app_code_server.md @@ -4,8 +4,8 @@ title: Code Server IDE ## Overview -The Code Server Interactive Development Environment (IDE) capacity within the APEx Project Environments primarily leverages the power -of the [Code Server software (Visual Studio Code in the Cloud)](#code-server-software-architecture). +The Code Server Interactive Development Environment (IDE) capacity within the APEx Project Environments primarily leverages +the power of the Code Server software (Visual Studio Code in the Cloud). ![Current APEx Code Server IDE](images/code_server.png) @@ -22,16 +22,17 @@ for programming languages and productivity plugins or extensions. ## Software Architecture The APEx Code Server solution is an Integrated Development Environment delivered as a cloud-based user workspace, tailored -to support the activities of Earth observation (EO) projects. +to support the activities of Earth observation (EO) projects. The Code Server IDE within the APEx Instantiation Services is built on a "User Workspace" system architecture, -leveraging Kubernetes and JupyterHub for orchestration and management, and accessing a file system secured and private to the authenticated user. +leveraging Kubernetes and JupyterHub for orchestration and management, and accessing a file system secured and private +to the authenticated user. ![The Code Server IDE in the current APEx workspaces offering ](images/applicationhub_codeserver.png) -Each Code Server user workspace comes equipped with the Visual Studio Code Server, an extension of Microsoft's popular VS Code -editor, as well as with a private data products catalogue. These features empower developer users to edit and build EO -data processing algorithms and workflows, accelerating project outcomes within a dedicated, tool-rich environment. +Each Code Server user workspace comes equipped with the Visual Studio Code Server, an extension of Microsoft's popular +VS Code editor, as well as with a private data products catalogue. These features empower developer users to edit and +build EO data processing algorithms and workflows, accelerating project outcomes within a dedicated, tool-rich environment. The Code Server setup encapsulates all the capabilities of Microsoft's popular VS Code editor and extends them to be run and accessed on a remote server. Beyond the core functionality of its desktop counterpart, the Code Server IDE offers @@ -90,11 +91,12 @@ assistant: It allows to connect any models and any context to build custom autocomplete and chat experiences inside Code Server: -* [Chat](https://continue.dev/docs/chat/how-to-use-it) makes it easy to ask for help from an LLM without needing to -leave the Code Server user interface -* [Autocomplete](https://continue.dev/docs/autocomplete/how-to-use-it) provides inline code suggestions as you type -* [Edit](https://continue.dev/docs/edit/how-to-use-it) is a convenient way to modify code without leaving your current file -* [Actions](https://continue.dev/docs/actions/how-to-use-it) are shortcuts for common use cases. +* [Chat](https://docs.continue.dev/ide-extensions/chat/quick-start) makes it easy to ask for help from an LLM without +needing to leave the Code Server user interface +* [Autocomplete](https://docs.continue.dev/ide-extensions/autocomplete/quick-start) provides inline code suggestions as +you type +* [Edit](https://docs.continue.dev/ide-extensions/edit/quick-start) is a convenient way to modify code without leaving +your current file This extension asks for API keys to use the models. This has been successfully tested and could be an option for the APEx use cases. @@ -125,18 +127,18 @@ libraries like SNAP, GDAL, and Orfeo Toolbox. Developers build container images command-line tools, along with necessary runtime environments, and publish these images on container registries for easy access and deployment. -The Code Server IDE supports the use of the Common Workflow Language (CWL), allowing developers to delineate and disseminate application -workflows in a recognised format. CWL documents comprehensively describe the data processing application, including parameters, -software items, executables, dependencies, and metadata. This standardisation enhances collaboration, clarity, and operational -consistency, ensuring that applications are reproducible and portable across various execution scenarios, including local -computers, cloud resources, high-performance computing (HPC) environments, Kubernetes clusters, and services deployed through -an OGC API - Processes interface. - -Version control and continuous integration are integral components of the Code Server IDE technical architecture. This enables access -to VCS (e.g. GitLab, GitHub) for efficient code repository management, version control, collaboration, and monitoring of -code changes. Automated continuous integration (CI) tools manage the build, test, and deployment tasks in response to code -modifications, ensuring that applications are always in a deployable state. This automation minimises manual testing overhead -and accelerates the rollout of new features or updates. +The Code Server IDE supports the use of the Common Workflow Language (CWL), allowing developers to delineate and disseminate +application workflows in a recognised format. CWL documents comprehensively describe the data processing application, including +parameters, software items, executables, dependencies, and metadata. This standardisation enhances collaboration, clarity, +and operational consistency, ensuring that applications are reproducible and portable across various execution scenarios, +including local computers, cloud resources, high-performance computing (HPC) environments, Kubernetes clusters, and services +deployed through an OGC API - Processes interface. + +Version control and continuous integration are integral components of the Code Server IDE technical architecture. This +enables access to VCS (e.g. GitLab, GitHub) for efficient code repository management, version control, collaboration, and +monitoring of code changes. Automated continuous integration (CI) tools manage the build, test, and deployment tasks in +response to code modifications, ensuring that applications are always in a deployable state. This automation minimises +manual testing overhead and accelerates the rollout of new features or updates. ## Examples From 7c72e64125b599464252e9130ce6b967d2fc5f54 Mon Sep 17 00:00:00 2001 From: bramjanssen Date: Wed, 4 Feb 2026 15:23:09 +0100 Subject: [PATCH 9/9] fix: config urls --- markdown-link-checker-config.json | 3 +++ 1 file changed, 3 insertions(+) diff --git a/markdown-link-checker-config.json b/markdown-link-checker-config.json index ff47e07..88a95c0 100644 --- a/markdown-link-checker-config.json +++ b/markdown-link-checker-config.json @@ -18,6 +18,9 @@ { "pattern": "https://data.esa.int/esado" }, + { + "pattern": "https://openeo.org/documentation/1.0/developers/api/reference.html" + }, { "pattern": "^@" }