diff --git a/APIs/OpenSearch.qmd b/APIs/OpenSearch.qmd deleted file mode 100644 index 18d0cd060e..0000000000 --- a/APIs/OpenSearch.qmd +++ /dev/null @@ -1,377 +0,0 @@ ---- -title: "OpenSearch Catalog web service" -format: - html: - code-fold: true -jupyter: python3 ---- - -```{python} -#| echo: false -#import packages -import requests -import pandas as pd -import xml.etree.ElementTree as ET -``` - -:::{.callout-important} -## Disclaimer: OpenSearch Catalogue API: Decommissioning Notice -We would like to inform you of the scheduled decommissioning of our OpenSearch Catalogue API interface, effective **2 March 2026**. - -As of this date, OpenSearch Catalogue API, available under the following endpoint: https://catalogue.dataspace.copernicus.eu/resto/api, will be officially decommissioned. The interface will no longer be supported or maintained, and access to the OpenSearch Catalogue API will be permanently disabled. - -In preparation for this change, all integrations and dependent systems currently utilizing OpenSearch Catalogue API must be transitioned to our other interfaces: - -- [STAC API endpoint](https://stac.dataspace.copernicus.eu/v1/){target='_blank'} -- [OData API endpoint](https://catalogue.dataspace.copernicus.eu/odata/v1/){target='_blank'} - -Both interfaces are actively developed and fully supported. - -To help with the migration process, we have published a dedicated [Jupyter Notebook](https://github.com/eu-cdse/notebook-samples/blob/main/geo/migration_of_opensearch_to_stac_guide.ipynb) on GitHub that serves as a comprehensive, hands-on guide with practical examples for transitioning from OpenSearch Catalogue API to the [STAC Catalogue API](https://documentation.dataspace.copernicus.eu/APIs/STAC.html). It covers key aspects such as accessing data, filtering and querying collections and items, working with STAC extensions, and other essential topics for adapting existing integrations and dependent systems - -**It is strongly advised to undertake the process of migration without delay to ensure operational continuity and avoid disruption to your current services.** - -Please note that no exceptions or extensions to the decommissioning timeline are planned. - -After **2 March 2026**, any requests to access OpenSearch Catalogue API will be rejected as the endpoint will not be available for use. -::: - -The OpenSearch catalogue allows you to search through Copernicus data using a standardized web service. The [OpenSearch specification](https://github.com/dewitt/opensearch/blob/master/opensearch-1-1-draft-6.md){target="_blank"} can be consulted for technical details of the standard. This web service returns results as GeoJSON feature collections. Each feature in the collection represents an earth observation 'product', referencing where the actual data can be found. - -We remark that this version does not implement the OGC OpenSearch standards, and a migration from other APIs named OpenSearch may require significant modifications. It mainly offers compatibility for existing users of a similar API on the CreoDIAS and Wekeo platforms and with client-side tools and workflows that have implemented support for this API. - -## Using OpenSearch interface to query Data Catalogue - -Since offset is not a recommended form of searching repository pages, we had to implement a limit to a maximum of 200k. The requests over the limit will be rejected with the code 400. -Therefore, we encourage you to limit your inquiries by geographic or temporal area. - -All queries may be executed as simple HTTPS-Get calls by typing the query in the web browser address line, using any HTTPS client, e.g., curl or wget, or from inside the users' program. The database is accessible free and anonymously (open for anonymous access for everyone; no authorization is used). It may be accessed both from the internal network (virtual machines in Creodias) and from outside, e.g. your home computer. Note that the actual EO data are restricted to authorized users; only the Data Catalogue is open. - -### General Rules - -The queries produce results in JSON format. Base URL: - -::: {.panel-tabset} - -# HTTPS Request -[https://catalogue.dataspace.copernicus.eu/resto/api/collections/search.json?](https://catalogue.dataspace.copernicus.eu/resto/api/collections/search.json?){target="_blank"} - - -# Python -```{python} -json = requests.get("https://catalogue.dataspace.copernicus.eu/resto/api/collections/search.json?").json() -pd.DataFrame.from_dict(json['features']).head(3) -``` - -::: - -::: {.callout-note} - -Most queries are case-sensitive. - -::: - -### Collections - -The data are organized in so-called collections corresponding to various satellites. A query may search for data in all collections or one particular collection only. If only one satellite is in the field of interest, the second approach is faster and more efficient than filtering the general query. For example, to find the ten most recent Sentinel-2 products with cloud cover below 10%, the query should look like: - -::: {.panel-tabset} - -# CLI -```{bash} -$ wget -O - "https://catalogue.dataspace.copernicus.eu/resto/api/collections/Sentinel2/search.json?cloudCover=[0,10]&startDate=2022-06-11T00:00:00Z&completionDate=2022-06-22T23:59:59Z&maxRecords=10" -``` -::: - -while if the collection field is missing in the URL, the products from all the satellites are returned: - -::: {.panel-tabset} - -# CLI -```{bash} -$ wget -O - "https://catalogue.dataspace.copernicus.eu/resto/api/collections/search.json?cloudCover=[0,10]&startDate=2022-06-11T00:00:00Z&completionDate=2022-06-22T23:59:59Z&maxRecords=10" -``` -::: - -As for today, the following collections are defined and may be used: - -* Copernicus Sentinel Mission - * **Sentinel1 or SENTINEL-1** - * **Sentinel2 or SENTINEL-2** - * **Sentinel3 or SENTINEL-3** - * **Sentinel5P or SENTINEL-5P** - * **Sentinel6 or SENTINEL-6** - * **Sentinel1RTC or SENTINEL-1-RTC** (Sentinel-1 Radiometric Terrain Corrected) - -* Complementary data - * **GLOBAL-MOSAICS** (Sentinel-1 and Sentinel-2 Global Mosaics) - * **SMOS** (Soil Moisture and Ocean Salinity) - * **ENVISAT or Envisat** (ENVISAT- Medium Resolution Imaging Spectrometer - MERIS) - * **Landsat5 or LANDSAT-5** - * **Landsat7 or LANDSAT-7** - * **Landsat8 or LANDSAT-8** - * **Landsat9 or LANDSAT-9** - * **COP-DEM** (Copernicus DEM) - * **TERRAAQUA** (Terra MODIS and Aqua MODIS) - * **S2GLC** (S2GLC 2017) - -* Copernicus Services - * **CLMS** (Copernicus Land Monitoring Service) - -::: {.callout-note} - -Note that collection names vary slightly from satellite names, as they are used in the EO Data repository. For example, the collection is named **Sentinel2,** while in the repository, its data are located within **/eodata/Sentinel-2/....** branch of the repository tree. - -::: - -### Output sorting and limiting - -By default, a maximum of 20 products are returned. You may change the limit (beware of long execution time for queries about thousands of products) using the phrase: - -::: {style="margin-left:20px"} - -**maxRecords=nnn** - -::: - -If the query is very general and the number of matching products is large, the following pages of products can be retrieved using: - -::: {style="margin-left:20px"} - -**page=nnn** - -::: - -It is also possible to alter the sequence in which the products are displayed by using a phrase similar to: - -::: {style="margin-left:20px"} - -**sortParam=startDate** - -::: - -This will sort the output by observation date. The following orderings can be implemented: - -::: {style="margin-left:20px"} - -**startDate** - the date when the observation was made (start) - -**completionDate** - the date when the observation was made (end) - -**published** - the date when the product got published in our repository - -**updated** - the date when the product got updated in our repository - -::: - -::: {.callout-tip} -When searching for products and adding a wide range of dates to the query, e.g. from 2017 to 2023, we recommend splitting the query into individual years, e.g. from January 1, 2023 to December 31, 2023. -::: - -Each of these orders can be accompanied by: - -::: {style="margin-left:20px"} - -**sortOrder=ascending or sortOrder=descending** - -::: - -For example, the query - -::: {.panel-tabset} - -# HTTPS Request -[https://catalogue.dataspace.copernicus.eu/resto/api/collections/Sentinel2/search.json?startDate=2021-07-01T00:00:00Z&completionDate=2021-07-31T23:59:59Z&sortParam=startDate&maxRecords=20](https://catalogue.dataspace.copernicus.eu/resto/api/collections/Sentinel2/search.json?startDate=2021-07-01T00:00:00Z&completionDate=2021-07-31T23:59:59Z&sortParam=startDate&maxRecords=20){target="_blank"} - - -# Python -```{python} -json = requests.get("https://catalogue.dataspace.copernicus.eu/resto/api/collections/Sentinel2/search.json?startDate=2021-07-01T00:00:00Z&completionDate=2021-07-31T23:59:59Z&sortParam=startDate&maxRecords=20").json() -pd.DataFrame.from_dict(json['features']).head(3) -``` - -::: - -The above request will return 20 products from July 2021, whereas the next query will return the next 20: - -::: {.panel-tabset} - -# HTTPS Request -[https://catalogue.dataspace.copernicus.eu/resto/api/collections/Sentinel2/search.json?startDate=2021-07-01T00:00:00Z&completionDate=2021-07-31T23:59:59Z&sortParam=startDate&maxRecords=20&page=2](https://catalogue.dataspace.copernicus.eu/resto/api/collections/Sentinel2/search.json?startDate=2021-07-01T00:00:00Z&completionDate=2021-07-31T23:59:59Z&sortParam=startDate&maxRecords=20&page=2){target="_blank"} - -# Python -```{python} -json = requests.get("https://catalogue.dataspace.copernicus.eu/resto/api/collections/Sentinel2/search.json?startDate=2021-07-01T00:00:00Z&completionDate=2021-07-31T23:59:59Z&sortParam=startDate&maxRecords=20&page=2").json() -pd.DataFrame.from_dict(json['features']).head(3) -``` - -::: - -### Formal queries - -The formal query is invoked as a sub-phrase sequence, separated by &. The result is a conjunction of all sub-phrases. It is impossible to use an alternative in the question. The query must be specified as a formal query. - -An example of a formal query - about cloudless (cloud cover lower or equal to 10%) products for a specific location: - -::: {.panel-tabset} - -# HTTPS Request -[https://catalogue.dataspace.copernicus.eu/resto/api/collections/Sentinel2/search.json?cloudCover=[0,10]&startDate=2021-06-21T00:00:00Z&completionDate=2021-09-22T23:59:59Z&lon=21.01&lat=52.22](https://catalogue.dataspace.copernicus.eu/resto/api/collections/Sentinel2/search.json?cloudCover=%5B0,10%5D&startDate=2021-06-21T00:00:00Z&completionDate=2021-09-22T23:59:59Z&lon=21.01&lat=52.22){target="_blank"} - - -# Python -```{python} -json = requests.get("https://catalogue.dataspace.copernicus.eu/resto/api/collections/Sentinel2/search.json?cloudCover=[0,10]&startDate=2021-06-21T00:00:00Z&completionDate=2021-09-22T23:59:59Z&lon=21.01&lat=52.22").json() -pd.DataFrame.from_dict(json['features']).head(3) -``` - -::: - -The queries are in the form **param=value or param=[minvalue,maxvalue]**. Most of the parameters are common for all collections. Still, some are specific for some of them (e.g. **cloudCover** applies to optical satellites, but polarisation applies to radar ones), or just a single one. - -An example of a formal query - about CLMS (Copernicus Land Monitoring Service) products for a specific product type: - -::: {.panel-tabset} - -# HTTPS Request -[https://catalogue.dataspace.copernicus.eu/resto/api/collections/CLMS/search.json?productType=soil_water_index](https://catalogue.dataspace.copernicus.eu/resto/api/collections/CLMS/search.json?productType=soil_water_index){target="_blank"} - - -# Python -```{python} -json = requests.get("https://catalogue.dataspace.copernicus.eu/resto/api/collections/CLMS/search.json?productType=soil_water_index").json() -pd.DataFrame.from_dict(json['features']).head(3) -``` - -::: - - -### Geography and time-frame - -The common set of parameters are: - -::: {style="margin-left:20px"} - -**startDate, completionDate** - the date limits of the observation. The time may also be specified, e.g. 2021-10-01T21:37:00Z - -**publishedAfter, publishedBefore** - the date limits when the product was published in our repository - -**lon, lat** - geographical position, expressed in military style (EPSG:4326, as a decimal fraction of degrees, positive for eastern latitude and northern longitude) - -**radius** - a region of interest, defined as a circle with centre in point determined by the longitude and latitude with radius expressed in meters (it won't work with point manually selected in EOFinder/Data Explorer) - -**geometry** - region of interest, defined as WKT string (POINT, POLYGON, etc.) - -**box** - a region of interest, defined as the rectangle with given (west, south, east, north) values. It should be defined this way: &box=west,south,east,north - -::: - -For example, the query can be: - -::: {.panel-tabset} - -# HTTPS Request -[https://catalogue.dataspace.copernicus.eu/resto/api/collections/Sentinel2/search.json?productType=S2MSI1C&cloudCover=[0,10]&startDate=2022-06-11T00:00:00Z&completionDate=2022-06-22T23:59:59Z&maxRecords=10&box=4,51,4.5,52](https://catalogue.dataspace.copernicus.eu/resto/api/collections/Sentinel2/search.json?productType=S2MSI1C&cloudCover=[0,10]&startDate=2022-06-11T00:00:00Z&completionDate=2022-06-22T23:59:59Z&maxRecords=10&box=4,51,4.5,52){target="_blank"} - - -# Python -```{python} -json = requests.get("https://catalogue.dataspace.copernicus.eu/resto/api/collections/Sentinel2/search.json?productType=S2MSI1C&cloudCover=[0,10]&startDate=2022-06-11T00:00:00Z&completionDate=2022-06-22T23:59:59Z&maxRecords=10&box=4,51,4.5,52").json() -pd.DataFrame.from_dict(json['features']).head(3) -``` - -::: - -or - -::: {.panel-tabset} - -# HTTPS Request - -[https://catalogue.dataspace.copernicus.eu/resto/api/collections/Sentinel2/search.json?cloudCover=[0,10]&startDate=2022-06-11T00:00:00Z&completionDate=2022-06-22T23:59:59Z&maxRecords=10&box=-21,23,-24,15](https://catalogue.dataspace.copernicus.eu/resto/api/collections/Sentinel2/search.json?cloudCover=[0,10]&startDate=2022-06-11T00:00:00Z&completionDate=2022-06-22T23:59:59Z&maxRecords=10&box=-21,23,-24,15){target="_blank"} - - -# Python -```{python} -json = requests.get("https://catalogue.dataspace.copernicus.eu/resto/api/collections/Sentinel2/search.json?cloudCover=[0,10]&startDate=2022-06-11T00:00:00Z&completionDate=2022-06-22T23:59:59Z&maxRecords=10&box=-21,23,-24,15").json() -pd.DataFrame.from_dict(json['features']).head(3) -``` - - -::: - -### Volatile features - -Some terrain-like feature masks are not permanent but describe a single scene only. The most commonly used feature is cloudiness, or cloudCover, which is defined for most of the products coming from optical sensors. For example: - -::: {style="margin-left:20px"} - -**cloudCover=[0,10]** - -::: - -This parameter selects only those scenes, which are covered by clouds by no more than 10%. - -::: {.callout-caution} - -## Caution - -To be meaningful, the cloudiness must be provided with each product, while in many products is missing. If the cloudiness is unknown for the scene, it is marked by a value of 0 or -1. cloudCover=0 is therefore ambiguous: it may either mean a totally cloudless sky or a cloudy scene for which cloud cover had not been estimated during original data processing. - -::: - -### Satellite features - -::: {style="margin-left:20px"} - -**instrument** - meaningful only for satellites equipped with multiple instruments. The possible values are satellite specific. - -**productType** - the actual types possible are specific for every satellite. - -**sensorMode** - also satellite and sensor specific. E.g. (for Sentinel-1): sensorMode=EW - -**orbitDirection** - ASCENDING or DESCENDING. For most heliosynchronous satellites descending orbits means the day scenes, while ascending means night ones. For many optical satellites (e.g. Sentinel-2) only day scenes are published. - -**resolution** - expected spatial resolution of the product defined in meters. - -**status:** - -::: - -::: {style="margin-left:40px"} - -- **ONLINE** -- **OFFLINE** -- **ALL** - -::: - -Some additional parameters are strictly satellite-specific, e.g. polarisation, which is defined only for Sentinel-1. - -For every satellite (collection) its set of query-able parameters may be obtained by a query such as: - -::: {.panel-tabset} - -# HTTPS Request - -[https://catalogue.dataspace.copernicus.eu/resto/api/collections/Sentinel1/describe.xml](https://catalogue.dataspace.copernicus.eu/resto/api/collections/Sentinel1/describe.xml){target="_blank"} - - -# Python - -```{python} -url = 'https://catalogue.dataspace.copernicus.eu/resto/api/collections/Sentinel1/describe.xml' -response = requests.get(url) - -root = ET.fromstring(response.content) - -for child in root: - if child.tag.endswith('ShortName') or child.tag.endswith('Description'): - print(f"{child.tag}: {child.text}") -``` - -::: - - -The resulting XML file provides a complete list of the collection parameters, with their brief descriptions. - diff --git a/APIs/Token.qmd b/APIs/Token.qmd index 78788b03bb..b81e2467a3 100644 --- a/APIs/Token.qmd +++ b/APIs/Token.qmd @@ -2,7 +2,7 @@ title: Copernicus Data Space Ecosystem Token Generation --- -Users must have an access token to download products from the CDSE catalogue using OData and OpenSearch API. This token can be generated in Linux and Windows OS using cURL or Python script. +Users must have an access token to download products from the CDSE catalogue using OData API. This token can be generated in Linux and Windows OS using cURL or Python script. ::: {.callout-tip} Username and password are for one-time use in the command. Please do not hardcode the username and password in the application code. Otherwise you risk a potential account takeover. diff --git a/Data/ComplementaryData/SMOS.qmd b/Data/ComplementaryData/SMOS.qmd index e991b37e8a..c76e5ca296 100644 --- a/Data/ComplementaryData/SMOS.qmd +++ b/Data/ComplementaryData/SMOS.qmd @@ -30,10 +30,6 @@ Access to SMOS data is possible via API ::: {.panel-tabset} -# OpenSearch -[https://catalogue.dataspace.copernicus.eu/resto/api/collections/SMOS/search.json?](https://catalogue.dataspace.copernicus.eu/resto/api/collections/SMOS/search.json?){target="_blank"} - - # OData [`https://catalogue.dataspace.copernicus.eu/odata/v1/Products?$filter=Collection/Name%20eq%20%27SMOS%27`](){target="_blank"} @@ -43,7 +39,6 @@ In order to get access to data at specific processing level as well as specific If it is required to customize query in respect to spatial and time coverage, satellite features etc. please, follow instructions on: -• [OpenSearch](https://documentation.dataspace.copernicus.eu/APIs/OpenSearch.html){target="_blank"}\ • [OData](https://documentation.dataspace.copernicus.eu/APIs/OData.html){target="_blank"} {{< include ../_render_collections.qmd >}} diff --git a/Data/SentinelMissions/Sentinel2.qmd b/Data/SentinelMissions/Sentinel2.qmd index 8771ad0b19..682bfc7d34 100644 --- a/Data/SentinelMissions/Sentinel2.qmd +++ b/Data/SentinelMissions/Sentinel2.qmd @@ -52,15 +52,11 @@ For further information on Sentinel-2 Level 2A WorldCover Annual Cloudless Mosai ## Sentinel-2 L2A 120m Mosaic The Sentinel-2 L2A 120m mosaic is a derived product, which contains best pixel values for 10-daily periods, modeled by removing the cloudy pixels and then performing interpolation among remaining values. They are produced for the years 2019 and 2020. -See [here](https://catalogue.dataspace.copernicus.eu/resto/api/collections/Sentinel2/search.json?productType=Mosaics){target="_blank"} for an example query in OpenSearch of the Sentinel-2 L2A 120m Mosaic. - ## Sentinel-2 Global Land Cover (S2GLC) The Sentinel-2 Global Land Cover (S2GLC) provides high resolution Poland (2019-2021) and Europe (2017) land cover map based on Sentinel-2 imagery. For further information on Sentinel-2 Global Land Cover (S2GLC) please also visit the [ESA WorldCover page](https://s2glc.cbk.waw.pl/){target="_blank"}. -See [here](https://catalogue.dataspace.copernicus.eu/resto/api/collections/S2GLC/search.json?){target="_blank"} for an example query in OpenSearch of the Sentinel-2 L2A 120m Mosaic. -