ACTION REQUIRED - new Catalogue API for CREODIAS data catalogue
We are thrilled to announce the introduction of a new Catalogue API for the CREODIAS data catalogue! New API is already accessible. Currently, the new Catalogue API enables users to perform an advanced search within the CREODIAS data catalogue in three well-known standards: OData, RESTO Open Search, and STAC.
Old Finder API (RESTO) will be deprecated in March 2023. New products will only be published until the 20th of February 2023.
Users can access interfaces under the following links:
- OData: https://datahub.creodias.eu/OData/v1/Products
- RESTO: https://datahub.creodias.eu/resto/api/collections/Sentinel1/search.json
- STAC: https://datahub.creodias.eu/stac/
Users are encouraged to migrate to the OData interface as the most efficient option. Documentation of the new OData API is available here: https://creodias.eu/eo-data-catalogue-api-manual.
New Catalogue API relies on a completely new database which has been adjusted to European Space Agency's Product Metadata Mapping therefore attributes, especially for Sentinel data, have been reorganized and renamed to match the official ESA documentation. These new attribute names (e.g. productType) and values (e.g. GRD) are present in OData and STAC interfaces. To preserve backward compatibility, both old and new attribute names can be used in the RESTO interface, however only new values can be applied.
To help RESTO users migrate, the linked table contains a comparison of old and new attribute values. The details of new attributes' values are also available under RESTO describe endpoints (e.g. https://datahub.creodias.eu/resto/api/collections/Sentinel1/describe.xml ).
New Catalogue RESTO interface preserves backward compatibility with the old RESTO (Finder API) in terms of:
- Query structure
- Names of search options
- Names of products' attributes
- Response structure in JSON
There is no backward compatibility in terms of attributes' values (e.g. Level 2A, L2, etc).
The following functionalities have been deprecated and will not be available:
- Natural Language Queries
- Query response in XML
- Pretty_query
- search.path
Also, we would like to inform you about additional changes in the CREODIAS data repository, that will be introduced due to the migration to new data sources and further adjustments of metadata.
On the 20th of February, the following changes will be introduced:
- Product paths for some new Sentinel products will be changed according to the table linked below. Historical data will not be migrated, it will stay under the current paths, therefore it is recommended to use the Data Catalogue to search for products and retrieve product paths from the response generated by the API.
Example of a changed path in the repository for Sentinel-1 GRD IW products:
PRODUCT LOCATION FOR ALL DATA INGESTED BEFORE FEBRUARY 20th: /eodata/Sentinel-1/SAR/GRD/2023/02/19/S1A_IW_GRDH_1SDV_20230119Txxxxxx_20230119Txxxxxx_yyyyyy_zzzzzz_wwww.SAFE
PRODUCT LOCATION FOR ALL DATA INGESTED AFTER FEBRUARY 20th: /eodata/Sentinel-1/SAR/IW_GRDH_1S/2023/02/20/S1A_IW_GRDH_1SDV_20230220Txxxxxx_20230220Txxxxxx_yyyyyy_zzzzzz_wwww.SAFE - As already announced the product types (value of productType field in the metadata) in the new Data Catalogue will change for multiple collections. The most significant change concerns Sentinel-1 where the legacy RAW, GRD, SLC, and OCN types will be changed to more detailed as described in the linked table.
To allow the users to smoothly migrate from the legacy product types to more detailed product types both new and old values for Sentinels 1,2,3 will be usable in the catalogue search until the end of the migration process to the new Copernicus Data Access Service in June 2023 (https://scihub.copernicus.eu/news/News01130).
After the migration, only the new product types will be available.
This temporary feature will only be applicable for inclusive search (using =/eq in the OData filter). Mapping will not be present for exclusive search (not/not eq).
Example of changes required for the RESTO API query for Sentinel-2 L1C:
LEGACY RESTO API: https://finder.creodias.eu/resto/api/collections/Sentinel2/search.json?maxRecords=10&startDate=2023-01-19T00%3A00%3A00Z&sortParam=startDate&sortOrder=descending&productType=L1C&status=all&dataset=ESA-DATASET
NEW RESTO API: https://datahub.creodias.eu/resto/api/collections/Sentinel2/search.json?maxRecords=10&startDate=2023-01-19T00%3A00%3A00Z&sortParam=startDate&sortOrder=descending&productType=S2MSI1C&status=all&dataset=ESA-DATASET
The best performance and flexibility can be achieved when using OData query:
Sentinel-2 L1C:
https://datahub.creodias.eu/odata/v1/Products?$filter=Collection/Name%20eq%20%27SENTINEL-2%27%20and%20Attributes/OData.CSC.StringAttribute/any(att:att/Name%20eq%20%27productType%27%20and%20att/OData.CSC.StringAttribute/Value%20eq%20%27S2MSI1C%27)%20and%20ContentDate/Start%20gt%202022-05-03T00:00:00.000Z%20and%20ContentDate/Start%20lt%202022-05-21T00:00:00.000Z
Sentinel-1 SLC:
https://datahub.creodias.eu/odata/v1/Products?$filter=Collection/Name%20eq%20%27SENTINEL-1%27%20and%20contains(Name,%27SLC%27)%20and%20ContentDate/Start%20gt%202022-05-03T00:00:00.000Z%20and%20ContentDate/Start%20lt%202022-05-21T00:00:00.000Z
Once again, we strongly recommend adopting OData as a replacement for the legacy RESTO API to unlock the full potential of the new CREODIAS data catalogue.
Documentation of the new OData API is available here: https://creodias.eu/eo-data-catalogue-api-manual.
Please be advised that if you have any questions about the new Data Catalog API or experience any problems or errors while implementing the changes or testing applications using the new catalogue after the February 20 changes, please contact our customer support department at support@cloudferro.com.