- What is CREODIAS?
- Computing & Cloud
- Data & Processing
- Pricing Plans
- Fight with COVID-19
- Examples of usage
- Applications of CREODIAS data
- CREODIAS for managing Urban Heat Islands
- CREODIAS for Digitising Green Spaces
- CREODIAS for Air Quality
- Advanced data processors on CREODIAS
- CREODIAS for your application
- Solutions for agriculture with CREODIAS
- Earth Observation data for Emergency response
- Security Applications with Satellite Data
- Climate Monitoring with Satellite Data
- Water Analysis on CREODIAS
- CREODIAS for land and agriculture monitoring
- Solutions for atmospheric analysis
- Example of tool usage
- Processing EO Data and Serving www services
- Processing and Storing EO
- Embedding OGC WMS Services into Your website
- GPU Use Case
- Using the EO Browser
- EO Data Finder API Manual
- Use of SNAP and QGIS on a CREODIAS Virtual Machine
- Use of WMS Configurator
- DNS as a Service - user documentation
- Use of Sinergise Sentinel Hub on the CREODIAS EO Data Hub
- Load Balancer as a Service
- Jupyter Hub
- Use of CREODIAS Finder for ordering data
- ESRI ArcGIS on CREODIAS
- Use of CEMS data through CREODIAS
- Searching, processing and analysis of Sentinel-5P data on CREODIAS
- ASAR data available on CREODIAS
- Public Reporting Dashboards
- Sentinel Hub Documentation
- Integration Guides
- OGC API
- Custom Processing Scripts
- Legal Matters
- Partner Services
- About Us
Data & Processing
EODATA Access Interfaces
In order to provide a very flexible functionality for Third Party Users, the CREODIAS data can be accessed using a complete set of interfaces. Raw data is stored on the Platform in its native format to ensure 100% conformity to the source data. It can be accessed directly through a standard HTTP SWIFT/S3 object interface. Legacy applications, virtual machines or interactive users can access the same data through a POSIX standard interface using filesystem emulation . Data stored in the repository can be also dynamically published in the form of user-configurable OGC WMS/WMTS/WCS web services provided by the EODATA Processing/Access HUB. Finally, CREODIAS provides a general purpose web interface for end-users – the EO Browser. Data products can also be accessed via HTTP Download while information services can be made accessible via an http Proxy service.
All CREODIAS data access interfaces are open source. The access methods are detailed in the following Sections.
The availability of the EODATA Access Interfaces depends on the CREODIAS Data Location. When data is present in the CREODIAS Locations Local and/or Cached – then Object Data SWIFT/S3, EO Data Hub (OGC related) and HTTP Download can be used. When data is in Location Orderable – HTTP Download can be used but this kind of data can be also Ordered to become Cached and be available as described above. The data or information in Location External can be accessed via Proxy interface.
The available EODATA Access Interfaces are described in general in sections below. Full details of the interfaces will be available in the Support Framework in the DIAS Web Site.
Please note that access to Copernicus data is always open and free for everybody.
Within the CREODIAS cloud users can use the data for free via rich multiple access interfaces like object s3, nfs, zipped download. CREODIAS registered users can benefit from free download of zipped Copernicus products from anywhere in the world. The additional commercial service „CloudFerro Remote Transfer for EODATA” regards object S3 transfer of EODATA to a remote customer anywhere in the world. A user pays for S3 transfer to his location.
Object Data Access API (SWIFT/S3)
The object access API is the main access method for CREODIAS data.
Tenants can access EODATA by sending HTTP requests to a standard SWIFT or S3 object interface. Appropriate client libraries, programming language bindings (Python, Java, C, Ruby, …), CLI tools and their documentation are available in the application repository and the Knowledgebase.
The object interface is suitable for Third Party applications, high performance parallel access and scalability.
Users can access individual products, their parts, such as granules or metadata. They can even access parts of a single data file, which may be useful for applications such as individual tiles extraction from large jpeg2000 image files.
The Object API can be used by Third Party Services, but it is also the generic data access method used by other EODATA access services and the EO Browser. It may also be used by registered Users to download data directly to their workstation.
While HTTP-based object interfaces are largely used in modern cloud applications, interactive human users and many legacy applications (including Sentinel Toolboxes) access data using a traditional POSIX filesystem interface. Usually, such users must first download the necessary files, store and access them from a local filesystem. This process is slow and requires additional temporary storage.
To address this challenge, CREODIAS makes the EO Data repository contents available directly through a POSIX filesystem interface. Users can access this interface from their Virtual Machines hosted on the CREODIAS infrastructure. Filesystem access may be achieved either through an emulation software that allows a Linux VM to ‘mount’ an object filestore as a filesystem or NFS proxy. In both cases, the mounted repository will be seen as a read-only file tree mounted under the user machine’s filesystem. Such mounting will be preconfigured on all VM templates, we will also provide detailed configuration instructions for those customers, who reinstall their systems on their own.
However filesystem interface is not recommended for processing chain applications and direct object access should be used whenever possible. The EO Data repository is an object storage and therefore any filesystem emulation or proxy lowers the performance and throughput of data access.
Users using EODATA access can be assured of minimal bandwidth in data transfer. If in the project Virtual Machines are used, the transfer will be no less than 100MB/s, whereas for the Dedicated Servers option it is 500MB/s.
Please also note that the NFS interface has some limitations including very slow listing of directories which have thousands of objects in their tree (for example ls on daily production of Sentinel-2 may take substantial amount of time or even not complete due to protocol defined timeouts). Therefore it is recommended to go directly to product path which can be discovered using the product search tool:
EO Data Processing/Access HUB
The EO Data Access Hub provides a complete set of mechanisms to access EO Data as dynamically generated OGC standard web services. The Hub is focused on Copernicus satellites but also supporting other sources such as Landsat, PlanetLabs and others. Image data from the EO Data Repository is processed in real time using the CREODIAS cloud infrastructure and presented to the users in the form of customizable WMS/WMTS/WCS/WFS web services. It can be integrated into any mapping or web application, providing an easy-to-use and cost-effective way to exploit the data. It removes the major hassle of downloading, archiving and processing petabytes of data and simply makes the full and global repository easily available immediately via web services. Application developers can focus on added value services and end-user applications rather than having to deal with the complexity of remote sensing data.
The Hub itself is completed by front-ends to search, discover browse and download EO Data and to configure custom Web Service layers (EO Browser, Mosaic Generator, WMS Configurator). The EO Data Access Hub and its frontends are described in more detail in Section Data Related Services below.
Basic EO Data Access Hub services are free of charge - These are:
- WFS access to catalogue data
- WMS with true-color composite, full resolution; meant for use on end-user applications (e.g. QGIS), not for integration in web applications.
- EO Browser
- Mosaic Generator
More advanced functions are available with monthly or longer subscription:
- WMS with various configuration options
- WCS with various configuration options
- WMTS with various configuration options
The detailed price list is available in the Price List section.
Most of data indexed in the CREODIAS catalog (Locations: Local, Cached, Orderable) can be downloaded by Users with the CREODIAS Platform acting as a proxy. This functionality enables registered Users to download data indexed in the CREODIAS catalogue without the need to register or log-in to a remote data store. The CREODIAS acts as a proxy, downloading the desired product from its source and forwarding it to the User’s storage. The functionality can be used by CREODIAS Third Parties to download data into their environment on the CREODIAS Platform. It can also be used by CREODIAS Users to download data to their workstation over the Internet.
Proxy to External Copernicus Services
Some Copernicus Services (Location: External) are accessible as standalone web services that cannot be mirrored or otherwise copied to the CREODIAS Platform. Such Services will be represented in the CREODIAS Collections Catalog with a link to the original service. Depending on the access policy and interfacing capabilities of the original service provider, a http proxy or a single sign-on protocol will be used so that the User doesn’t have to login to the such remote system.
CloudFerro Remote Transfer for EODATA
For users who do not have cloud resources on the CREODIAS platform or have their own infrastructure and need fast and full access to EODATA - CloudFerro has prepared an additional service allowing unlimited external S3 access to CREODIAS EODATA (satellite data archive).
This functionality allows to locate and download individual products, parts such as granules and metadata, and even access parts of a single data file which may be useful for applications such as isolated tiles extraction. For this solution, customers can use multiple 10GE of CloudFerro cloud bandwidth. Any user who wants to connect from an external infrastructure to the CREODIAS collection can do so through the CloudFerro solution, which uses the S3 protocol. The user can simply use CREODIAS credentials to login and generate S3 keys. After generating the S3 key (Access and Secret Key) and using the software supporting S3 protocol, the user has the possibility to connect remotely to CREODIAS satellite data with full S3 support.
If you are a registered user of the CREODIAS platform, use your login credentials using this Transfer for EODATA" provided by CloudFerro, please register first using this link.
A tutorial on how to access the service and how to generate a key can be found in our FAQ.
To learn the prices of the „CloudFerro Remote Transfer for EODATA”, visit our price list or contact our sales department.
Please note that access to Copernicus data is always open and free for everybody. The additional service regards external data transfers to remote customer locations. Within the CREODIAS cloud users can use the data for free.