- What is CREODIAS?
- Computing & Cloud
- Data & Processing
- Pricing Plans
- Fight with COVID-19
- Examples of usage
- Processing Sentinel-5P data using HARP and Python
- EO Data Access (R)evolution
- Land cover classification using remote sensing and AI/ML technology
- AI-based satellite image enhancer and mosaicking tools
- Monitoring air pollution using Sentinel-5P data
- Species classification of forests
- Enabling AI / ML workflows with CREODIAS vGPUs
- Satellite remote sensing analyses of the forest
- Satellite-based Urban Heat Island Mapping on CREODIAS
- Old but gold - historical EO data immediately available and widely used on CREODIAS
- CREODIAS for emergency fire management
- AgroTech project as an example of how CREODIAS can be used for food and environmental research
- Monitoring Air Quality of Germany in Pre vs During COVID Lockdown Period
- EO4UA
- Common Agricultural Policy monitoring with Earth Observation
- Applications of CREODIAS data
- Meteorological data usage on the CREODIAS platform
- Building added value under Horizon Europe with CREODIAS
- CREODIAS: Introduction to SAR Sentinel-1 data
- Land subsidence and landslides monitoring based on satellite data
- Satellite imagery in support of the Common Agriculture Policy (CAP) and crop statistics
- Useful tools for data processing, available on CREODIAS platform
- CREODIAS for hydrological drought modelling
- CREODIAS for managing Urban Heat Islands
- CREODIAS for Digitising Green Spaces
- CREODIAS for Air Quality
- Advanced data processors on CREODIAS
- CREODIAS for your application
- Solutions for agriculture with CREODIAS
- Earth Observation data for Emergency response
- Security Applications with Satellite Data
- Climate Monitoring with Satellite Data
- Water Analysis on CREODIAS
- CREODIAS for land and agriculture monitoring
- Solutions for atmospheric analysis
- Example of tool usage
- Processing EO Data and Serving www services
- Processing and Storing EO
- Embedding OGC WMS Services into Your website
- GPU Use Case
- Using the EO Browser
- EO Data Finder API Manual
- Use of SNAP and QGIS on a CREODIAS Virtual Machine
- Use of WMS Configurator
- DNS as a Service - user documentation
- Use of Sinergise Sentinel Hub on the CREODIAS EO Data Hub
- Load Balancer as a Service
- Jupyter Hub
- Use of CREODIAS Finder for ordering data
- ESRI ArcGIS on CREODIAS
- Use of CEMS data through CREODIAS
- Searching, processing and analysis of Sentinel-5P data on CREODIAS
- ASAR data available on CREODIAS
- Satellite remote sensing analyses of the forest
- EO Data Catalogue API Manual
- Public Reporting Dashboards
- Sentinel Hub Documentation
- Legal Matters
- FAQ
- News
- Partner Services
- About Us
- Forum
- Knowledgebase
Your Processing Environment
Searching, processing and analysis of Sentinel-5P data on CREODIAS
Introduction
On the CREODIAS the full archive of the Sentinel-5P mission is hosted. The Sentinel-5P satellite, with onboard the TROPOMI sensor, is designed as a precursor satellite, to fill in the data gap and provide data continuity between the retirement of the Envisat satellite and NASA's Aura mission and the launch of Sentinel-5. The mission performs atmospheric monitoring and was launched in October 2017. By accessing the Sentinel-5P data repository through a CREODIAS Virtual Machine, users can process data without any need for downloading data. In this case study, Sentinel-5P absorbing aerosol index (AAI) data is collected for India in October and November 2019 with the CREODIAS Finder API. Subsequently, the data is pre-processed with HARP tools and results are visualised, all on a CREODIAS VM.
The Absorbing Aerosol Index (AAI) indicates the presence of elevated absorbing aerosols in the Earth's atmosphere. The aerosol types that are mostly seen in the AAI are desert dust and biomass burning aerosols. In the extreme air pollution suffered in India in October and November 2019, AAI values of >3 have been recorded.
Sentinel-5P data access
For this use case, a Windows VM on CREODIAS has been used. On the VM, all the Sentinel-5P data is mounted as a network drive that can be accessed through File Explorer and directly loaded in software like ESA SNAP.
The data catalogue can be queried through the CREODIAS Finder or Finder API. Through the API, a query can be submitted through Python or any other scripting language. The returned information is in either JSON, ATOM format, or just the path names of the files that match the query.
For this use case, a Python script is used to send an API query and parse the results into a data frame. From this data frame the column with path names are extracted to be used for the processing. Using the CREODIAS Finder API is free of charge.
Figure 1: Screendump API query Python script
Processing Sentinel-5P data
For the processing of Sentinel-5P, the HARP tools are used. This extensive Python library is developed by Science [&] Technology Corporation (S[&]T) to process and analyse a wide range of atmospheric EO data sets and is well-equipped to deal with Sentinel-5P data.
The path names to the Sentinel-5P aerosol images that were returned in the Finder API query are ingested in the HARP software, image data that does not meet a quality threshold are filtered out and the remaining data is combined into a daily average. This can be done for any Sentinel-5P dataset, for any area and for any date range. The resulting data is converted to a geotiff for further analysis. Besides the automated Python scripting, there is also a GUI available to use the HARP tools in a more interactive way, named VISAN. With this tool, Sentinel-5P data can be checked and plotted easily.
Figure 2 Example of Sentinel-5P NO2 data visualised in VISAN
Visualisation of daily aerosol data
The daily averages of aerosol optical depth over India for the months October/November 2019 were prepared with HARP. AAI raster images were visualised with mapping libraries in R into daily maps. These were stacked into an animated gif file, to show the development of AAI over two months. It is clear from this image that the peak of AAI occurred around late October 2019.
Figure 3 Map of Absorbing Aerosol Index over India on 31/10/2019
Use potential
This use case shows how a CREODIAS VM allows a user to search, process and visualise Sentinel-5P data without any need to download data. This provides opportunity to use these datasets storage and computationally efficient, without any need for large storage and processing facilities on premise. The scripts used in this use case are simple and can be adapted to any other Sentinel-5P data product, time range, area as wished.
Summary
Full copies of the Sentinel-5P data sets are stored on CREODIAS. Using a VM on CREODIAS allows a user to search, process, analyse and visualise data through simple scripts, through open source Python and R libraries.