- What is CREODIAS?
- Computing & Cloud
- Data & Processing
- Pricing Plans
- Fight with COVID-19
- Examples of usage
- Utility of CREODIAS data
- Advanced data processors on CREODIAS
- CREODIAS for your application
- Solutions for agriculture with CREODIAS
- Earth Observation data for Emergency response
- Security Applications with Satellite Data
- Climate Monitoring with Satellite Data
- Water Analysis on CREODIAS
- CREODIAS for land and agriculture monitoring
- Solutions for atmospheric analysis
- Example of tool usage
- Processing EO Data and Serving www services
- Processing and Storing EO
- Embedding OGC WMS Services into Your website
- GPU Use Case
- Using the EO Browser
- EO Data Finder API Manual
- Use of SNAP and QGIS on a CREODIAS Virtual Machine
- Use of WMS Configurator
- DNS as a Service - user documentation
- Use of Sinergise Sentinel Hub on the CREODIAS EO Data Hub
- Load Balancer as a Service
- Jupyter Hub
- Use of CREODIAS Finder for ordering data
- ESRI ArcGIS on CREODIAS
- Use of CEMS data through CREODIAS
- Searching, processing and analysis of Sentinel-5P data on CREODIAS
- ASAR data available on CREODIAS
- Public Reporting Dashboards
- Sentinel Hub Documentation
- Integration Guides
- OGC API
- Custom Processing Scripts
- Legal Matters
- Partner Services
- About Us
Computing & Cloud
Large-scale data storage and access technology to power European and global research and industry
Large research projects and such important European initiatives like Digital Twin Earth, require collecting, storing, processing and analyzing huge amounts of data. Good news is that big data storage and access technologies are mature and available. CREODIAS is ready to serve over 2PB daily.
The Fourth Paradigm is a concept that is focused on advancing science by open or increased access to data. In the deluge of new data, it became not only possible, but also necessary to supplement classical scientific paradigms: observation, theory and simulation with a fourth one: large-scale Data Exploration. Something that will unify observation, theory and simulation in an extensive system.
A number of large research projects and initiatives emerging in recent years highlights that necessity and displays the importance of making the data, tools, technologies and platforms accessible. Initiatives like Digital Twin Earth, Human Brain Project, Digital Twin Manufacturing and many others are frameworks for understanding, modelling and forecasting the behavior of extremely complex systems. For such frameworks to work it is indispensable to store and manage huge amounts of heterogenous data and to make it available through unified, flexible, streamlined interfaces to multiple user communities.
Storing and dissemination of peta - or exabytes of heterogenous data in an open and flexible manner poses a serious technical challenge. Object storage provides a solution that is cost-effective, easily scalable and accessible. It allows for storing unstructured or extremely diversely structured data, thanks to the lack of hierarchical directory structure. Instead, object storage uses a unique identifier for each object. This convention and “flat” architecture allows also for massive, dynamic scaling of the storage where the scale is nearly infinite. The matter is to employ it in a real life, commercially viable scenario. Something that requires advanced cataloguing and network services. Altogether guaranteeing automated, fast and easy access to data stored online for immediate use.
CREODIAS currently stores almost 21 PB of Earth Observation data, ingests on average 25 TB of data daily and disseminates it to more than 6 thousand registered users and countless non-registered ones. Using opensource CEPH software for building storage - because of its ability to build and manage object storage for OpenStack – and advanced cataloguing solution, CREODIAS can serve data inside and outside of its cloud via graphical application (CREODIAS Finder) and variety of access interfaces. This machine2machine interfacing enables stakeholders to leverage the data in their processing chains in an automated fashion, both on CREODIAS cloud and on any other infrastructure of their choice.
CloudFerro, which is the CREODIAS operator, has recently conducted tests that show ability to serve 2PB of data daily from its repositories. It is even possible to double that rate. With all the prerequisites: current 21 PB of EO data and possible growth to 50PB if necessary in a near future, benchmarking and tests results and experience from building and operating numerous cloud platforms - Climate Data Store, CODE-DE, WEkEO, EO IPT and others - with combined storage of over 100PB - CloudFerro can operate at a scale required by initiatives like Destination Earth. Building on expertise and lessons learned from previous projects we are able to ingest, store, index and disseminate massive amounts of EO data, tens and hundreds of Petabytes. We can provide easy, remote, broadband and scalable access to online, granular data in a cost-efficient manner. And those are vital capabilities when the fourth paradigm is in force.