- What is CREODIAS?
- Computing & Cloud
- Data & Processing
- Pricing Plans
- Fight with COVID-19
- Examples of usage
- Example of tool usage
- Processing EO Data and Serving www services
- Processing and Storing EO
- Embedding OGC WMS Services into Your website
- GPU Use Case
- Using the EO Browser
- EO Data Finder API Manual
- Use of SNAP and QGIS on a CREODIAS Virtual Machine
- Use of WMS Configurator
- DNS as a Service - user documentation
- Use of Sinergise Sentinel Hub on the CREODIAS EO Data Hub
- Load Balancer as a Service
- Jupyter Hub
- Use of CREODIAS Finder for ordering data
- ESRI ArcGIS on CREODIAS
- Use of CEMS data through CREODIAS
- Searching, processing and analysis of Sentinel-5P data on CREODIAS
- ASAR data available on CREODIAS
- Public Reporting Dashboards
- Sentinel Hub Documentation
- Integration Guides
- OGC API
- Custom Processing Scripts
- Legal Matters
- Partner Services
- About Us
Computing & Cloud
The CREODIAS Platform provides Users with an orchestration service to ease and simplify virtual infrastructure deployment and management. User can describe the infrastructure in a template file and deploy/delete it with one action. Templates can contain configuration of Virtual Networks, Virtual Routers, Floating IPs, Security Groups, VMs, Volume Storage and many other Resources together with their parameters to create a fully functional and replicable virtual environment. Templates can also specify relationships between Resources. Orchestration service manages the whole lifecycle of the virtual environment and can apply configuration changes in a smart way (without whole infrastructure redeployment). The OpenStack orchestration service (Heat) is based on the HOT templates (see OpenStack documentation).
The Orchestration service can be used via API or via the Cloud Dashboard. The Orchestration service is compatible with that of Amazon Web Services (CloudFormation templates).
The orchestration service is free of charge.
Figure 1 - Orchestration
Reporting, Monitoring and billing API
IaaS resource reporting and IaaS billing can be accessed interactively via the Cloud Dashboard and Customer Portal as well as via API. In this way a tenant of the platform can get full information about provisioned resources and about related charges.
Interactively via the Cloud Dashbaord and User Portal and machine-to-machine via API.
The resource reporting and billing is free of charge.
The CREODIAS Platform provides specialized internet and data transmission services.
Fixed bandwidth internet connection
A User can request a fixed bandwidth Internet pipe to his Environment. Such dedicated internet connection is not shared with other customers (or Administrative Domains) and can be fully saturated by a given User traffic without any constraints.. From the user point of view this internet connection is seen as a dedicated Virtual Network in the User Environment in the Cloud Dashboard. This Network can be also attributed external public addresses. User’s resources like computing instances (or VMs) and other networks can be connected to this network directly or via Virtual Routers.
In this case internet connection is billed not per transferred data but per available bandwidth in Mbit/s (see a relevant position in the service Price List)
Direct data connections – Data transmission
Similarly - dedicated data transmission lines can be established between customer premises and the CREODIAS Platform points-of-presence (or POPs) in the main neutral Data Centers in Poland and one in Germany. Such data transmission links (in Ethernet technology) can provide dedicated connectivity between customer infrastructure and the CREODIAS Platform – with available bandwidths ranging from 1 Mbit/s to 10 Gbit/s. Again – such connection is seen from the User’s point of view as a special Virtual Network in the Cloud Dashboard.
The CREODIAS Platform POPs are now available at the following neutral DCs:
- T-Mobile Warsaw, 24 Piękna St.; Warsaw;
- ATMAN Poland, 5 Konstruktorska St., Warsaw;
- Equinix LIM Poland, 65/79 Al. Jerozolimskie St.; Warsaw;
- 3s DC, Warsaw.
- DC Frankfurt, Germany
Pricing for such connections is negotiated individually as it depends substantially on the available last mile solutions to the customer premises.
Description and Provisioning
The EO Data Marketplace includes a set of functionalities allowing easy publishing, licensing and sale of geo-data based data products. These functionalities are targeted to geodata publishers who can sell their data products to other CREODIAS Platform Users.
Data Publishers can offer their data by creating Data Repositories, uploading data into them and defining Collections that users can subscribe and access. Data Consumers can downloaded data via HTTP/Swift. Data repositories can also be mounted and accessed directly from a user's VM running within the CREODIAS Platform. From the User's perspective, Publisher-provided data collections behave in a similar way as the standard EO Data collections provided by the Platform.
The functionality of the Marketplace will be fully integrated with the existing components of the CREODIAS Platform. The Customer Portal will be used to register users, browse and purchase access to the Data Collections offered by Publishers. The EO Browser will be used to search for EO Data. Existing authentication mechanisms (Keystone) will be used to control user access. The billing mechanism will be extended to provide billing functionalities for Publishers. The billing mechanism will provide functionality for pay-per-use type of billing – per product, per km2 or per pixel.
The functionality will be implemented as an extension of the existing software components: the Customer Portal (WHMCS), EO Browser, SWIFT HTTP Object Server, the Billing Subsystem.
There will be 3 categories of Users interacting with Marketplace functionality of the EO Marketplace: Data Publishers, Data Customers and Guests
Data Publishers are companies or institutions who want to offer their geo-data on the Marketplace.
Data Customers are users who want process, download or otherwise use the data offered on the Marketplace.
Guests are non-registered users, who can browse, search and preview the available data collections, but are not entitled to download or otherwise access the data details.
Data Publishers and Data Customers register through the Customer Portal using the standard registration process and receive appropriate login credentials. Guests don’t need to register or login
User authorization and authentication is performed by the standard CREODIAS Platform Keystone component.
A Data Publisher uses the Marketplace to make his geo data available to Customers.
The publisher first needs to create a dedicated container in OpenStack object storage (Swift). The container must be tagged with the appropriate storage policy. Once the container is created, data may be uploaded into it using standard Swift interfaces. Data uploaded to the container will be automatically indexed to be present in the product catalog.
Data uploaded to the Repository needs to have a tree structure reflecting the logical structure of the geo data offered by the Publisher. The bottom level of the directory should contain individual geo-data products. Every product can be composed of several files contained in a single directory.
In order to enable automated indexing of Products and their visibility in EO Finder, the Publisher must provide the following information:
- Directory depth that contains individual products/scenes
- Localization of required product metadata (product geometry and acquisition time)
- Localization of optional product metadata (cloud coverage, instruments etc)
- Availability rules for guest-accessible files, localization of preview images
As part of the repository configuration process, the Platform will provide the necessary customer support to enable product indexing.
In order to make the contents of their repositories available to customers, Publishers need to define Data Collections. Data collections will be presented as a separate category of products in the Customer Portal, with the following attributes:
- Short description
- Small Image/Logo
- Spatial resolution
- Update frequency
- Licensing terms
- Long description (HTML)
- Geographical coverage
- Temporal coverage
Data Collections provide ‘filtered views’ of repositories that are made available to Customers on common conditions. Filters may be defined as:
- regular expression masks applied to the product access paths
- geographical area filters in KML format
The filtering mechanism allows Publishers to offer different data subsets based on the same repository. A Publisher could offer full repository access for a certain price, geographically limited access for another price and free access to sample old data.
Due to technical limitations of POSIX file systems, direct file system access from customer VM-s will be possible only for full, unfiltered data collections. The filtered collections will be available for Swift/HTTP access only.
For every Collection, Publishers will be able to define pricing in one of the following models:
- Limited time subscription (per hour, per month, per year)
- Pay once, unlimited time access
- Per GB (over HTTP only)
- Per product (over HTTP only)
- Per km2
- Per pixel
Billing will be performed by the CREODIAS Platform on the Publisher’s behalf, based on pricing defined by the Publisher. Customers accessing unfiltered repositories over POSIX Filesystem will be billed on a subscription basis if the Publisher allows such an option. Customers accessing data over Swift/HTTP will be billed depending on the billing model selected by the Publisher.
The platform will provide detailed billing reports for Publishers and Customers.
Payments clearing between the CREODIAS Platform operator and the Publisher will be done in monthly periods, for subscriptions sold during the previous month, based on billing reports provided by the CREODIAS Platform and invoices issued by Publishers.
The CREODIAS Platform will propose a standard data access licensing agreement model for the Publishers’ use. However, Publishers will be free to provide their own licensing agreement that the Data Customers will accept before gaining access to the data.
Non-registered Users can browse, search and preview data collections in a limited way. All Data Collections and their descriptions are visible in the Marketplace geo data catalog on the Customer Portal.
A guest may also access the EO Browser to search for actual geo data provided by Publishers and preview product content in a similar way as for standard free products offered on the CREODIAS Platform. Collections provided by Publishers will be indexed and visible in EO Browser alongside the free collections (Mandatory, Complementary Collections).
A Data Customer is a CREODIAS Platform User who wants to access Data Collections provided by other users (Data Publishers). In order to access a data collection, a user needs to register with the Platform and subscribe for access to the appropriate Collection.
Data Customers may browse the data collections and search for data in a similar manner as Guests. In addition, they can download zipped data products and they can mount and access full (unfiltered) data collections from within their VM-s if the Publisher allows such access mode.
EO Data Marketplace functions are billed in the revenue sharing model. The Platform provides billing for the Data Publisher. The Data Publisher pays the Platform with the standard share of the received price.
Figure 2 - Data Marketplace
Engineering support and consulting, project implementation and Third Party on-boarding
Apart from IaaS ICT services our platform offers also several elements of the engineering support. We provide the following engineering services:
- General support for the platform Users (Third Parties) aimed at seamless utilization of the platform. These are services aimed at relatively advanced users that are able to on-board in general on their own – but sometimes can have problems or difficulties when using the platform services. In this case they can open support tickets which are then solved by support engineers in 5 days a week working hours regime. These issues are organized in ticketing model, no complex projects are open . These support services are free for paying IaaS customers.
- Extended support and on-boarding of the Users on the platform. In this model we can first provide the consulting and design services that will provide the Third Party with the detailed design of the technical and organizational services needed to realize its needs. After the consulting/design phase there can be the engineering on-boarding phase where our engineers will perform actual works aimed at on-boarding of the Third Party solution on the CREODIAS platform infrastructure. These services are paid extra according to the price listed hourly work price – for consulting/design services and for engineering services. After the on-boarding of the Third Party – it is then charged for the relevant IaaS services.
- Extended design and processing services – in this model a customer defines the processing chain to be realized on the platform and the resulting products to be stored in the platform storage. Our engineers (together with specialists from our knowledge partners) will then design: the whole processing and analysis chain, related hardware (IaaS)/software setup and data access process. After accepting by the customer – the service is fully implemented and then run by the platform.
Standard support services are provided by support engineers in the answer to opened support tickets – via email and platform web page. The progress of support solution is managed via standard ticketing system. All related communication is later available for the User and for the platform operator via platform web page.
Extended support services/on-boarding are started via platform web page. After identifying an extended support service/on-boarding request a dedicated project manager is assigned. The scope of work (design, support, implementation) is specified. The initial project plan and pricing is also specified. After the initial project acceptance the relevant work is performed. The stages of the project are then accepted by the customer.
Extended design and processing services are processed in the similar way to the extended support services/on-boarding. A dedicated project manager is assigned. The scope of work (design, implementation, necessary IaaS resources) is specified. The initial project plan and pricing is also specified. After the initial project acceptance the relevant work is performed. The stages of the implementation project are then accepted by the customer. Later on the processing process is being implemented and performed and included into operation plan.
Standard support services are free of charge for paying IaaS customers.
Extended support services/on-boarding are priced per hour of engineer consulting/design work and per hour of an engineer implementation work. The sizing in hours of categories of work is going to be fixed in the project management process – see section provisioning.
Extended design and processing services are priced according to prices negotiated in the project specification and setup process. These prices cannot be predefined in any way at this stage as such projects can be complex and complicated.