- What We Offer
- Your Processing Environment
- Example of tool usage
- Processing EO Data and Serving www services
- Processing and Storing EO
- Embedding OGC WMS Services into Your website
- GPU Use Case
- Using the EO Browser
- EO Data Finder API Manual
- Use of SNAP and QGIS on a CREODIAS Virtual Machine
- Use of WMS Configurator
- DNS as a Service - user documentation
- Use of Sinergise Sentinel Hub on the CREODIAS EO Data Hub
- Load Balancer as a Service
- Jupyter Hub
- Use of CREODIAS Finder for ordering data
- ESRI ArcGIS on CREODIAS
- se of SNAP and QGIS on a CREODIAS Virtual Machine
- Use of CEMS data through CREODIAS
- Searching, processing and analysis of Sentinel-5P data on CREODIAS
- Legal Matters
- Pricing Plans
- Public Reporting Dashboards
- Sentinel Hub Documentation
- Integration Guides
- OGC API
- Custom Processing Scripts
How to access EO DATA and Object Storage using s3cmd (Linux)?
How to access EO DATA using s3cmd (Linux)
Your virtual machine has to be launched in project with EO DATA!
You can install the s3cmd using Python PIP or from Linux repository.
Installation from system repository on Debian/Ubuntu systems:
Check for updates:
$ sudo apt update
Installing from repository:
$ sudo apt install s3cmd
Installation from Python repository (on most Linux distributions with python and pip preinstalled):
Installing with PIP:
Check if you have PIP installed
$ pipThe program 'pip' is currently not installed. To run 'pip' please ask your administrator to install the package 'python-pip'
If not installed (Ubuntu):
$ pip --version
pip 8.1.1 from
If you see the following:
add the following line:
to the file:
Now you can check the .profile:
$ cat ~/.profile export LC_ALL=en_US.UTF-8 $ source ~/.profile $ s3cmd --version s3cmd version 2.0.1
Enter new values or accept defaults in brackets with Enter.
Refer to user manual for detailed description of all options.
Access key and Secret key are your identifiers for Amazon S3. Leave them empty for using the env variables.
Access Key [access]:<ENTER> Secret Key [access]:<ENTER> Default Region [RegionOne]: <ENTER> Use "s3.amazonaws.com" for S3 Endpoint and not modify it to the target Amazon S3. S3 Endpoint [data.cloudferro.com:] <ENTER> Use "%(bucket)s.s3.amazonaws.com" to the target Amazon S3. "%(bucket)s" and "%(location)s" vars can be used if the target S3 system supports dns based buckets. DNS-style bucket+hostname:port template for accessing a bucket [%(bucket)s.s3.amazonaws.com]: <ENTER> Encryption password is used to protect your files from reading by unauthorized persons while in transfer to S3 Encryption password: <ENTER> Path to GPG program [/usr/bin/gpg]: <ENTER> When using secure HTTPS protocol all communication with Amazon S3 servers is protected from 3rd party eavesdropping. This method is slower than plain HTTP, and can only be proxied with Python 2.7 or newer Use HTTPS protocol [No]: <ENTER> On some networks all internet access must go through a HTTP proxy. Try setting it here if you can't connect to S3 directly HTTP Proxy server name: <ENTER> New settings: Access Key: access Secret Key: access Default Region: RegionOne S3 Endpoint: data.cloudferro.com DNS-style bucket+hostname:port template for accessing a bucket: %(bucket)s.s3.amazonaws.com Encryption password: Path to GPG program: /usr/bin/gpg Use HTTPS protocol: False HTTP Proxy server name: _____ HTTP Proxy server port: 0 Test access with supplied credentials? [Y/n] <ENTER> Please wait, attempting to list all buckets... Success. Your access key and secret key worked fine :-) Now verifying that encryption works... Not configured. Never mind. Save settings? [y/N] y <ENTER> Configuration saved to '/home/eouser/.s3cfg'
Now you can use s3cmd commands (additional information about s3cmd: http://s3tools.org/usage)
2017-12-11 15:30 s3:
2017-12-11 15:30 s3:
2017-12-11 15:30 s3:
In order to acquire access to Object Storage buckets via s3cmd, first you have to generate your own ec2 credentials with this tutorial.
After creation of credentials please remove file .s3cfg in Home folder and then reconfigure s3cmd by entering:
and following values:
New settings: Access Key: (your EC2 credentials) Secret Key: (your EC2 credentials) Default Region: none S3 Endpoint: cf2.cloudferro.com:8080 DNS-style bucket+hostname:port template for accessing a bucket: cf2.cloudferro.com:8080 Encryption password: (your password) Path to GPG program: /usr/bin/gpg Use HTTPS protocol: True HTTP Proxy server name: HTTP Proxy server port: 0
After this operation, you should be allowed to list and access your Object Storage.