How to download EODATA products using s3cmd?

Establishing connection

At first, please verify your connection with EO DATA bucket by using list command.

CREODIAS_USER@MACHINE:~$ s3cmd ls

You should obtain this output.

2017-12-11 15:30 s3://DIAS

2017-12-11 15:30 s3://EOCLOUD

2017-12-11 15:30 s3://EODATA

If you have encountered any problems, please check all steps from this tutorial.

FAQ S3CMD

Downloading EOData product

Entire product is being structured in a whole directory, not a single file. Therefore, you have to use –recursive parameter in order to avoid any “skipping” of files.

Example for downloading a Sentinel-1 product:

s3cmd get --recursive s3://EODATA/Sentinel-1/SAR/SLC/2018/10/01/S1B_IW_SLC__1SDV_20181001T030840_20181001T030907_012953_017EC9_BA79.SAFE

The above command will download S1B_IW_SLC__1SDV_20181001T030840_20181001T030907_012953_017EC9_BA79.SAFE product to your local folder from which you are issuing the command.
Please mind that if you get the path of the product from our Finder, you will have to alter the original path received from Finder. You have to replace /eodata with s3://EODATA.
Please see it at presented example:

a) original path:

/eodata/Sentinel-1/SAR/SLC/2018/10/01/S1B_IW_SLC__1SDV_20181001T030840_20181001T030907_012953_017EC9_BA79.SAFE

b) altered path for s3cmd:

s3://EODATA/Sentinel-1/SAR/SLC/2018/10/01/S1B_IW_SLC__1SDV_20181001T030840_20181001T030907_012953_017EC9_BA79.SAFE

Troubleshooting

If nothing happens or you could not download a product properly, please issue -d switch to see a debug output for further diagnosis:

s3cmd -d get --recursive s3://EODATA/Sentinel-1/SAR/SLC/2016/05/14/S1A_IW_SLC__1SDV_20160514T030935_20160514T031002_011249_011058_2013.SAFE

Alternative solution

For syncing you are able to use s3cmd sync command (for the first time it downloads a whole product).

We can find a description in manual page (man s3cmd):

s3cmd sync LOCAL_DIR s3://BUCKET[/PREFIX] or s3://BUCKET[/PREFIX] LOCAL_DIR
Synchronize a directory tree to S3 (checks files freshness using size and md5 checksum, unless overridden by options, see below)

In order to sync data to local folder, you have to put a dot (. → location of the current directory) as a last sign.

s3cmd sync s3://EODATA/Sentinel-1/SAR/SLC/2016/05/14/S1A_IW_SLC__1SDV_20160514T030935_20160514T031002_011249_011058_2013.SAFE .

TAGS