How to download EODATA products using s3cmd?
Establishing connection
At first, please verify your connection with EO DATA bucket by using list command.
CREODIAS_USER@MACHINE:~$ s3cmd
ls
You should obtain this output.
2017-12-11 15:30 s3:
//DIAS
2017-12-11 15:30 s3:
//EOCLOUD
2017-12-11 15:30 s3:
//EODATA
If you have encountered any problems, please check all steps from this tutorial.
Downloading EOData product
Entire product is being structured in a whole directory, not a single file. Therefore, you have to use –recursive parameter in order to avoid any “skipping” of files.
Example for downloading a Sentinel-1 product:
s3cmd get --recursive s3://EODATA/Sentinel-1/SAR/SLC/2018/10/01/S1B_IW_SLC__1SDV_20181001T030840_20181001T030907_012953_017EC9_BA79.SAFE
The above command will download S1B_IW_SLC__1SDV_20181001T030840_20181001T030907_012953_017EC9_BA79.SAFE product to your local folder from which you are issuing the command.
Please mind that if you get the path of the product from our Finder, you will have to alter the original path received from Finder. You have to replace /eodata with s3://EODATA.
Please see it at presented example:
a) original path:
/eodata/Sentinel-1/SAR/SLC/2018/10/01/S1B_IW_SLC__1SDV_20181001T030840_20181001T030907_012953_017EC9_BA79.SAFEb) altered path for s3cmd:
s3://EODATA/Sentinel-1/SAR/SLC/2018/10/01/S1B_IW_SLC__1SDV_20181001T030840_20181001T030907_012953_017EC9_BA79.SAFE
Troubleshooting
If nothing happens or you could not download a product properly, please issue -d
switch to see a debug output for further diagnosis:
s3cmd -d get --recursive s3:
//EODATA/Sentinel-1/SAR/SLC/2016/05/14/S1A_IW_SLC__1SDV_20160514T030935_20160514T031002_011249_011058_2013
.SAFE
Alternative solution
For syncing you are able to use s3cmd sync command (for the first time it downloads a whole product).
We can find a description in manual page (man s3cmd):
s3cmd sync LOCAL_DIR s3://BUCKET[/PREFIX] or s3://BUCKET[/PREFIX] LOCAL_DIR Synchronize a directory tree to S3 (checks files freshness using size and md5 checksum, unless overridden by options, see below)
In order to sync data to local folder, you have to put a dot (. → location of the current directory) as a last sign.
s3cmd sync s3://EODATA/Sentinel-1/SAR/SLC/2016/05/14/S1A_IW_SLC__1SDV_20160514T030935_20160514T031002_011249_011058_2013.SAFE .