Forum

Message Boards Message Boards

Back

Fails when downloading Sentinel-3 using wget

Toggle
Hi
I have done a search using finder.creodias.eu/resto/api... and I get an xml file with the result of the query. I'm reading the xml-file and finding both the filename and the url for the specific file. But when trying to download it doesn't work. I have tried different variants of the url, but nothing works. Below is the wget command and also the url that I use.

wg='wget --content-disposition --no-check-certificate --user=username --password=pwd --continue --output-document='
commande_wget = '%s%s "%s"' % (wg, pathfilename, URL)
#print(commande_wget)
os.system(commande_wget)


https://zipper.creodias.eu/download/c01c3912-fff4-5c09-af65-ff3c80b71fb5


Below it the message I get for each file in the search loop of my xml-file.

D:\S3\test\S3A_OL_1_EFR____20161129T091051_20161129T091300_20161129T112528_0128_011_264_2031_SVL_O_NR_002.zip
https://zipper.creodias.eu/download/92debb03-8f70-52ae-850a-1a48dd0ea308
SYSTEM_WGETRC = c:/progra~1/wget/etc/wgetrc
syswgetrc = C:\Program Files (x86)\GnuWin32/etc/wgetrc
--2019-05-21 09:55:02--  https://zipper.creodias.eu/download/92debb03-8f70-52ae-850a-1a48dd0ea308
Resolving zipper.creodias.eu... 185.48.233.248
Connecting to zipper.creodias.eu|185.48.233.248|:443... connected.
WARNING: cannot verify zipper.creodias.eu's certificate, issued by `/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3':
  Unable to locally verify the issuer's authority.
HTTP foresprrsel sendt, mottar topptekster... 401 UNAUTHORIZED
Ukjent autorisasjons-protokoll.
Autorisasjon mislyktes
0 (0 Votes)

RE: Fails when downloading Sentinel-3 using wget
Answer
21 May 2019 1:12 as a reply to Anna Birgitta Ledang.
In order to download a product using EOFinder API you have to pass authorisation process by generating and providing keycloak token.
You can obtain it using curl or python itself.
CURL:
123456
export KEYCLOAK_TOKEN=$(curl -d 'client_id=CLOUDFERRO_PUBLIC' \
                             -d "username=${OS_USERNAME}" \
                             -d "password=${OS_PASSWORD}" \
                             -d 'grant_type=password' \
                             'https://auth.creodias.eu/auth/realms/DIAS/protocol/openid-connect/token' | \
                             python -m json.tool | grep "access_token" | awk -F\" '{print $4}')
OS_USERNAME and OS_PASSWORD as environmental variables, which were provided after sourcing RC File. (You can download it in Horizon dashboard)
Python:
123456789101112131415161718
import requests
import json

def get_keycloak_token():
    h = {
        'Content-Type': 'application/x-www-form-urlencoded'
    }
    d = {
        'client_id': 'CLOUDFERRO_PUBLIC',
        'password': 'YOUR_PASSWORD',
        'username': 'YOUR_USERNAME',
        'grant_type': 'password'
    }
    resp = requests.post('https://auth.creodias.eu/auth/realms/dias/protocol/openid-connect/token', data=d, headers=h)
    return json.loads(resp.content.decode('utf-8'))['access_token']

if __name__ == "__main__":
    print (get_keycloak_token())

Remember, that the token lifespan is set on 600 seconds. You'll have to include a refreshing method if you plan to download a lot of product in the same time.
In this thread you can examine how to add token to the URL properly.
https://creodias.eu/-/how-to-download-eodata-products-using-eofinder-api-

If you need more specific tips, we are ready to deliver an aid.

Regards,
Mateusz
0 (0 Votes)