Message Boards Message Boards


Fails when downloading Sentinel-3 using wget

I have done a search using and I get an xml file with the result of the query. I'm reading the xml-file and finding both the filename and the url for the specific file. But when trying to download it doesn't work. I have tried different variants of the url, but nothing works. Below is the wget command and also the url that I use.

wg='wget --content-disposition --no-check-certificate --user=username --password=pwd --continue --output-document='
commande_wget = '%s%s "%s"' % (wg, pathfilename, URL)

Below it the message I get for each file in the search loop of my xml-file.

SYSTEM_WGETRC = c:/progra~1/wget/etc/wgetrc
syswgetrc = C:\Program Files (x86)\GnuWin32/etc/wgetrc
--2019-05-21 09:55:02--
Connecting to||:443... connected.
WARNING: cannot verify's certificate, issued by `/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3':
  Unable to locally verify the issuer's authority.
HTTP foresprrsel sendt, mottar topptekster... 401 UNAUTHORIZED
Ukjent autorisasjons-protokoll.
Autorisasjon mislyktes
0 (0 Votes)

RE: Fails when downloading Sentinel-3 using wget
21 May 2019 1:12 as a reply to Anna Birgitta Ledang.
In order to download a product using EOFinder API you have to pass authorisation process by generating and providing keycloak token.
You can obtain it using curl or python itself.
export KEYCLOAK_TOKEN=$(curl -d 'client_id=CLOUDFERRO_PUBLIC' \
                             -d "username=${OS_USERNAME}" \
                             -d "password=${OS_PASSWORD}" \
                             -d 'grant_type=password' \
                             '' | \
                             python -m json.tool | grep "access_token" | awk -F\" '{print $4}')
OS_USERNAME and OS_PASSWORD as environmental variables, which were provided after sourcing RC File. (You can download it in Horizon dashboard)
import requests
import json

def get_keycloak_token():
    h = {
        'Content-Type': 'application/x-www-form-urlencoded'
    d = {
        'client_id': 'CLOUDFERRO_PUBLIC',
        'password': 'YOUR_PASSWORD',
        'username': 'YOUR_USERNAME',
        'grant_type': 'password'
    resp ='', data=d, headers=h)
    return json.loads(resp.content.decode('utf-8'))['access_token']

if __name__ == "__main__":
    print (get_keycloak_token())

Remember, that the token lifespan is set on 600 seconds. You'll have to include a refreshing method if you plan to download a lot of product in the same time.
In this thread you can examine how to add token to the URL properly.

If you need more specific tips, we are ready to deliver an aid.

0 (0 Votes)