- What We Offer
- Your Processing Environment
- Example of tool usage
- Processing EO Data and Serving www services
- Processing and Storing EO
- Embedding OGC WMS Services into Your website
- GPU Use Case
- Using the EO Browser
- EO Data Finder API Manual
- Use of SNAP and QGIS on a CREODIAS Virtual Machine
- Use of WMS Configurator
- DNS as a Service - user documentation
- Use of Sinergise Sentinel Hub on the CREODIAS EO Data Hub
- Load Balancer as a Service
- Jupyter Hub
- Use of CREODIAS Finder for ordering data
- Legal Matters
- Pricing Plans
- Public Reporting Dashboards
- Sentinel Hub Documentation
- Integration Guides
- OGC API
- Custom Processing Scripts
How to upload and synchronize files with SCP / RSYNC?
SCP and rsync are capable of copying and sychronizing our files between separate locations. Although their destination in achieving the goal is the same, their techniques and concepts differs a little bit.
Let's introduce both of them:
SCP(secure copy) Its concept is to read the source file and assign the full plain linear copy to the destination place: locally or through the network. It is based on ssh.
RSYNCis also responsible for copying files locally or over the network as scp stated above. But moreover it implements some special algorithm to synchronize the data. As you may realize, it does not copy whole directory. It detects changes and redirect them to the destination source.
scp [options] [source] [destination]
scp home/test/file1 email@example.com:/home/eouser/
home/test/file1 is the source file
eouser is a hostname of the server
184.108.40.206 is a ip address of the remote server
home/eouser/ is the destination folder
file1 100% 230 43.6KB/s 00:00
-r It is mainly used for copying folders. Its goal is to Recursively copy entire directories.
Specifies the port to connect to on the remote host.
Selects the cipher to use for encrypting the data transfer.
-C' Compression enable.
This option is necessary when we does not use default files for authenticating ssh connection (id_rsa)
rsync [options] [source] [destination]
rsync -avz –progress -e "ssh -i /home/account/.ssh/id_rsa" file1 firstname.lastname@example.org:/home/eouser/
sending incremental file list file1 sent 135 bytes received 35 bytes 113.33 bytes/sec total size is 57 speedup is 0.3
-progress bar is reponsible for giving results presented above.
As you can see I had to add my private ssh key for authentication with e parameter. There’s the case if we are not interested in sharing different passwords for both protocols.
v : verbose
r : copies data recursively (be cautious, it does not transduce permission while transferring data)
a : archive mode, archive mode allows copying files recursively and it also preserves symbolic links, file permissions, user & group ownerships and timestamps
z : compress file data
h : human-readable, output numbers in a human-readable format