Getting started with CREODIAS: Using Object Storage in the CloudFerro cloud
We continue the cycle of technical guides showing how to use CloudFerro cloud infrastructure. In the third article, we focus on using Object Storage.
General Object Storage Management
Creating and managing the Object Storage Container via the Horizon management panel is explained in this article: How to use Object Storage on CREODIAS
Practical accessing Object Storage
The use cases described above are sufficient for basic usage of this storage type. If we were interested in frequent updates of container or use it as ordinary file, access via Horizon panel and pubic links in browser would become inefficient and insecure.
To get more benefits from Object Storage, we can mount it to our filesystem. All the necessary information is in three manuals as listed below.
The first one describes an entire process. The second one (also linked in the first one) shows how to add credentials. The third one refers to a case when you do not have the s3fs software installed.
- How to Mount Object Storage Container as a File System in Linux Using s3fs on CREODIAS
- How to generate and manage EC2 credentials on CREODIAS
- How to install s3cmd on Linux on CREODIAS
Please note two useful hints:
- If you need to mount Object Storage where data was already uploaded, supply mount "s3fs" command with non empty option by adding "-o nonempty".
- If you use multiple cloud projects in your cloud tenant, prepare separate credentials for each project.
If you need to mount Object storage to your local Windows computer or Windows VM in cloud, please take a look at the following manuals:
- How to mount object storage container from CREODIAS as file system on local Windows computer
- How To Mount Object Storage Container as File System on Windows VM on CREODIAS
Advanced options of accessing Object Storage
In some cases users get a single object or a few objects with various frequencies. This causes longer periods when storage is not used. There are also cases with repeating network issues. In those cases mounting the storage would be unnecessary, and access on demand would be a better option. To access object storage on demand, you can use the command line tool s3cmd accessing storage only when it is run.
Another use case is accessing the object storage within your application. As the object storage on CREODIAS is based on Ceph, it is compatible with the S3 standard. You can use s3 libraries for a programming language of your choice. For Python, it is the boto3 library.
Both cases described above are covered in this document: How to access private object storage using S3cmd or boto3 on CREODIAS.
See also other mini guides on how to use CloudFerro cloud resources: