1. How to install s3cmd

MacOS

It is recommended to install s3cmd via Homebrew for Mac users but you can also download S3cmd directly from the source and install the package yourself as well.

1. install Homebrew

$ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

2. install s3cmd (after Homebrew is installed)

$ brew update
$ brew install s3cmd

Ubuntu, Debian

$ sudo apt-get update
$ sudo apt-get install s3cmd

RHEL, CentOS or Fedora

$ sudo yum -y update
$ sudo yum install s3cmd

After the installation, open a Terminal window and execute the following command:

  $ s3cmd --version

and you should see the followings (the version number can vary):

  $ s3cmd version 2.0.2

If you can, you have successfully install s3cmd.

Windows

s3cmd is exclusively for Linux and Mac, Windows users please refer to S3Express (a command-line client for Windows).

2. Obtain the access details of your S3-compatible object storage

You will be provided with the access information (i.e. access_key and secret_key) on how to access your S3 object storage via email.

3. Create your s3cmd configuration file

Once your user account (either admin or access account) is created, you will need to create a s3cmd configuration file using the information provided in the provision email.

Please create a file (without the extension) and copy and paste the following information inside and save it as s3cfg_Your-Project-Name (e.g. s3cfg_2018uom000):

[default]
access_key = <<will be provided to you>>
secret_key = <<will be provided to you>>
enable_multipart = True
host_base = objects.storage.unimelb.edu.au
host_bucket = objects.storage.unimelb.edu.au
human_readable_sizes = True
use_https = True

Please replace both access_key and secret_key with the ones listed in your email. This config file is need in order to access your S3 object storage. You will need to create multiple s3cmd config files if you have applied for more than one RDSS S3-compatible object storage.

You can modify the values within this config file but please note that the above values (set by default) are needed in order to access your RDSS S3 object storage. An example of a full s3cmd config file can be found here.

NOTE: please keep your access_key and secret_key secure!!

4. Test access to your S3-compatible object storage

Open a Terminal window and executes the following command (make sure your s3cmd config file, i.e. s3cfg_2018uom000 in this example, is in the same directory where you execute the command):

  $ s3cmd -c s3cfg_2018uom000 ls -H

and you probably will see nothing return as you have not created/ingested any bucket/object yet. Try the following command (to create a bucket/folder):

  $ s3cmd -c s3cfg_2018uom000 mb s3://testBucket

and you should see the following message:

Bucket 's3://testBucket/' created

if you re-run the s3cmd -c s3cfg_2018uom000 ls -H, you will see the following output:

2018-09-21 04:03  s3://testBucket

5. Some useful examples of s3cmd

5.1. How to upload file to your S3-compatible object storage

  $ s3cmd -c s3cfg_2018uom000 put testFile s3://testBucket

and you should see the following progress:

upload: 'testFile' -> 's3://testBucket/testFile'  [1 of 1]
 862208 of 862208   100% in    0s     9.10 MB/s  done

5.2. How to calculate the usage of your S3-compatible object storage

  $ s3cmd -c s3cfg_2018uom000 du s3://testBucket

5.3. How to list all files inside your bucket (folder)

  $ s3cmd -c s3cfg_2018uom000 ls -H  s3://testBucket