S3cmd is a free command-line tool. It allows you to upload and download your data to the S3 object storage. S3cmd is written in Python. S3cmd is an open-source project available under GNU Public License v2 (GPLv2) and it is free for personal as well as commercial usage.
S3cmd is available in the system repositories for CentOS, RHEL and Ubuntu. You can install it via following guide.
On CentOS/RHEL:
$ sudo yum install s3cmd
On Ubuntu/Debian:
$ sudo apt install s3cmd
Please insert the following lines into the config file located at /home/user/.s3cfg:
[default] host_base = https://s3.clX.du.cesnet.cz use_https = True access_key = xxxxxxxxxxxxxxxxxxxxxx secret_key = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx host_bucket = s3.clX.du.cesnet.cz
Host base and Host bucket is S3 endpoint URL, which you received via email together with Access Key and Secret Key. You should receive it via email during the S3 account creation.
Config file with GPG encryption
Config file with GPG encryption
[default] host_base = https://s3.clX.du.cesnet.cz use_https = True access_key = xxxxxxxxxxxxxxxxxxxxxx secret_key = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx host_bucket = s3.clX.du.cesnet.cz gpg_command = /usr/bin/gpg gpg_decrypt = %(gpg_command)s -d --verbose --no-use-agent --batch --yes --passphrase-fd %(passphrase_fd)s -o %(output_file)s %(input_file)s gpg_encrypt = %(gpg_command)s -c --verbose --no-use-agent --batch --yes --passphrase-fd %(passphrase_fd)s -o %(output_file)s %(input_file)s gpg_passphrase = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
S3cmd commands support elementary operations with buckets - creation, listing, and deletion.
Buckets operation
Buckets operation
Listing all s3 buckets
$ s3cmd ls
Creation of new s3 bucket
$ s3cmd mb s3://newbucket
Removing s3 bucket
$ s3cmd rb s3://newbucket
Only emptied bucket can be removed!
Listing s3 bucket size
$ s3cmd du s3://newbucket/
Files and directories operations
Files and directories operations
Listing of s3 bucket
$ s3cmd ls s3://newbucket/
Data upload into s3 bucket
File upload
$ s3cmd put file.txt s3://newbucket/
Upload of encrypted files
$ s3cmd put -e file.txt s3://newbucket/
Directory upload
$ s3cmd put -r directory s3://newbucket/
Please make sure, that you didn't forget to remove the trailing slash (e.g. .: directory/), trailing slash denotes uploading only the content of the desired directory.
File downloading from s3 bucket
$ s3cmd get s3://newbucket/file.txt
Data deletion from s3 bucket
$ s3cmd del s3://newbucket/file.txt
$ s3cmd del s3://newbucket/directory
Data sync into s3 bucket
$ s3cmd sync /local/path/ s3://newbucket/backup/
Data sync from s3 bucket
$ s3cmd sync s3://newbucket/backup/ ~/restore/
CESNET, z. s. p. o.
Generála Píky 26
16000 Prague 6
Tel: +420 234 680 222
Fax: +420 224 320 269
info@cesnet.cz
Tel: +420 234 680 222
GSM: +420 602 252 531
Fax: +420 224 313 211
support@cesnet.cz