Rsync

• This variant is suitable for storing the files of one user, assuming that there is no need to keep information about the files owner
• It is possible access to data storage from different machines - for example, back up one system but do recover files on multiple machines (with different bond between user account and user identifiers uid/gid).

In a GNU/Linux is rsync installation available directly from the package manager (in most distributions).

Example of use rsync for backup directory tree folder to account user server host into directory path (rsync format):

rsync -av folder user@host:path

Example of restore directory tree folder from account user server host into working directory.

rsync -av user@host:path/folder .

Graphical interface of Grsync in Windows:

Graphical interface in GNU/Linux / Xorg:

1. … you can define server setting here and save it
2. … source location (rsync format, i.e. user@host:path)
3. … destination location (rsync format)
4. … caution: prompt for server password or SSH keys may appears in the command window from which the program was run
5. … if you move mouse over the checkbox then it displays information about appropriate parameter for line rsync.

• This option of use the service rsync is suitable for storing synchronized backup file system with data ownership and access rights, where such data is necessary for files to keep.
• E.g. back up file server which is used to by multiple users and when restore of files from a backup happens then have to be set ownership and permissions same as the original files.

• during the use of service over certain directory tree should be still calling rsync with te same parameters setting --numeric-ids and --fake-super
• set of users in the source system have to be fixed to assign numerical identifiers accounts (uid/gid).
• to data storage should be accessed only from one source system or multiple systems where they have the same numeric identifiers accounts (uid/gid).
• we don't recommend usage GUI client grsync in Windows in this variant - in tests were appeared incorrect mapping users uid/gid.

If you break these conditions it will very likely leads to inconsistent data about files ownership.

--numeric-ids

• when you are using this parameter the server process doesn't apply for mapping uid/gid values ​​of the user/group name but it will use directly numerical values ​​(uid/gid) of the source system.
• therefore the values uid/gid mustn't be changed and the parameter must be constantly use in one directory tree or not use at all.

--rsync-path

• this parameter allows you to pass parameters to the server process - this option is used to pass parameters --fake-super:

  --rsync-path="rsync --fake-super"

--fake-super

• this parameter enables server-side emulation of superuser mode and then rsync stores the data about ownership and access rights to the extended attributes (xattrs)

--super

• this parameter forces root behaviour - changes in ownership and rights to files. In the tests were needed only for restore the files to the client with Windows OS. Caution: when you are sending files on non-root accountthen you can't use this parameter (right choice int this case is --fake-super in --rsync-path).

Example of use rsync for backup directory tree folder to account user server host into directory path:

rsync -av --numeric-ids --rsync-path="rsync --fake-super" folder user@host:path

Example of restore directory tree folder from account user server host into working directory.

rsync -av --numeric-ids --rsync-path="rsync --fake-super" user@host:path/folder .

Example of back up and restore directory of user Adminstrator in Windows:

rsync -av --numeric-ids --rsync-path="rsync --fake-super" /cygdrive/c/Documents\ and\ Settings/Administrator user@host:path

I you are doing restore in Windows you may need additional parameter --super for proper recovery of information about files ownership and access rights:

rsync -av --numeric-ids --super --rsync-path="rsync --fake-super" user@host:path/ /cygdrive/c/Documents\ and\ Settings/restore

Notation for remote location

user@server:path

where:

• user … username of the account which is used to access data storage.
• server … domain name of data storage server.
• path … relative path (relate to the user's home directory) in the file system data storage.
  

Example:

user@ssh.du4.cesnet.cz:VO_storage-tape_tape/zalohy/1205-kveten

To check the integrity of your files after transfer you can use Hasher app, which is described in the following guide.

Website of the project rsync: http://rsync.samba.org/
Website of the project Grsync: http://www.opbyte.it/grsync/
Website of the project Grsync for Windows: http://grsync-win.sourceforge.net/
Website of the project Grsync for MacOS X: http://grsync-mac.tuxfamily.org/?lang=en

The Network-attached storage (NAS) from Synology is implicitly communicating via Rsync process which is not running on the server side due to security reasons. It is possible to do backups from NAS Synology via Rsync by switching the interface from GUI to CLI and using standard SSH / rsync via script which will be called by cron. Please note that the shell in NAS Synology is not bash, sh, tcsh, csh … but ash.

   
#!/bin/ash

LOG_FILE=/var/log/backup_`date '+%F'`.log
LOG_FILE_ERR=/var/log/backup_errors_`date '+%F'`.log

echo "`date '+%b %d %T'` ==== Start of transmission ====" >> $LOG_FILE

rsync -a --exclude="[#]recycle/" /volume1/backup service_account@ssh.duX.cesnet.cz:~/VO_nazev-tape_tape/ 1>> $LOG_FILE 2>> $LOG_FILE_ERR

RETURN_CODE=`echo $?`
echo "`date '+%b %d %T'`==== End of transmission ====" >> $LOG_FILE

# Back up logs as well
rsync -a /var/log/backup* service_account@ssh.duX.cesnet.cz:~/VO_nazev-tape_tape/