This document describes directory (folder) structures on CESNET hierarchical storage facilities.
A user home directory contains subdirectories that correspond to virtual organizations (VO) the user is a member of, and migration policies. These subdirectories are used to store data. They are symbolic links (shortcut folders) pointing to the directory structure of the VO.
The home directory isn't designed to store your data. Its just for storing SSH keys and other configuration files, and for creating links to the real data. The home directory has soft quotas 50 MiB/500 files 1) and hard 5 GiB/5000 files 2).
Example for user
“user” with membership in two VOs (eg. VO_compass and VO_storage), one of them uses two different migration policies for du1, du2, du3:
/home/user: |-- .bash_history |-- .screenrc |-- .ssh | \-- authorized_keys |-- VO_compass-tape_tape -> /exports/tape_tape/VO_compass/home/user |-- VO_compass-disk_only -> /exports/disk_only/VO_compass/home/user \-- VO_storage-cache_tape -> /exports/cache_tape/VO_storage/home/user
Example for user
“user” with membership in two VOs (eg. VO_compass and VO_storage), one of them uses two different migration policies for du4, du5, …:
/home/user: |-- .bash_history |-- .screenrc |-- .ssh | \-- authorized_keys |-- VO_compass-tape_tape -> /exports/tape_tape/backup/VO_compass/home/user |-- VO_compass-tape_tape-archive -> /exports/tape_tape/archive/VO_compass/home/user |-- VO_compass-disk_only -> /exports/disk_only/backup/VO_compass/home/user
This structure we will automatically prepare to the user whenever he becomes a member of VO. Users can of course create a different structure using symbolic links. However, the links created by us, we do not recommended to delete.
We are preparing an Accounting system. This system will show occupied space and quota settings. Accounting system is coming soon.
Migration policy determines what happens to your data when you run out of space on the disks, where data that no one else is using will be moved etc. Read the page on HSM (Hierarchical Storage Management) if you need information about the principles of data migration.
Migration policy agreed administrator of virtual organization with administrators of data storage CESNET. Virtual organizations often use only one or a few policies.
Currently available are the following migration policies.
tape- data on SATA disks and copy on a single tape (suitable for second or third backups)
tape_tape- data on SATA disks and copy on two different tapes (for standard data and first backups)
disk_only- data is only for disks (RAID), and never moved nor backed up (for fast-changing data, e.g., backups you delete in several days)
cache_tape- primary data is on fast Fibre Channel drives, one copy on the tape and SATA drives. If run out of space on the SATA area, it will create a second copy on tape, i.e., two different tapes in total (suitable for faster access to standard data).
If the user
member in “storage”) wants to use specific migration policy
“tape_tape” (data on two different tapes) than he uploads data into the directory
“VO_storage-tape_tape” in his home directory.
Below is the list of migration policies and their quotas contained in the virtual organization „storage“ which each registered user is automatically a member.
cache_tape- policy is suitable for storing livelier data. It is not recommended to store backups and archives.
tape_tape- because the data for this policy are stored on two different tapes, it is suitable to use this policy primarily for storing backups and archives.