NFS

CESNET provides a large a storage that can be accessed via NFSv4 protocol.

These storages can be also accessed through your personal computer. This guide contains instructions how to do it on machines running Linux OS. Unfortunately for MS Windows isn't NFSv4 client which would be free, there's only commercial NFS Maestro.

To ensure security of data is required authenticated connection via Kerberos. When you use proper options of mount command you can get encrypted communication.

To access the NFSv4 servers that CESNET provides you'll need the following things:

Hostname that can be transleted. You can vetify it with:

• ping -c 1 `hostname`
• ping -c 1 `hostname -f`
• Empty directory used for mounting point
• Installed support for Kerberos
• Kerberos ticket allowing to mount the drive (ticket available under root, prospective keytab)
• Cleint's NFS tools
• Kerberos ticket for data access (available under the user which you are working)
• correctly filled /etc/krb5.conf, more in section Kerberos. You can use this krb5.conf (if you mount MetaCentrum storage as well, read this guide)
• root account on the local computer
• correctly set the system time
• In case you are using CentOS 5.8 you will need:
• Kernel version 2.6.18-308.4 or newer (otherwise you will get: cd: [directory]: Not a directory)
• nfs-utils version 1.0.9-61 that you can donwload here (otherwise you will get message: Permission denied)
• For all machines except mobile clients we recommended to have your FQDN hostname in /etc/hosts (output of hostname -f command), e.g.

198.51.100.10   myclient.domain.com myclient

A shared space should be mapped to any directory. Sdílený prostor je třeba mapovat do nějakého adresáře. We recommend to follow the convention established in MetaCentrum, i.e. subdirectory in /storage:

  mkdir -p /storage/cesnet-du4

After you will attach mount point directory you need ticket. Ticket can get only root and he can get it one of two choices:

• Use user ticket with expiration period
• Use the credentials saved in the /etc/krb5.keytab

Depending on used way, the client's tools configuration is a little bit different.

As root is need to make a ticket with command

kinit user@EINFRA

where user is your login. Verify validity of ticket is described on this site. Disadvantage of this process is a limited ticket validity which means also limited time of mount device. After expiration of ticket is impossible to enter to device. Maximum period for this method of mounting is 8 days (maximum period of validity of renewable ticket).

If you get a special keytab for service indentities, you can skip this section.

Long-lasting keytab for users can be get by following method:

ssh -o PubkeyAuthentication=no -o GSSAPIAuthentication=no USER@SERVER "remctl kdccesnet.ics.muni.cz accounts nfskeytab" > krb5.keytab

where USER is your login used in EINFRA and SERVER depends, where you wish to connect (e.g. for Ostrava data storage “ssh.du4.cesnet.cz”).

Place the obtain file krb5.keytab to /etc/krb5.keytab.

The file owner should be root and permission for read and write should have only root, so:

 # ls -l /etc/krb5.keytab 
-rw------- 1 root root 57 Nov  2 09:48 /etc/krb5.keytab

We recommend to install tools from our repositories. These packages already contains patches for right function of some attributes. Debian9 / Ubuntu18.04 (i386/amd64)

apt update
apt install nfs-common libnfsidmap2

Debian6 / Ubuntu12.04 (i386/amd64)

wget ftp://homeproj.cesnet.cz/apt/du/du.list -O /etc/apt/sources.list.d/cesnet-du.list
wget ftp://homeproj.cesnet.cz/apt/du/APT-GPG-KEY-du -O - | apt-key add -
apt-get update
apt-get install nfs-common libnfsidmap2

openSUSE12 (only x86_64)

zypper addrepo ftp://homeproj.cesnet.cz/rpm/du-opensuse/stable/x86_64/
zypper install --no-recommends nfs-client libnfsidmap

CentOS6 (only x86_64)

wget https://du.cesnet.cz/_media/cs/navody/nfs/cesnet-du.repo -O /etc/yum.repos.d/cesnet-du.repo
yum install nfs-utils nfs-utils-lib

For another OS we recommend have installed nfs-utils version 1.1.0 and higher.

At the same time you will need running rpcbind or (in older versions) portmap. It should be installed in dependencies of nfs-utils. If it's not, use same-named package (portmap, rpcbind).

After correct setup nfs-utils should be possible to mount volume as root. You still have to do next steps.

Setting nfs-utils is in file /etc/default/nfs-common. Set the values like this:

NEED_STATD=yes
STATDOPTS=
NEED_IDMAPD=yes
NEED_GSSD=yes

Next you need to add a line in /etc/default/nfs-common (for Debian and Ubuntu less than 12.04):

RPCGSSDOPTS="-n"

For Ubuntu 12.04 and newer is change in /etc/init/gssd.conf: Original line:

exec rpc.gssd

replace by:

exec rpc.gssd -n  
  

After these changes is necessary to restart service (if it has run - nfs-common/service gssd).

Setting of nfs-utils is in file /etc/sysconfig/nfs. Set up at least these values:

NFS_SECURITY_GSS="yes"
NFS4_SUPPORT="yes"  
RPCGSSDARGS="-n"

Setup for nfs-utils is in the file /etc/nfs.conf. Set up at least these values:

SECURE_NFS="yes"  
RPCGSSDARGS="-n"

Settings for nfs-utils is in the file /etc/default/nfs-common. Set the values like this:

NEED_STATD=yes
STATDOPTS=
NEED_IDMAPD=yes
NEED_GSSD=yes      

Setting of nfs-utils is in file /etc/sysconfig/nfs. Set up at least these values:

NFS_SECURITY_GSS="yes"
NFS4_SUPPORT="yes"

Setup for nfs-utils is in the file /etc/sysconfig/nfs. Set up at least this value:

SECURE_NFS="yes"

To access connected drive under different user than root is necessary to generate user ticket under the identity of local user. That can be done by using kinit command.

$localuser@localhost > kinit USER@EINFRA
Password for USER@EINFRA: 

where USER is your login in EINFRA.

To write out the validity of your ticket

$localuser@localhost > klist
Ticket cache: FILE:/tmp/krb5cc_1000
Default principal: USER@EINFRA

Valid starting    Expires           Service principal
27/06/2012 13:52  28/06/2012 13:52  krbtgt/EINFRA@EINFRA
$USER@localhost >

In some special cases (typically backuping) could be preferable, that special (service) user could access the data storage without necessity to generate kerberos ticket or the password usage. That could be done by using keytab. Connection the drive. In the following guide we will assume, that the local user name is bacula.

First ew can copy the file /etc/krb5.keytab, which should be accessible only for user root into different location and we will set the premissions for bacula user:

root@localhost > cp /etc/krb5.keytab /etc/krb5.keytab.bacula
root@localhost > chown bacula:bacula /etc/krb5.keytab.bacula

From this file is then possible periodically obtain the ticket by using following command (under bacula bacula user):

bacula@localhost > kinit -k -t /etc/krb5.keytab.bacula nfs/USER@EINFRA 

Like this we will get ticket with one day validity (under the user bacula). The command is no longer required to enter a password, therefore can be automate eg. using cron.

Correct format of principal (string nfs/USER@EINFRA) can be get by:

klist -k /etc/krb5.keytab.bacula
Keytab name: WRFILE:/etc/krb5.keytab.bacula
KVNO Principal
---- --------------------------------------------------------------------------
   1 nfs/USER@EINFRA

Into file /etc/fstab write line:

nfs.du4.cesnet.cz:/ /storage/cesnet-du4   nfs4   sec=krb5,rsize=1048576,wsize=1048576   0 0

Volume will be then mount automatically.

Then you need to start the nfs and ensure its automatic start after reboot:

/etc/init.d/nfs start 
insserv /etc/init.d/nfs

/etc/init.d/nfs-common start

service portmap start
service gssd start

chkconfig rpcbind on
chkconfig nfs on
chkconfig rpcsvcgssd on
chkconfig rpcidmapd on
service nfs

systemctl enable nfs-idmap
systemctl start nfs-idmap

systemctl enable nfs-secure
systemctl start nfs-secure

File /etc/idmapd.conf set up NFSv4 mapping (NFSv4 works with text principal and POSIX interface works with numeric representation of users and groups).

Simple settings in /etc/idmapd.conf is to set following line in section [General]:

  Domain = EINFRA

After that will be all users from domain @EINFRA mapped by using the files /etc/passwd and /etc/group. It means that for the identity login@EINFRA must exist the record in /etc/passwd with the login name. Common programs (ls -l) shall show names and groups from domain EINFRA correctly, only if there will be for their principal the name in the particular files. For non-existing name will be displayed the mapping as nobody and nogroup.

Probably you will see owner only of your own files (if you have the same login on your computer like in EINFRA. Otherwise you will see the owner “nobody” and group “nogroup” in every file), and in other files the owner will be “nobody” and group “nogroup”. This image can have influence to permissions to the files, depends on optimization of client's kernel, even that permissions are dealed by server and its configuration. So it's possible that with a wrong configuration of mapping, some operation won't be done, even the server sees everything is alright – client doesn't send request to server at all. But it's not possible to client get more permissions like it actually should get.

Actual versions nfs-utils offering the option of the static users mapping (useful when you have a different login on your local computer then login in EINFRA), but an implementation is function-less. Setting of a function static mapping is more difficult and it's described bellow. Standard support is only for mapping method “nsswitch”, so this is necessary in appropriate section:

[Translation]

Method = nsswitch

or you don't have to that at all, method nsswitch is implied.

In some cases is not possible to ensure the same user's login in client's side, as in the storage. Then, some operations can't work properly, for example you can't use ENCFS over NFS volume. For functional static mapping is necessary to have a newer version of program libnfsidmapd, which includes this patch.

For the compilation, we first need to download the latest version of libnfsidmap:

cd ~/
git clone git://git.linux-nfs.org/projects/steved/libnfsidmap.git

Then make compilation:

cd libnfsidmap
./autogen.sh 
./configure
make

It is also necessary run make install with root privileges. When you start rpc.idmapd daemon again it will use this version of the library.

Then it is necessary to modify the relevant sections in the configuration file /etc/idmapd.conf:

[Translation]
Method = static, nsswitch

[Static]

user@EINFRA = localuser
user2@EINFRA = anotherlocaluser
storage@EINFRA = users

Configuration shows static mapping of user user@EINFRA, user2@EINFRA and group storage@EINFRA. In case of other users and groups will be mapping performed by plugin 'nsswitch which is described above. Users who don't use any of these two methods will be mapped as nobody''.

nfs/USER@EINFRA = localuser

nfs/backup_VO@EINFRA-SERVICES = localuser

Connection of volume isn't necessary to perform manually (for Ubuntu and openSUSE 11.1 and higher). The nfs service starts automatically because of record in /etc/fstab.

If the connection doesn't established automatically, you can do it manually. Connection of volumes must be performed with root privileges. Make sure you have the correct /etc/krb5.keytab or created user ticket under the root. Also make sure that portmap and nfs-common (nfs-utils) are running. You can verify it by following instructions:

ps ax | grep rpc.gssd
ps ax | grep portmap

To connect perform following command:

mount -t nfs  nfs.du4.cesnet.cz:/ /storage/cesnet-du4 -o vers=4 -o sec=krb5 -o rsize=1048576,wsize=1048576

You can use instead of /storage/xxx je možno zadat any other local directory, where do you want to mount NFS volume. If you use sec=krb5 option instead of sec=krb5i the integrity will be checked. If you use sec=krb5p then data will be transmitted encrypted.

If you followed steps in this guide you have mounted storage and you can use ticket to access data storage.

In case of problems, please contact our support see FAQ. To speed up the solution you can send output of the following commands (report.log) to to the support. All command must be pertorm as root or with sudo.

wget http://www.fi.muni.cz/~xhejtman/check-nfs.sh
mount nfs4 [příkaz pro mount, který následně selže nebo jiný příkaz, který se chová jinak, než by dle návodu měl]
. ./check-nfs.sh &> /tmp/report.log