NFS (network drive)

The NFSv3 protocol is available on request for existing Mediaflux projects. It is generally simpler to access Mediaflux through any of the other protocols; we generally recommend using NFS only if you know you absolutely need it. Get in touch with the team to discuss your requirements.

Currently only the NFSv3 protocol is supported. NFSv4 may come in a future release of Mediaflux.

Share an existing Mediaflux project using NFS

To share an existing Mediaflux project using the NFS protocol, we will need the following information:

Project name
UID to domain:username mapping
GID to group mapping (optional)
The list of IP addresses that should be enabled for normal user access
The list of IP addresses that should be enabled for root access (if not specified, root will be squashed).

IP address-based access restriction

In order to access your Mediaflux project with NFS is a list of the IP addresses belonging to the machine(s) that will be allowed to mount the NFS share. An IPv4 address is expressed in dot-decimal notation which has the following format: x.x.x.x where x is called an octet and must be a decimal value between 0 and 255 (for example, 192.168.1.1). With each IP address we require a netmask. Netmasks are also expressed in dot-decimal notation like an IP address. Usually, the netmask will be 255.255.255.255, which specifies a single host.

A second list of IP address/netmask pairs is required for any machines that require root access to the NFS share. If none are specified root will be squashed which is usually what you want.

IP address restrictions determine whether an NFS share can be mounted on a host. However, it is when a user first interacts with the mount that Mediaflux attempts to log that user in on the server side. Note that this could potentially trigger other processes associated with the login, such as MFA. We currently have MFA disabled for NFS access, though this could change in future. This user who triggers this login process is referred to as the caller. This user may be used to determine the mode bits associated with a file (see below).

Users, groups and IDs

The main difference between accessing your Mediaflux project using the NFS protocol and using a dedicated NFS server relate to how user and group ownerships and permissions are handled. A regular NFS server offers access only through the NFS protocol, and no attempt is made to keep track of what user accounts relate to which user IDs. Because Mediaflux offers access to your files through many protocols, we must map all NFS user and group IDs to equivalent user and role entities in Mediaflux. We must also generate an owning user and group for each file and directory. Considerable complexity can arise from this process.

If you only use Mediaflux through POSIX protocols like NFS and SMB, then a lot of the complexity won’t be evident.

Synthesised owning user and group

Every file and directory in NFS is owned by a user and a group. There are three types of access permissions: read, write, and execute. Different access permissions can be applied to a file's owning user, owning group, and others (those without ownership).

rwh@thinkpad:~/example$ ls -l
total 12
drwxrwxr-x 2 rwh posix 4096 Sep  2 09:56 directory
-rw-rw-r-- 1 rwh posix   26 Sep  2 09:56 test.txt
-rw-rw-r-- 1 rwh rwh     26 Sep  2 10:31 test2.txt

In Mediaflux, these owning user, owning group and access permissions are synthesised from the access information for each file or directory. Mediaflux uses the following sources of information to do this:

UID and GID mappings to determine which Mediaflux user account to use
Various access permission methods that Mediaflux supports (XXX see below)

It is important to realise that as Mediaflux uses a finer-grained set of permissions for its ACLs and synthesises the Linux ownership and access modes from those ACLs, it is possible to have a situation where it appears you should be able to do an action through NFS when you can’t actually do so. One example of this would be where your user has the ACLs to read, write and execute a file, but not have the ACL to delete that file. There is no way to represent that situation with only the read, write and execute permissions that are supported on Linux. In this case, strictly only the access you have on the Mediaflux server side will be allowed.

UID mappings

You must provide mappings from numeric NFS UID to domain:username in Mediaflux. These numeric IDs can usually be found in the /etc/passwd file on the host machine, or you can use the id command to return the user ID of the current user.

rwh@thinkpad:~$ grep rwh /etc/passwd
rwh:x:1000:1000:Robert Hutton:/home/user:/bin/bash
rwh@thinkpad:~$ id
uid=1000(rwh) gid=1000(rwh) groups=1000(rwh),1003(posix)

In this case, the rwh user needs to be mapped to a user that exists in the Mediaflux project.

UIDs must map to a single user in Mediaflux. An example might be UID 1000 on a Linux machine mapping to unimelb:rhutton in Mediaflux.

GID mappings

Mediaflux handles NFS group functionality by mapping from NFS GID to a role in Mediaflux. Usually, we map one of the project roles to an NFS GID. If you need to use group functionality beyond this basic setup, then you will likely need the following:

A group on the Linux side, with user accounts added to it in the usual way
UID mappings to unimelb accounts in Mediaflux for the users that are members of the Linux group
A custom role within your Mediaflux project with the same list of unimelb accounts added to it
A GID mapping, which maps the Linux GID to the custom role.

By default in Mediaflux, if you add a group to a directory in this way, it will be equivalent to having the setgid bit set. That is to say, all the contents of a directory will also inherit the group ownership.

GIDs must map to a single role in Mediaflux. This might be an existing project role like participant-acmd-n or a custom role created within the project namespace.

How Mediaflux determines access permissions

Owner

Files and directories may have an owner in Mediaflux. Typically this will be the case if the asset was created through a posix protocol (e.g. smb, nfs).

admin:rh@test:/projects/proj-rob-1247.5.36# asset.get :id 58157150
	:asset -id "58157150" -version "1" -vid "161903919"
[snip]
		:owner -id "22184" -pid "1247.28683097"
			:domain "unimelb"
			:user "rhutton"
			:name "Robert Hutton"
			:email "robert.hutton@unimelb.edu.au"

If the asset does not have an owner set, then the first user ACL (if any) is deemed to be the owner. If there is no owner found using this algorithm, then the owner is set to UID 0 (the root user). For Mediaflux projects where many files or directories have been created through non-posix protocols, many files will be owned by root in this way and it will be the group permissions that determine a user’s access.

asset.owner.set in aterm may be used to set the owner of an asset. This is analogous to the chown command on Linux. asset.owner.clear will remove the explicit ownership.

Group

The group of an asset cannot be explicitly set. The group that is shown for an asset is the first ACL that is assigned to a non-user role where that role also exists in the identity map. For example:

admin:rh@test:/# ds.project.posix.map.describe :project-id 36
	:map -name "proj-rob-1247.5.36"
		:user -uid "1000" -id "22184" "unimelb:rhutton"
		:role -gid "1000" "proj-rob-1247.5.36:participant-acmd-n"
		:role -gid "1003" "proj-rob-1247.5.36:posix"
		:nb-users "1"
		:nb-groups "2"

The order of the non-user role ACLs on most projects is:

administrator
participant-a
participant-acm
participant-acmd
participant-acmd-n

Recommendation: add only one of the standard project roles as a group in the uid/gid map. If you need additional groups, create additional roles in the project namespace, add those to the gid map, then apply an ACL to the correct collection to provide access for that group.

File mode (read/write/execute)

If the asset has an owner, then the owner mode bits are generated from the access the owner (user) has to the asset. That will be determined by any ACLs on the asset and/or collection, including those that are possibly inherited from a parent. The permissions that are relevant are read, write, and execute. Once access ACLs are resolved:

See Owner above for how the owner is determined. The owner mode bits are generated from the first ACL that matches the owner (user) for the asset. If the asset does not have a specified owner, then the first ACL matching the caller is used. If the user has no ACL, then the first ACL for any user is used to generate the owner mode bits. If no suitable ACL is found, then the owner mode bits are set according to the permissions available to the caller.
The group mode bits are generated from the first ACL that has a non-user type of role. If no such ACL is found, then those bits are set to zero (---).
The other mode bits are generated from the first ACL that matches the role user. If no such ACL is found, then those bits are set to zero (---).

Since there may be other ACLs that do allow the caller (based on their uid) to access/modify the asset then operations may in fact proceed, despite what might be reported by the mode bits. This situation arises because there is no POSIX convention for representing the permissions held by the caller (and NFS clients cache per server, not per user) - mode bits are mapped only to owner, group and other.

Often, the read, write, and execute permissions derive from the ACLs on the Mediaflux project root level. We use standard user roles to define a user's access at the project's root level. The standard user roles map to the following:

project role	file mode (for group)	notes

project role	file mode (for group)	notes
administrator	`rwx`
participant-a	`r--`
participant-acm	`rwx`	Cannot delete files or directories
participant-acmd	`rwx`	Cannot delete directories
participant-acmd-n	`rwx`

chmod, chown and chgrp support

The default configuration for Mediaflux NFS shares is:

:read-only false
:apply-mode-bits false
:attributes-from-meta false

This means that by default, users can write to the filesystem, but that they cannot use chmod, chown or chgrp to work with file ownership or modes. Instead, you can work with these permissions on the Mediaflux server side, through aterm, e.g. asset.owner.set and asset.owner.clear are the equivalent of chown.

If you know that you will need to use these commands, please contact us to discuss.

apply-mode-bits

If you set :apply-mode-bits true on a project, the user gains the ability to set file modes with the chmod command on the client. By default, these modes are stored as ACLs. See attributes-from-meta, below.

attributes-from-meta

If you set :apply-mode-bits true :attributes-from-meta true then the mode bits are stored in arc.posix.attributes metadata, rather than the asset itself (e.g. from ACLs).

Mounting on the client

Create the mountpoint, e.g.

sudo mkdir /mnt/proj-projmanme-1128.4.1234

From the command line:

sudo mount -v -t nfs -o mountport=9001,port=9001,nolock,proto=tcp,nfsvers=3 research-nfs.storage.unimelb.edu.au:/proj-projname-1128.4.1234 /mnt/proj-projmanme-1128.4.1234

Using fstab. First add a line to the /etc/fstab file:

research-nfs.storage.unimelb.edu.au:/proj-projmanme-1128.4.1234 /mnt/proj-projmanme-1128.4.1234 nfs mountport=9001,port=9001,nolock,proto=tcp,nfsvers=3,_netdev,nofail	0	0

Then mount with:

sudo mount /mnt/proj-rob-1247.5.36