This means somebody else wants to share data with you and they have sent you a URL to paste in your browser (or use as an argument to Unix tools like wget or curl).  When you paste it into your browser and press return, this will activate a download process.  The difference with direct shareable links is that rather than directly downloading the data, it will download a zip file holding scripts/applications (perhaps a download manager) that itself will download the data for you.

With this approach, the download can be more efficient and robust.

• Can be restartable
• Can download in parallel in some cases

After download, unpack the container (it will be a standard zip file) to access the scripts.  We offer two types of data download scripts at present (and the person provisioning your link will have discussed with you which is more suitable). We also offer scripts for Unix and Windows (and both flavours will be in the downloaded zip file).

ATERM wrapper

• These wrappers need Java 8 installed on the computer you are running them on (contact your local IT if you need help)
• They utilise the ATERM download command
• All you needs to do is execute the script (on Unix systems you can make it executable chmod +x <my script> or use the command source to execute it).
• The script will then fetch the ATERM Jar file and use it to download your data - it will be held in a temporary directory that is deleted after it finishes.
• By default the data go into the current working directory, but you can optionally specify the output directory as the first argument. The person that provisioned the shareable link will have decided how many parallel threads to use (typically at least 2 and not more than 4). You can change that by editing the script if you want.
• If the download fails, e.g. your network drops out, then you can restart. The application will skip files it has already downloaded.

Shell Wrapper

• In these scripts, each asset is downloaded from Mediaflux with one line of the script per asset.
• They use pre-installed tools like curl and wget on Unix systems and powershell on Windows systems, so Java is not required.
• These kinds of scripts cannot download data in parallel.
• If the process fails, e.g. the your network drops out, then you can restart. The application will skip files it has already downloaded

Running downloads in the background

nohup

When launching scripts (be they shell or ATERM wrappers) that potentially need to run for long periods of time (for downloading a lot of data), we recommend that for Unix systems, you preface with the nohup command (ignores terminal hangups) and run in the background (then you can log out if you want). The log will be stored in a file called nohup.out

nohup myscript &

screen/tmux

Screen and tmux are utilities that allow you run interactive terminals within a wrapper that can be detached and reattached.  This allows you to start the download, detach the screen/tmux session, disconnect from the host, then come back at a later time, reconnect to the host, and reattach the interactive dowload session.  For more information see:

• GNU Screen User Manual
• tmux wiki

Research Computing Services will have assisted you to deploy a script, which, when executed, can generate an indirect shareable link and email it to a user. To use the script, you will need to access your computer's command-line interface.

Provisioning Indirect Shareable Links

If you have been provided with a script for generating Indirect Shareable Links, here is how to get started generating the links:

Windows

• Start a Command Prompt terminal window by pressing the Start button and entering cmd

• In the Command Prompt window, navigate to the directory where your script was installed, e.g.:

  cd C:\Mediaflux
  

  or

  cd C:\Users\nkilleen\Mediaflux
  

• You can check you are in the right place with the command dir which will list the contents of that directory
• Execute the script and supply the correct arguments (the name and value of the parameters that control what the script does). The script name may be specialised to you when we deploy it - it's the first part that changes from facility to something for you like mcfp or x617lab in the example)

• To list all available arguments:

  facility-download-shell-script-url-create.cmd --help
  

• To create a sharable link:

  facility-download-shell-script-url-create.cmd --expire-days <number of days before link expires>  --email <email address of link recipient> <namespace to create link for>
  

• Example where facility has been named x617lab on deployment and the child namespace neils_data in the project proj-test-1128.2.3 is provisioned with a link:

  x617lab-download-shell-script-url-create.cmd --expire-days 30 --email nkilleen@unimelb.edu.au proj-test-1128.2.3/neils_data