Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

Version 1 Next »

Introduction

Welcome to the step-by-step tutorial on how to use the rsync command line with your Mediaflux project at the University of Melbourne (UoM). This guide will help you understand the basic usage and various options of rsync command you can use to access your UoM Mediaflux projects for efficient file synchronization and transfer.

Please note, this is a special access method offered by the UoM Mediaflux service and not a standard Mediaflux function.

Currently EXPERIMENTAL and on a USE AT OWN RISK basis, please verify data integrity after using and before deleting other copies of your data.

Prerequisites

  • A UoM Mediaflux project

  • Access to UoM VPN

  • Access to a terminal or command prompt

  • Installed rsync on your system (ensure the rsync protocol version is 30 or later)

  • OPTIONAL: Below steps are only necessary when using rsync on UoM Mediaflux Test server:

    • Add your public key to the UoM Test Mediaflux server

    • Connect to RCS VPN

    • Set up port 2001 for forwarding with this command on your computer and keep that terminal window open throughout your rsync session:

      • $ ssh -L 2001:localhost:6600  mflux@mediaflux-test.researchsoftware.unimelb.edu.au  -p 22022

    • Ensure the port 2001 on your computer is not being used by any other application, if so you can replace with another free port number greater than 2001, e.g., using port 3001:

      • $ ssh -L 3001:localhost:6600  mflux@mediaflux-test.researchsoftware.unimelb.edu.au  -p 22022

Example 1: Upload Files From Your Local Computer Into Your Mediaflux Project

$ rsync -rv -e 'ssh -p 6600 -l unimelb:YOUR_UOM_USERNAME' YOUR_LOCAL_DIRECTORY/ mediaflux-test.researchsoftware.unimelb.edu.au::YOUR_MEDIAFLUX_PROJECT/FOLDER/

$ rsync -rv -e 'ssh -p 6600 -l unimelb:smithj' /Downloads/files/ mediaflux-test.researchsoftware.unimelb.edu.au::proj-test-1234.5.6/dir01/folder01/

Example 2: Download Files From Your Mediaflux Project to Your Local Computer

$ rsync -rv -e 'ssh -p 6600 -l unimelb:YOUR_UOM_USERNAME' mediaflux-test.researchsoftware.unimelb.edu.au::YOUR_MEDIAFLUX_PROJECT/FOLDER/ YOUR_LOCAL_DIRECTORY/

$ rsync -rv -e 'ssh -p 6600 -l unimelb:smithj' mediaflux-test.researchsoftware.unimelb.edu.au::proj-test-1234.5.6/dir01/folder01/ /Downloads/files/

Example 3: Using "Exclude" & "Include" Directories and Files

Exclude and/or include specific files or directories during synchronisation:

  • For include (need to be paired with exclude option): $ rsync -rv --include "FILTER_CONDITION" --exclude="*" -e 'ssh -p 6600 -l unimelb:YOUR_UOM_USERNAME' YOUR_LOCAL_DIRECTORY/ mediaflux-test.researchsoftware.unimelb.edu.au::YOUR_MEDIAFLUX_PROJECT/FOLDER/

$ rsync -rv --include "*.zip" --exclude "*" -e 'ssh -p 6600 -l unimelb:smithj' /Downloads/files/ mediaflux-test.researchsoftware.unimelb.edu.au::proj-test-1234.5.6/dir01/folder01/

  • For exclude: $ rsync -rv --exclude "FILTER_CONDITION" -e 'ssh -p 6600 -l unimelb:YOUR_UOM_USERNAME' YOUR_LOCAL_DIRECTORY/ mediaflux-test.researchsoftware.unimelb.edu.au::YOUR_MEDIAFLUX_PROJECT/FOLDER/

$ rsync -rv --exclude "*.zip"  -e 'ssh -p 6600 -l unimelb:smithj' /Downloads/files/ mediaflux-test.researchsoftware.unimelb.edu.au::proj-test-1234.5.6/dir01/folder01/

Example 4: Delete Directories/Files in the Destination Directory

Deletes directories / files in the destination directory if they no longer exist in the source directory:

$ rsync -rv --delete -e 'ssh -p 6600 -l unimelb:YOUR_UOM_USERNAME' SOURCE_DIRECTORY/ DESTINATION_DIRECTORY/

$ rsync -rv --delete -e 'ssh -p 6600 -l unimelb:smithj' /Downloads/files/ mediaflux-test.researchsoftware.unimelb.edu.au::proj-test-1234.5.6/dir01/folder01/

Example 5: Only show changes that would be made(doesn't do them) Recommended to do before every transfer

The -n flag previews the changes that would be made without actually doing them. We recommend you preview your changes with this flag before actually doing the upload/download to help prevent data loss.

$ rsync -rv -n  -e 'ssh -p 6600 -l unimelb:YOUR_UOM_USERNAME' SOURCE_DIRECTORY/ DESTINATION_DIRECTORY/ 

$ rsync -rv -n -e 'ssh -p 6600 -l unimelb:smithj' /Downloads/files/ mediaflux-test.researchsoftware.unimelb.edu.au::proj-test-1234.5.6/dir01/folder01/

Conclusion

Congratulations! You've learned the basics of using rsync to synchronise your data to your UoM Mediaflux project. If you have any question, please feel free to contact us via http://go.unimelb.edu.au/or96 .

  • No labels