...
We have collected a set of examples that we hope are useful for those transferring data into or out of a remote endpoint to Mediaflux. Please let us know if there is a cloud service that we have missed.
Mediaflux SFTP
Copying data to Mediaflux over SFTP
Using Mediaflux through the SFTP protocol as an rclone remote. You can also use the SMB protocol to connect to Mediaflux (see below), but SFTP is likely a better choice due to it being encrypted. SFTP has also been in rclone for longer so may be more mature.
If you choose to store your mediaflux password, as in the example below, we recommend that you set a password on your rclone configuration file (see Securing your configuration file, above).
Configuring rclone to use Mediaflux as an SFTP remote
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
rwh@thinkpad:~$ rclone config
e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> n
name> mediaflux-sftp
Type of storage to configure.
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value
[snip]
27 / SSH/SFTP Connection
\ "sftp"
[snip]
Storage> 27
** See help for sftp backend at: https://rclone.org/sftp/ **
SSH host to connect to
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value
1 / Connect to example.com
\ "example.com"
host> mediaflux.researchsoftware.unimelb.edu.au
SSH username, leave blank for current username, rwh
Enter a string value. Press Enter for the default ("").
user> unimelb:mfuser
SSH port, leave blank to use default (22)
Enter a string value. Press Enter for the default ("").
port>
SSH password, leave blank to use ssh-agent.
y) Yes type in my own password
g) Generate random password
n) No leave this optional password blank (default)
y/g/n> y
Enter the password:
password: (your university password)
Confirm the password:
password: (your university password)
Raw PEM-encoded private key, If specified, will override key_file parameter.
Enter a string value. Press Enter for the default ("").
key_pem>
Path to PEM-encoded private key file, leave blank or set key-use-agent to use ssh-agent.
Leading `~` will be expanded in the file name as will environment variables such as `${RCLONE_CONFIG_DIR}`.
Enter a string value. Press Enter for the default ("").
key_file>
The passphrase to decrypt the PEM-encoded private key file.
Only PEM encrypted key files (old OpenSSH format) are supported. Encrypted keys
in the new OpenSSH format can't be used.
y) Yes type in my own password
g) Generate random password
n) No leave this optional password blank (default)
y/g/n>
When set forces the usage of the ssh-agent.
When key-file is also set, the ".pub" file of the specified key-file is read and only the associated key is
requested from the ssh-agent. This allows to avoid `Too many authentication failures for *username*` errors
when the ssh-agent contains many keys.
Enter a boolean value (true or false). Press Enter for the default ("false").
key_use_agent>
Enable the use of insecure ciphers and key exchange methods.
This enables the use of the following insecure ciphers and key exchange methods:
- aes128-cbc
- aes192-cbc
- aes256-cbc
- 3des-cbc
- diffie-hellman-group-exchange-sha256
- diffie-hellman-group-exchange-sha1
Those algorithms are insecure and may allow plaintext data to be recovered by an attacker.
Enter a boolean value (true or false). Press Enter for the default ("false").
Choose a number from below, or type in your own value
1 / Use default Cipher list.
\ "false"
2 / Enables the use of the aes128-cbc cipher and diffie-hellman-group-exchange-sha256, diffie-hellman-group-exchange-sha1 key exchange.
\ "true"
use_insecure_cipher>
Disable the execution of SSH commands to determine if remote file hashing is available.
Leave blank or set to false to enable hashing (recommended), set to true to disable hashing.
Enter a boolean value (true or false). Press Enter for the default ("false").
disable_hashcheck>
Edit advanced config? (y/n)
y) Yes
n) No (default)
y/n>
Remote config
--------------------
[mediaflux-sftp]
host = mediaflux.researchsoftware.unimelb.edu.au
user = unimelb:rhutton
--------------------
y) Yes this is OK (default)
e) Edit this remote
d) Delete this remote
y/e/d>
Current remotes:
Name Type
==== ====
ceph-s3 s3
cloudstor webdav
dropbox dropbox
google-drive drive
mediaflux-sftp sftp
swift swift
e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> q |
Copying data to Mediaflux over SFTP
An example transferring files from the local system to Mediaflux and then checking the upload.
(We've used the -P option, to provide real-time copy progress updates).
...
| Code Block | ||||
|---|---|---|---|---|
| ||||
rwh@thinkpad:~/wd$ rclone -P copy mediaflux-sftp:/Volumes/proj-demonstration-1128.4.15/mediaflux-test cloudstor:/mediaflux-test Enter configuration password: password: Transferred: 63.503M / 63.503 MBytes, 100%, 1.150 MBytes/s, ETA 0s Transferred: 117 / 117, 100% |
Mediaflux SMB
Using Mediaflux through the SFTP protocol as an rclone remote. You can also use the SMB protocol to connect to Mediaflux (see above).
Note that in order to access Mediaflux SMB, you will need to be on the university network or connected with a university VPN.
If you choose to store your mediaflux password, as in the example below, we recommend that you set a password on your rclone configuration file (see Securing your configuration file, above).
Configuring rclone to use Mediaflux as an SMB remote
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
rwh@thinkpad:~$ rclone config
e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> n
Enter name for new remote.
name> mediaflux-smb
Option Storage.
Type of storage to configure.
Choose a number from below, or type in your own value.
[snip]
35 / SMB / CIFS
\ (smb)
[snip]
Storage> smb
Option host.
SMB server hostname to connect to.
E.g. "example.com".
Enter a value.
host> mediaflux.researchsoftware.unimelb.edu.au
Option user.
SMB username.
Enter a string value. Press Enter for the default (rwh).
user> mfuser
Option port.
SMB port number.
Enter a signed integer. Press Enter for the default (445).
port>
Option pass.
SMB password.
Choose an alternative below. Press Enter for the default (n).
y) Yes, type in my own password
g) Generate random password
n) No, leave this optional password blank (default)
y/g/n> y
Enter the password:
password: (your university password)
Confirm the password:
password: (your university password)
Option domain.
Domain name for NTLM authentication.
Enter a string value. Press Enter for the default (WORKGROUP).
domain> unimelb
Edit advanced config?
y) Yes
n) No (default)
y/n> n
Configuration complete.
Options:
- type: smb
- host: mediaflux.researchsoftware.unimelb.edu.au
- user: mfuser
- pass: *** ENCRYPTED ***
- domain: unimelb
Keep this "mediaflux-smb" remote?
y) Yes this is OK (default)
e) Edit this remote
d) Delete this remote
y/e/d> y
Current remotes:
Name Type
==== ====
mediaflux-smb smb
e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> q |
Copying data to Mediaflux over SMB
| Code Block |
|---|
rwh@thinkpad:~$ rclone copy ./mediaflux-test mediaflux-smb:proj-demonstration-1128.4.15/mediaflux-test |
Copying data from Mediaflux over SMB
| Code Block |
|---|
rwh@thinkpad:~$ rclone copy mediaflux-smb:proj-demonstration-1128.4.15/mediaflux-test ./mediaflux-test |
Google Drive
Documentation for configuring a Google Drive remote
...