Background

Rclone is effectively an alternative to using rsync but it streamlines interfacing with cloud providers e.g backblaze or AWS S3.

With rclone, there are 3 modes of use:

  1. Mounting the remote drive - use file system to update/delete data
  2. rclone copy - Copy files from source to dest, skipping identical files. This one is equivilent to vanilla rsync.
  3. rclone sync - Note, this will delete files in dest if not in src. Be careful with this one.

Caveat to option mounting option

File systems expect things to be 100% reliable, whereas cloud storage systems are a long way from 100% reliable. The rclone sync/copy commands cope with this with lots of retries. However rclone mount can’t use retries in the same way without making local copies of the uploads. Look at the VFS File Caching for solutions to make mount more reliable.

Increasing transfer speed

--transfers 16 --checkers 32

Boost transfer speed by running the transfer with increased number of processes. It seems like there is a ratio that needs to be respected here between number of transfers and checkers.

Reminder

Always test your backups. Having data in the cloud is wonderful, but if you haven’t verified it’s really there you could be in for a nasty surprise when you need it!