Don’t risk data loss. Back up your valuable data from the Linux command line. We’ll be using the rsync command for this, and we’ve even found some nice optional graphical interfaces for it.
rsync is the Best for Backups
There are many ways to accomplish making a backup copy of your files. We wanted to show you a robust, flexible, and reliable way to protect your data. We choose rsync because of its well-respected algorithms that calculate the differences between files in the source directory and the target directory. Only the differences between two versions of a file are transferred, not the whole file if that can be avoided.
When this efficiency is paired with its solid track record in performing file copies and directory synchronizations since the mid-1990’s, rsync is a perfect candidate for creating backups from the Linux command line.
Additionally, there are independent software programs that act as a front-end for rsync. They provide graphical user interfaces (GUIs) to rsync which some people may find easier to use.
The simpler and faster it is to make a backup, the more likely you are to do so.
Using rsync With an External Hard Drive
To make a backup copy of your data to an external hard drive, the hard drive must be mounted and accessible to you. If you can write to it, then so can rsync. In this example, an external USB hard drive called SILVERXHD (for “Silver eXternal Hard Drive”) is plugged into the Linux computer. It has been auto-mounted by the operating system.
You will need to know the path to the drive. In GNOME, open the Nautilus file browser and locate the name of the drive in the sidebar.
Hover the mouse pointer over the name of the external drive and a tooltip will show you the path to the drive.
In this example, the tooltip informs us that the mount point for the filesystem on the external drive is “/media/dave/SILVERXHD.”
If your file browser does not do this, browse to the external drive and open a terminal window in that location. Use the pwd command to print the path to the terminal window.
Copying the Contents From the Source Directory
To use rsync to copy the contents of a directory to your backup destination, use the following command. The -r (recursive) option causes rsync to copy all nested subdirectories and their contents. Note that there is forward slash “/” at the end of the word “SILVERXHD,” but it has wrapped round to the next line in the screenshot.
rsync -r /home/dave/Documents/ /media/dave/SILVERXHD/
The file copy takes place, and you are returned to the command line prompt.
If we look at the external USB drive, we see the directories that are in the Documents directory have been copied to the root of the external drive.
ls
Copying the Source Directory and Its Contents
If you had wanted to have the Documents directory and its contents copied to the external drive, remove the “/” from the end of “/home/dave/Documents” in the command line, like this:
rsync -r /home/dave/Documents /media/dave/SILVERXHD/
To avoid confusion, I removed the two previously copied directories from the external drive before this second command was executed.
If we let the second copy complete and take another look at the external drive, we see the Documents directory has been copied over. Its contents are within that directory. They are not in the root of the external drive.
Copying to a Specific Target Directory
To copy to a specific directory on the target hard drive, add the name of the directory to the target path. Let’s suppose we want to copy the contents of the “/home/dave/Documents” directory to a directory called “backups” on the external drive.
We’d do this with the following command.
rsync -r /home/dave/Documents/ /media/dave/SILVERXHD/backups/
Checking on the external drive we can see the backups directory has been created, and within that directory are the contents of the “/home/dave/Documents” directory.
ls
ls backups
Preserving File Ownership and Permissions
Use the -a (archive) option to preserve file attributes such as modification dates, file ownership, access permissions, and more, for copied files, symlinks, and special block files.
rsync -ra /home/dave/Documents/ /media/dave/SILVERXHD/backups/
Using Verbose Mode
The -v (verbose) option forces rsync to list the files as they are being copied.
rsync -rav /home/dave/Documents/ /media/dave/SILVERXHD/backups/
A summary of the backup is presented when the copying is complete.
- Sent: The bytes transferred to the target.
- Received: The bytes received at the host.
- Bytes/sec: is the effective transfer rate.
- Total size: Represents the size of the data that would have been sent if you were not using
rsync. On subsequent runs ofrsyncit will only transfer the file differences. This figure will represent the data that did not have to be transferred. - Speedup: This is the ratio between the amount of data that had to be sent and the total amount of data that there is. If
rsyncneeds to copy all of the files in their entirety (the first time it is run, for example) the speedup will be 1.0. Whenrsyncis next used, it will optimize the transfers. It will only send the differences between the files, not the entire files. FIles with no changes will be ignored. The speedup figure will represent the ratio between the small amount of data that was required to be transferred versus the total size of the files.
Using The Progress Option
The -P (progress) option causes rsync to generate a small progress report after each file is copied.
rsync -raP /home/dave/Documents/ /media/dave/SILVERXHD/backups/
The information provided can be seen between each copied file.
The information provided is:
- Byte size: Data transferred for this file.
- Percentage: Percentage of the file transferred.
- B/s: Data transfer rate.
- Time remaining: Estimated time left to transfer this file.
- xfr#: The number of files transferred so far.
- to-chk: The number of files left to be checked and verified by the optimization algorithms.
Adding More Speed
To speed up transfers, use the -z (compression) option. This compresses the file in transfer, but the file is stored uncompressed in the target directory.
The compression option will not yield significant benefits for transfers involving many small files. For collections of larger files, it can reduce the transfer time in a meaningful way.
We’re also using the --partial option here. rsync will delete partially transferred files caused by network glitches or other interruptions. The --partial option forces rsync to leave the partially transferred files on the target. The net time rsync runs it will not have to re-transfer the portions of the partially transferred files.
Note that you might not want to use this option if there is a risk that someone will mistake the partially transferred files for completely transferred files.
rsync -ravz --partial /home/dave/Documents/ /media/dave/SILVERXHD/backups/
In our example, the benefits are marginal.
The speedup ratio has improved but by two-hundredths of a percent! In a real-world scenario, your speed improvements will be more impressive.
Using rsync Over A Network
So far we’ve been targetting an external USB drive. To use a network location as the target for the backup, use the path to that location on the command line. There is a network attached storage device (NAS) on the network that this article was researched on.
We can use the same trick we used earlier to identify the path to the NAS, by hovering the mouse over the connection to that device in Nautilus.
There are no special options to backup across a network; these are all options we have already used.
rsync -ravz --partial /home/dave/Documents/ /media/dave/NAS/dave/backups/
There is no difference in the format of the output.
Not surprisingly, there is a significant improvement in the Bytes/sec figure.
If we run rsync once again, we can see that there are no files to transfer because there have been no changes, but there are still some bytes transferred back and forth. This is the amount of data that needs to be transferred to compare the file list on the target with the file list on the source.
The speedup ratio is an order of magnitude better in this instance. In practice, your performance ratios will be somewhere between our two pseudo-artificial readings.
Using rsync Over SSH
rsync supports backing up across an SSH connection. We need to provide the user account name and the SSH location on the command line. We’re using a network name here, but you can also use an IP address.
Note the “:” between the SSH connection details and the start of the network path on the remote target.
rsync -ravz --partial /home/dave/Documents/ dave@sulaco.local:/home/dave/Backups/
You will be asked for the password of the user account on the remote machine. This isn’t your password on the source machine.
The backup will complete as usual. The throughput isn’t as fast as a regular network connection, because of the encryption and decryption that takes place in the secure shell connection.
Automating Your Backups
We can easily create automated backups by adding entries to your crontab file.
crontab -e
We’ll set up an automated backup to run each day at 04:30 (if the computer is on at that time, of course). The syntax for the rsync command doesn’t change at all.
Ctrl+O will write your changes to the file, and Ctrl+X will close the nano editor.
Putting a Friendly Face on Rsync
People who are less comfortable with the command line can use one of a number of programs that put a graphical user interface (GUI) on rsync. Two good examples are luckyBackup and Grsync. Both of these programs allow many of the rsync options to be selected through the user interface.
The Grsync program concentrates on being a visual wrapper for rync. It provides easy access to the rsync options and adds only a limited set of new functionality.
The luckyBackup program is much more than a simple wrapper for rsync. It is a backup program that uses rsync behind the scenes. For example, luckyBackup can make multiple “snapshots” of your backup. You can then “roll back” to the versions of the files in any of the snapshots.
To install Grsync
To install Grsync in Ubuntu, use this command:
sudo apt install grsync
To install Grsync in Fedora, use this command:
sudo dnf install grsync
To install Grsync in Manaro use this command:
sudo pacman -Syu grsync
To Install luckyBackup
To install luckyBackup in Ubuntu, use the following command:
sudo apt install luckybackup
To install luckyBackup in Fedora use the following command:
sudo dnf install luckybackup
In Manjaro you must install luckyBackup from the Arch User Repository (AUR). You can do this with the pamac package manager.
Don’t Risk It, Back Up Your Data Often
Backups are absolutely vital. Back up frequently, back up to many locations, and back up to different media. Once it is set up, rsync can do all of that for you.
