Server Backups with rsync

If you do not backup your server regularly, you have almost no way of restoring lost files. As a result, backup copies should always play an important role in your server planning. The rsync synchronization tool/protocol is one of the many backup solutions available.

What is rsync, and what is it used for?

The name rsync refers to a network protocol for file synchronization developed in 1996 by Andrew Tridgell (among others), who headed up the SAMBA project. In addition to the protocol, there is a tool of the same name that facilitates communication over the protocol. The GPL-licensed application can be used free of charge and transfers files from a source directory to the required target directory – locally or via a public network. The file size and creation time of all files in the source and target folders are compared during syncing, so that each sync process only copies changed files. For this reason, the rsync backup process is called an incremental backup variant, which offers the advantage that backups are fast and take up little storage space.

rsync is available for all standard UNIX operating systems like OS/2, Linux or macOS. Using the API wrapper Cygwin, it can also be used on Microsoft Windows.

Typical use scenarios for rsync include data backups and generation of mirror servers (complete server images) as well as data synchronization in companies with various locations and weak data connections. In particular, the latter function is almost exclusively performed by modern cloud technologies today – also because weak data connections are becoming increasingly rare.

Overview of the most important rsync options

The outstanding feature of rsync backups is their efficiency. The incremental approach minimizes network utilization, which is an advantage for large files in particular – regardless of whether the changes to the files are major or minimal. As a command line tool, rsync is controlled via the Terminal by default – or via the Command Prompt, if you use rsync on a Windows device. The entries have the following syntax:

rsync -(-)Options Source path Target path

As a source and target path, you specify the respective start directory and the directory where you want rsync to save the backup copy. You can define individual settings for your rsync backups using the options, which are either abbreviated to letters or written out. The table below summarizes the most important parameters that can be combined with one another:

Option

Function

-r, --recursive

rsync backup takes all sub-directories into account

-u, --update

Instruction to skip files newer in the target directory than in the source directory

-c, --checksum

Distinguishes source and target files based on checksums

-l, --links

Symbolic links are copied as such (and not as files)

-p, --perms

File permissions are retained

-g, --group

Group file permissions are retained

-t, --times

File time stamps (last change) are retained

-o, --owner

File owners are retained (only if administrators)

-D, --devices

Device data is retained

-z, --compress

Automatic compression of the transferred files

--compress-level=NUM

Determines the compression level; values (“NUM”) between 0 (no compression) and 9 (maximum compression) are possible

-v, --verbose

More comprehensive details during the backup processes

-q, --quiet

Hide all details on the backup process (except error messages)

-a, --archive

Archival mode used as standard mode and identical to the combination of options -rlptgoD

-n, --dry-run

Test run in which no actual changes are made

-h, --help

Auxiliary menu (can only be used without indicating source and target directories or other arguments)

--bwlimit=KBPS

Restrict bandwidth (kilobytes per seconds); e.g. --bwlimit=30 (limit of 30 kbit/s)

--exclude=SAMPLE

Exclude a pattern from synchronization; e.g. --exclude sample folder (the “Sample folder” folder is not synchronized.)

--delete

Delete all files that are in the target directory but not in the source directory

--progress

Show the duration of the rsync backups and the transfer speed

--list-only

List files instead of a backup

--stats

Comprehensive report on the transferred data (number, size)

--max-size=SIZE

Define a maximum file size; e.g. --max-size=10MB (only files with a size up to 10 MB are transferred.)

--ignore-errors

Prevent cancellation of the backup process in the event of an error

Setting up an rsync backup on Linux servers

To use rsync on Linux operating systems, install the protocol in the package of the same name and create your backups using Terminal commands. Alternatively you can use applications like Back In Time, rsnapshot (for regular automated backups) or Unison to control the backup process via a graphical user interface. In the example below using Ubuntu, we show you the most important steps to set up backup processes using rsync:

rsync is installed by default under Ubuntu. If this is not the case, install it using the following command.

sudo apt-get install rsync

When rsync is installed, use the required Terminal commands to specify the source and target directories and the backup options. For example, you can run the standard mode (“Archive”) as follows:

rsync -a Source directory Target directory
Tip

It is advisable to use the test run (-n) and check the accuracy of the parameters and directories specified; inaccurate specifications can result in data losses in a worst case scenario. If some files are not copied as required during the process, this is often due to missing access rights. In cases like this, try running the command as an administrator by prefacing it with the “sudo” parameter.

5 popular rsync backup scenarios

Once you know the fundamental commands, rsync is an outstanding tool for copying files and making backups. You can either test and apply individual command setups or use proven combinations of the available rsync parameters. For example, the following rsync backup scenarios are particularly popular:

Standard backup with archival mode

The archival mode, which copies all files from the source directory to the target directory (including all sub-directories) and retains all authorizations, time stamps and device data, is the ideal and simplest solution in many cases as it combines various options in a single parameter. If you combine the mode with the –v parameter, you will also receive comprehensive status information during the backup process.

rsync -av Source directory Target directory

Identical copy of the source directory

Not only can rsync transfer files from A to B, it can also make identical copies of folders or entire directories. After standard archival, the files that were in the target folder before rsync, but are not in the source folder, are deleted.

rsync -av --delete Source directory Target directory

Backup excluding files of a specific format

If you want to run a backup excluding files in a specific format, you can use the --exclude command to do so. The parameter allows you to define an individual character pattern, that rsync uses as an indicator to ignore a file. The following sample code excludes text files .txt.

rsync -av --exclude'*.txt' Source directory Target directory

Backing up files with a defined minimum or maximum size

If you don’t want a specific character pattern, but the file size to influence exclusion of specific files in an rsync backup, you can use the --max-size and --min-size parameters. If you use the following command, only files of at least 10 MB and at most 100 MB are copied:

rsync -av --min-size=10MB --max-size=100MB Source directory Target directory

Backup including character format conversion

You may have to convert files into a different character format in the target directory – for example if you want to transfer data from a Mac to a Linux server. As standard, Apple devices use UTF8-MAC, which is not available on Linux systems and would cause problems with special characters and umlauts. The --iconv option lets you easily adapt the character encoding as part of the rsync backup process (from UTF8-MAC to UTF8 in the example):

rsync -av --iconv=UTF8-MAC,UTF8 Source directory Target directory

We’re all in this together. IONOS is
#HereForYou
For your business. When times are tough. When it matters most.
Keep your business going or start one with your own online store.
3 months free
Online Store
Get started in the world of eCommerce free for 3 months.