Rsnapshot - A backup Utility

There are many ways to backup Servers. One of the better ways to accomplish this is using rsnapshot. Rsnapshot is nothing but a filesystem snapshot utility for backing up local and remote systems.

Rsnapshot is written in Perl, and depends on rsync. With ssh access, it is possible to backup remote servers.

Why Rsnapshot?

We have many familiar ways to generate full backup and copy it to another server which includes ftp, scp and all. But now the question that comes to your mind will be “Why should I use rsnapshot ? or What is so special about this tool?”
Listed below are the major advantages of rsnapshot, which make you feel that it is worth using.

• Using rsync and hard links, it is possible to keep multiple, full backups instantly available. The disk space required is just a little more than the space of one full backup, plus incrementals. This comes as a criteria when your drive is lacking enough free space to accommodate 3 copies of backup.
• Depending on your configuration, it is quite possible to set up in just a few minutes. Files can be restored by the users who own them, without the root user getting involved.
• There are no tapes to change, so once it’s set up, you may never need to think about it again.
• Rsnapshot takes advantage of hard links (multiple directory pointers to the same data) to give the appearance of multiple full backups yet requires only enough disk capacity to store the complete data set plus any changed files. Thus we have the illusion of full hourly, daily, weekly, and monthly backups without having the physical space to hold that many copies.
• Another benefit is that rsync is cross-platform, so it isn’t constrained to *nix systems.

Installation of rsnapshot

Rsnapshot can be installed in a few minutes of time. It requires the following prerequisites.
1) Perl
2) rsync

i). Download the latest source tarball from http://www.rsnapshot.org/downloads.html

ii). Untar the source code package

tar xzvf rsnapshot-1.3.0.tar.gz

iii). Change to the source directory.

cd rsnapshot-1.3.0/

iv). Select the directory to install the files. Usually the files are placed in /usr/local directory and the config files in the /etc directory.

v). Run the configure script

./configure --sysconfdir=/etc

vi). Install the program by

make install

Now rsnapshot is installed under /usr/local, with the config file in /etc.

Configuration

A sample copy of the rsnapshot config file is provided with the package. We need to just copy the file.

cp /etc/rsnapshot.conf.default /etc/rsnapshot.conf

The directives that need to be configured in rsnapshot.conf

1. snapshot_root
This is the snapshot root directory which holds the file system snapshots.

snapshot_root   /.snapshots/

2. Modify the path to the various programs like rm for removing files, rsync, ssh etc. Usually not much modification is needed here.

3. Specify thebackup intervals.

interval        hourly  6

interval        daily   7

interval        weekly  4

In this example, rsnapshot is taken every four hours, or six times a day (these are the hourly intervals), 7 times a week, 4 times a month. Thus it covers the whole month (4 weeks)

4. Select verbose level, loglevel and logfile path. Typical values used are

verbose         2

loglevel        3

logfile        /var/log/rsnapshot

5. Next main configuration to be done is BACKUP POINTS / SCRIPTS

This is the section where you tell rsnapshot what files you actually want to back up. You put a “backup” parameter first, followed by the full path to the directory or network path you’re backing up. The third column is the relative path you want to back up to inside the snapshot root.

# LOCALHOST

backup   /home/          localhost/

backup   /etc          localhost/

backup  /usr/local/      localhost/

For example, take case 1, where the parameter is “backup”. We are backing up the /home partition of the server on the rsnapshot root of our server itself. Thus with this backup parameter, a backup of /home is created in /root/.snapshots

In addition to full paths on the same server, we can also backup filesystems on remote systems using rsync over ssh. To get this done,

a) The ssh daemon must be running on the remote server.

b) You must have access to the account you specify the remote machine, in this case the root user on remote server.

c) You must have key-based logins enabled for the root user at remote server, without passphrases.

# EXAMPLE.COM

backup root@example.com:/home/ example.com/
backup root@example.com:/home/ example.com/ exclude=mtab,exclude=core

In example 1, /home partion of remote server example.com is backed up to the server where snapshot is running. In example 2, similarly /etc partion is backed up, excluding mtab and core directories.
This backup parameter is commonly used in live servers for backup configuration.

Allowing remote logins with no passphrase is a security risk that may or may not be acceptable in your situation. Make sure you guard access to the backup server very carefully! If we wish to perform backup as another user, we could specify the other user instead of root for the source (i.e. user@example.com).

Now with this we have completed the basic configuration of rsnapshot.

6. Testing configuration file.

rsnapshot configtest

If all is well, it should say Syntax OK, or else the errors are shown. We have to use tabs in the config file and not spaces.

7. Test run of rsnapshot

rsnapshot -t hourly

This tells rsnapshot to simulate an “hourly” backup. It will print out the commands it will perform when it runs for real.

Automating rsnapshot using cron job

Edit root’s crontab by command

crontab -e

Add the following entries,
0 */4 * * * /usr/local/bin/rsnapshot hourly
30 23 * * * /usr/local/bin/rsnapshot daily

Cron should be timed in a way that the hourly backup is finished before performing the daily backup.

How it works

All backups are stored in the snapshot directory. New directories inside the snapshot root are created when rsnapshot hourly and weekly are run. Thus when rsnapshot hourly is run 6 times, the directories with name hourly.0, hourly.1, ….hourly.5 are created. Similarly when rsnapshot weekly is run, 7 new directories are created namely weekly.0, weekly.1, weekly.2 till weekly.6.

Each subsequent time rsnapshot is run with the hourly command, it will rotate the hourly.X directories, and then “copy” the contents of the hourly.0 directory (using hard links) into hourly.1.

When rsnapshot daily is run, it will rotate all the daily.X directories, then copy the contents of hourly.5 into daily.0.

hourly.0 will always contain the most recent snapshot, and daily.6 will always contain a snapshot from a week ago.
Unless the files change between snapshots, the “full” backups are really just multiple hard links to the same files.
That is how rsnapshot saves disk space. If the file changes at any point, the next backup will unlink the hard link in hourly.0, and replace it with a brand new file.

When weekly, monthly, and yearly intervals defined (in that order), the weekly ones would get updated directly from the filesystem, the monthly ones would get updated from weekly, and the yearly ones would get updated from monthly.

Conclusion

When the aforesaid instructions are followed, rsnapshot installation is quite simple and is very efficient in performing automatic backups of your system. The amount of disk space taken up will be equal to one full backup, plus an additional copy of every file that is changed.

Reference:

http://www.rsnapshot.org/

rsync examples from command line.

To copy all the files in /home/lokams/* of remote host to local directory /backup/home/lokams/# rsync -e ssh -avz --delete --exclude dir/* --delete-excluded --stats user@remotehost:/home/lokams/ /backup/home/lokams/

To copy the directory /home/lokams of remote host to local directory /backup/home. Directory "lokams" will get created in /backup/home.# rsync -e ssh -avz --delete --exclude-from=~/.rsync/excludeFile --delete-excluded --stats user@remotehost:/home/lokams /backup/home/

To ignore the permissions errors while copying to remote windows/mac systems, use "-rlt" option.# rsync -e ssh -avz --delete --exclude dir/* --delete-excluded --stats -rlt user@remotehost:/home/lokams/ /backup/home/lokams/

rsync behaves differently if the source directory has a trailing slash. Study and learn the difference between the following two commands before moving on. Use the -n option to rsync when testing to preview what would happen.
$ rsync -n -av /tmp .
$ rsync -n -av /tmp/ .

More rsync examples:
--------------------------

# rsync -crogpvltz -e ssh --exclude "bin" --exclude "ifany" --delete --delete-after --bwlimit 20 root@10.144.17.1:/apps/uae/ /apps/uae > /apps/uae/logs/sync.log

# rsync -avz -e ssh root@remotehost:/backup/reptest /backtmp/reptest/

# rsync -avz -e ssh /logstage/archive/DXBSEC/NODE0000/ db2inst1@10.20.202.236:/db2/archivelog/security/logstage/retrieve/DXBSEC/NODE0000/

# rsync -e ssh -avz --timeout=999 --delete --exclude dir-or-file-to-exclude --delete-excluded --stats -rlt user@remotehost:/home/lokams/

rsync useful options:
-------------------------
-a, --archive archive mode, equivalent to -rlptgoD
-n, --dry-run show what would have been transferred ( Preview mode )

-c - always checksum
-r - recursive into directories
-o - preserve owner
-g - preserve group
-p - preserve permissions
-t - preserve times
-v - verbose
-l - copy symlinks as symlinks
-z - compress file data
-P - show progress during transfer
-q - quite (decrease verbosity)
-e - specify the remote shell
-b - make backup
-R - Use relative path names
-u - skip files that are newer on the receiver

--stats give some file-transfer stats
--timeout=TIME set I/O timeout in seconds

--backup-dir - make backups into this directory
--bwlimit=KBPS - limit I/O bandwidth, KBytes per second
--delete - delete files that don't exist on the sending side
--delete-after - receiver deletes after transferring, not before
--daemon run as an rsync daemon
--address=ADDRESS bind to the specified address
--exclude=PATTERN exclude files matching PATTERN
--exclude-from=FILE exclude patterns listed in FILE
--include=PATTERN don't exclude files matching PATTERN
--include-from=FILE don't exclude patterns listed in FILE
--min-size=SIZE don't transfer any file smaller than SIZE
--max-size=SIZE don't transfer any file larger than SIZE


# --numeric-ids:
Tells rsync to not map user and group id numbers local user and group names
# --delete:
Makes server copy an exact copy of the source by removing any files that have been removed on the remote machine

Applying filter in rsync

As we know rsync command is very helpful in taking backup. Sometime need to backup of some folder by including some sub folders and excluding some of them. by an e synchronizing with remote system we can mention folders which to be included and which to be excluded as command line options. Let us understand it by an example, suppose one need to take take backup of / by excluding /home and by including /home/user2 .


Following command help out

root# rsync -av --filter=+home --filter=+/home/user2 --filter=-/home/* / user1@remotehost:/folder

Without the delete option per directory options are only relevant on sending side, to exclude the merge files themselves without affecting the transfer. The -e modifier can do this easily. For example

root# rsync -av --filter=':e .mp3' host:src/dir /dst

rsync in Batch mode

Batch mode can be used to apply same set of changes to many identical system. Suppose same of tree of a filesystem has to be replicated a number of hosts. Updates in one tree has be propagated to many hosts. In order to achieve this rsync can be used in batch mode. The --write-batch option tell rsync client to store in a batch file all the information needed to repeat this information against the other.

To apply the recorded change rsync has to be run with --read-batch option by specifying the same batch file. One thing also need to keep in mind that one bash file also get created in this process. Let us put this in an example command

root#rsync --write-batch=lists -a host:/source/dir /testdir

root#rsync --read-batch=lists -a /datadir

What is archive mode in rsync ?

As you know when i apecify -a option with rsync that means i want to apply rsync command in archive mode. Archive mode considered as most suitable for taking backup. Putting -a basically a shortcut of bunch of switches which includes follwings

-r recursive

-l copy symlinks as symlinks

-p preserve permission

-t preserve modification time

-g preserve group

-o preserve owner

-D device specials

And which exclude followings

-H preserve hard link

-A preserve ACLs

-X preserve extended attribute

So how concise and smart to use -a instead of mentioning above given bunch of switches.

rsync command with examples

rsync: Remote Sync.

Description: rsync utility is used to synchronize the files and directories from one location to another in an effective way. Backup location could be on local server or on remote server.

Features:
(*).One of the main feature of rsync is that it transfers only the changed block to the destination, instead of sending the whole file.

Syntax:
rsync options source destination (or)
rsync [OPTION]... SRC [SRC]... [USER@]HOST:DEST (or)
rsync [OPTION]... [USER@]HOST:SRC [DEST]

a - Archive same -rptgolD, [Its enough to copy entire folder of file even with out changing file permission]
-r - Recursive [Copy entire directive]
-p, --perms preserve(Unalter) permissions
-t, --times preserve times
-o, --owner preserve owner (super-user only)
-g, --group preserve group
-l, --links copy symlinks as symlinks
-D same as --devices --specials
--devices preserve device files (super-user only)
--specials preserve special files

-z, --compress - Compress file data during the transfer.

-v - verbose, Explain what happen in view

-u - Do Not Overwrite the Modified Files at the Destination [if the file in the destination in modified and if u use -u option in rsync, the modified file wont be overwritten].

--rsh=COMMAND -- specify the remote shell to use. (e.g) --rsh='ssh -p1055' to login to remote system.

--progress - View the rsync Progress during Transfer

--stats - give some file-transfer stats

--delete - This tells rsync to delete extraneous files from the receiving side (ones that aren’t on the sending side), but only for the directories that are being synchronized.

--exclude - This option is used to exclude some certain files

--max-size=SIZE - This tells rsync to avoid transferring any file that is larger than the specified SIZE

--max-size=SIZE - This tells rsync to avoid transferring any file that is smaller than the specified SIZE


Examples:

1.rsync Command to copy only Files from a folder

#rsync -avz doc/ Desktop/welcome/
#ls Desktop/welcome/
taglist.txt

===>taglist.txt is the file present in the folder doc

2.rsync command to copy entire folder to the destination

#rsync -avz doc Desktop/welcome/
#ls Desktop/welcome/
doc

===>doc is the folder we copied.

3.rsync command with --progress option

#rsync -avz --progress doc Desktop/welcome/
building file list ...
2 files to consider
doc/
doc/taglist.txt
69366 100% 11.63MB/s 0:00:00 (xfer#1, to-check=0/2)

sent 18509 bytes received 48 bytes 37114.00 bytes/sec
total size is 69366 speedup is 3.74

===> --progress option gives you a progress meter of data send to the destination.

4.rsync command with --progress and --stats options

#rsync -avz --progress --stats doc Desktop/welcome/
building file list ...
2 files to consider
doc/
doc/taglist.txt
69366 100% 11.63MB/s 0:00:00 (xfer#1, to-check=0/2)

Number of files: 2
Number of files transferred: 1
Total file size: 69366 bytes
Total transferred file size: 69366 bytes
Literal data: 69366 bytes
Matched data: 0 bytes
File list size: 83
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 18509
Total bytes received: 48

sent 18509 bytes received 48 bytes 37114.00 bytes/sec
total size is 69366 speedup is 3.74

===> --stats option tells you a status information of file transfered.

5.rsync command with --delete option

#cd Desktop/welcome/
#cat > hi
#hello
#rsync -avz --delete doc Desktop/welcome/
building file list ... done
deleting doc/hi
doc/

sent 105 bytes received 26 bytes 262.00 bytes/sec
total size is 69366 speedup is 529.51

===> --delete command is used to delete files in the destination if the source doesn't have the file.

#cd Desktop/welcome/
#ls
taglist.txt

6.rsync command with --exclude option.

#cd doc
#cat hello
this is hello file

#rsync -avz --exclude doc/hi doc Desktop/welcome/
building file list ... done
doc/

sent 105 bytes received 26 bytes 262.00 bytes/sec
total size is 69366 speedup is 529.51

#cd Desktop/welcome/
#ls
taglist.txt

7.rsync command to transfer file from Source system to Destination system.

#rsync -avz --progress --stats --delete --rsh='ssh -Xp36985' doc rajm@192.168.1.5:Desktop/

===> The above command is used to copy file from source system to Desktop of destination system .

8.rsync command to transfer file from destination system to source system.


#rsync -avz --progress --stats --rsh='ssh -Xp36985' rajm@192.168.1.5:Desktop/doc .

===> The above command is used to copy files from Desktop system to source of destination system .

Rsync tips

1 Server to Local

1.1 With Interactive Password Entry Prompt

rsync -apvz \
   --recursive \
   --links \
   --perms \
   --times \
   --devices \
   --specials \
   --partial \
   --progress \
   --delete \
   --timeout=300 \
   --exclude='readwrite' \
   --exclude='linux-2.6.30.7'  \
   --exclude='arnold' \
   --exclude='kernels' \
   -e  "ssh " \
       suresh@192.168.0.100:/mnt/boot-images /FC10/boot-images

1.2 With Shared ssh-keys (private)  

rsync -apvz \
   --recursive \
   --links \
   --perms \
   --times \
   --devices \
   --specials \
   --partial \
   --progress \
   --delete \
   --timeout=300 \
   --exclude='readwrite' \
   --exclude='linux-2.6.30.7'  \
   --exclude='arnold' \
   --exclude='kernels' \
   -e  "ssh -i /root/.ssh/id_dsa_rsyncusr -o PreferredAuthentications=publickey -o StrictHostKeyChecking=no" \
     suresh@192.168.0.100:/mnt/boot-images /FC10/boot-images 

2 Local to Server 

rsync -apvz \
   --recursive \
   --links \
   --perms \
   --times \
   --devices \
   --specials \
   --partial \
   --progress \
   --delete \
   --timeout=300 \
   --exclude='readwrite' \
   --exclude='linux-2.6.30.7'  \
   --exclude='arnold' \
   --exclude='kernels' \
   -e  "ssh " \
      /var/www/html suresh@192.168.0.100:/mnt/backup

3 Two-Way-Sync 

rsync -apvz \
   --update \
   --recursive \
   --links \
   --perms \
   --times \
   --devices \
   --specials \
   --partial \
   --progress \
   --delete \
   --timeout=300 \
   --exclude='readwrite' \
   --exclude='linux-2.6.30.7'  \
   --exclude='arnold' \
   --exclude='kernels' \
   -e  "ssh " \
      /var/www/html suresh@192.168.0.100:/mnt/backup

How to take Linux backups powered by Rsync

RSync backups data and does it very clean and well. Rsync only transfers those data that have been modified and changed so that the destination host has an exact replica from the source host. Rysnc is a command line backup tool that handles data transfers in an effective and secure manner like any other known commercial backup softwares around. Rync blends in and integrates flawlessly with linux shell commands combined linux I/O redirections.

Man Rsync:

Rsync uses a reliable algorithm to bring remote and host files into sync very quickly. Rsync is fast because it just sends the differences in the files over the network instead of sending the complete files. Rsync is often used as a very powerful mirroring process or just as a more capable replacement for the rcp command. The rsync remote-update protocol allows rsync to transfer just the differences between two sets of files across the network connection, using an efficient checksum-search algorithm.

INSTALLATION:
~~~~~~~~~~~~~~~~

Rsync installation is installed by default System Tool group installation. If rsync is not available from command line and from rpm database, you can install from yum repo using yum like so

# yum -y install rsync

To verify that rsync has been installed successfully

# rpm -qa rsync

There are two different approach on establishing rsync communication between two hosts.

First is by using a remote-shell program such as ssh or rsh. If you want to use this approach, it is required that openssh server or an active ssh connection is present from both sending and receiving host. Openssh installation and configuration would not be covered from this entry. This entry would assume that ssh daemon service is currently configured and listening properly to assigned host port.

Secondly, is by using rsync listening INETD service directly.

This entry hope to cover both of them two worlds.

Rsync usage from command line terminal
======================================

Transfer and/or update files from local host into a remote host using rsync
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Basic rsync argument is to specify file glob, a source and a destination folder. Destination can be a local host or a remote host. A basic example to transfer from local to remote would be

# rsync -t *.mp3 remoteusername@remotehost:remote_destination_folder

The above would transfer, using rsync, all *.mp3 files from current local directory into remote_destination_folder of remotehost using remoteusername for authentication. A destination host can be locally or remote host and can be specified as hostname or IP address.

Legend:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-t preserve time of selected files
*.mp3 selected files to be transferred
remoteusername bash enabled user account from destination host
remotehost destination host where remoteusername is allowed to have access
remote_destination_folder destination folder from destination host owned and accessible
by remoteusername
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

For recursive traversing of directory folders for transferring data from current host to another host using rsync, this would be like so

# rsync -avrzt /var/www myuser@server1:/var/backup/
Legends:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-a archived mode enabled
-v verbosed mode enabled
-t preserve time stamp
-z compression transfer enabled
-r resursive mode enabled
/var/www selected folder or files glob source location
myuser user account from destination host
server1 destination host
/var/backup destination folder
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The above command would issues rsync with verbose mode enabled, compression enabled, arhived mode of rsync data transfer/update. Rsync transfers files and folders from /var/www (including the www folder) into /var/backup folder of server1 host using myuser as login credentials. The /var/backup from destination host is owned or writeable by myuser . All rsync files are created with file ownership and permission owned by myuser having 600 file mode. The transfer would be done preserving symbolic links, devices, attributes, permissions and ownerships of files.

# rsync -avz /var/www/ myuser@server1:/var/backup/www

Appending / from selected file glob like the above command issues the same rsync command with same argument. The only difference is that source folder is not created .

Transfer files from remote host to local host using rsync
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

# rsync -avrzt remoteusername@remotehost:remote_source_folder destination_folder
# rsync -avrz myuser@server1:/var/backup/www /var/www/

To transfer multiple files with different file extension from local host to remote host using rsync with file glob would be

# rsync -avrz file1.mp3 *.doc myuser@server1:/home/folder/location

To transfer multiple files from multiple directory sources using rsync would be

# rsync -avz `find /etc -name *.conf` user@remotehost:/user/folder

Linux command line tools when combined with each other creates another set of power tools. The above command searches all *.conf files from /etc folder and transfer them to /usr/folder of remotehost via rsync and using user the user login name. This can be handy if you like to backup specific pattern of files from your system or user accounts individual address books, files like that.

To transfer multiple files from multiple directory with some file excemptions using rsync and grep from local to remote would be like so:

# rsync -avz `find /etc -name *.conf | grep -v yum.conf` user@remotehost:/user/folder

Alternatively,

# rsync -avz $(find /etc -name *.conf | grep -v yum.conf) user@remotehost:/user/folder

and for multiple file rsync with multiple file from multiple source location with multiple file transfer exceptions would be like so

# rsync -avz `find /etc -name *.conf | grep -v ‘yum.conf\|xorg.conf’` user@remotehost:/user/folder

To transfer of all your back up files ending in .tar file extension from any location would be like so

# rsync -avz `find / -name *.tar` user@remotehost:/user/folder

You will noticed that rsync can transfer data at high speed rate using rsync algorithm specially if those files and folders to be transferred are existing already from remote host.

To transfer multiple file(s) with multiple exclusions using rsync would be like so:

# rsync -avz * user@remotehost:/location –exclude=*.php –exclude=*.mp3

To transfer files in batch mode based from file lists using rsync would be

Assuming listing.txt is created with contents like below

# cat listing.txt
~~~~~~~~~~~~~~~~~~~~~
files0123.txt
files0124.txt
files6124.txt
files6126.txt
…
snipped
…
files32126.txt
~~~~~~~~~~~~~~~~~~~~~

and feeding the above file to rsync as batch mode input like so

# rsync -avzt –files-from=listing.txt user@remotehost

:/destination/

To rsync multiple folder source location using rsync would be like so

# rsync -avz –files-from=/home1 /home2 user@remotehost:/destination/

If both location contains abc.txt, the latest abc.txt would be transferred to remote host. If /home1/www exist and /home2/www exist, the files from both source kicatuib would be merged into /destination/www .

Fire up two new terminal windows, and from the first window, establish ssh connection from local to remote rsync destination. Then from the second windows try to issue these rsync command. You will notice that rsync never ask any password any more since there is an existing ssh connection with local host to remote host.

More rsync arguments
====================

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-W transfer the whole files without considering any update changes from existing destination file or folder
–progress show progress bar and/or percentage
-4 prefers IPv4
-6 prefers IPv6
–bwlimit=KBPS execute rsync with bandwidth limit rate in KBPS
-h display a more human readable screen output
–log-file=FILE dumps file activity into a file
–ignore-existing ignores already existing copy from destination host
–max-size tells rsync to avoid transferring files larger than specified size like
–max-size=10m avoid file transfer with 10MB in filesize
–port tells rsync to connect to rsync server with a different port, default is 873
–stats tells more file transfer info and rsync algorithm stats on the fly
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

At this point, be reminded to kick the black ninja box to refresh possible monitor related eye strains.

Using Rsync in Daemon Xinetd Mode
=================================

Edit /etc/xinetd.d/rsync and modify

disable = yes
to
disable = no

To run rsync in deamon mode via xinetd, make sure you have similar lines from your /etc/xinetd.d/rsync file like shown below:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
service rsync
{
disable = no
socket_type = stream
wait = no
user = root
server = /usr/bin/rsync
server_args = –daemon –config=/etc/rsync/rsyncd.conf -v
log_on_failure += USERID
}
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Create /etc/rsync/rsyncd.conf as a default conf file for rsync in daemon xinetd mode.

Sample rsyncd.conf file
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
uid = rsync-user
gid = rsync-user
use chroot = yes
read only = yes
pid file = /var/run/rsyncd.pid

# access list, edit the IP network block to suit your needs
hosts allow=192.168.0.0/255.255.0.0 192.168.1.0/255.255.255.0
# deny anything else
hosts deny=*

# limit connections
max connections = 5
#greeting file
motd file = /etc/rsync/rsyncd.motd
#log file
log file = /var/log/rsync.log

#rsync shared folder
[myrsync]
#make your UID/GUID above owns the below folder and files
#all files and folder would be seen by rsync client
path=/home/rsync-user/rsync-folder
comment = Linux Rsync Server
exclude = *.mp3

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Create rsync greeting MOTD file like so

# cat /etc/rsync/rsyncd.motd
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Welcome to my Rsync Server!
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Restart xinetd service for the changes to take effect like so

# service xinetd restart

Verify that rsync is running as xinetd daemon service mode using one tool like ss

# ss -a | grep rsync
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
LISTEN 0 0 *:rsync *:*
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Rsync uses port 873 as its default port for xinetd service. Make sure it is also open from your current firewall settings. A sample firewall rule for opeing rsync from your firewall would be like so

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-A RH-Firewall-1-INPUT -m state –state NEW -m tcp -p tcp –dport 873 -j ACCEPT
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# service iptables restart

If you wish to have a passwordless rsync, you need to refer to passwordless/passphraseless ssh from one of last month’s entry.

Rsync in daemon mode via command line
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

An alternative to run rsync in daemon mode is by specifying it from command line. Like so

# rsync –daemon –address host-IP-address –config=/etc/rsync/rsyncd.conf -v –port=873

Legend:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
–daemon enables rsync to be run as daemon mode from terminal
–port specifies port number to use
–config specified rsync conf file
host-IP-address IP address where to bind rsync from
-v enables verbose mode
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Listing out files and folder from rsync server
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To list out the files from rsync server, simply point your rsync client to the host running rsync in daemon mode or server mode. To test your rsync server from a client linux host would be like so

# rsync host-IP-address::

Alternatively,

# rsync rsync://host-IP-address/myrsync
# rsync rsync://host-IP-address/myrsync/folder1

From the above rsync command, you should be seeing files from /home/rsync-user/rsync-folder folder from rsync server you defined from /etc/rsync/rsyncd.conf.

Syncing File and Folders from Rsync Server
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To sync and download files from rsync server.

# rsync rsync://host-IP-address/myrsync/folder1 .

The above command downloads files including the whole folder1 folder from rsync server and saves it to current directory

# rsync rsync://host-IP-address/myrsync/folder1/ .

The above command downloads only files from folder1 to current folder location

Rsync Log Monitoring
~~~~~~~~~~~~~~~~~~~~

Monitoring rsync server messages for any errors or system messages would be like so

# tail -f /var/log/rsync.log

Final Note:

Rsync can also be used for source code control and management for delivering a centralized and distributed sync source codes from rsync server to group of programmers or source developers among departments, more like a CVS approach.

Using rsync linux command provides many abilities and benefits. To name a few, mirror an entire harddisk or partitions, folders and files, entire domain websites, mail spools for replica servers, user’s home folder, a mirror FTP site and much more.

Backup like the enterprise way, use RSync.