• Howdy! Welcome to our community of more than 130.000 members devoted to web hosting. This is a great place to get special offers from web hosts and post your own requests or ads. To start posting sign up here. Cheers! /Peo, FreeWebSpace.net
managed wordpress hosting

rsync - A quick HOWTO guide

justjustincase

New Member
rsync - A quick HOWTO guide


What is rsync?
rsync is a cross platform program that is used for file transfers. rsync updates only files that have changed, so you do not need to retransfer all your data whenever you run rsync. It only mirrors new/changed files, and it can also delete files from the webhost that have been deleted on the content server. In addition to that it can preserve permissions and ownerships of mirrored files and directories. rsync is great for transferring your files to a webhost and for doing backups.

This guide only covers one way directional transfers. This guide also only covers linux to linux transfers. While rsync is cross platform, someone else can write up guides for the other OSes.

If you have root access, just install rsync using your normal distro package manager. If your distro doesn't have a compiled version, you can download the source from here:
http://www.samba.org/rsync/

If you have root access you have the option of running rsync as an xinetd service. By default rsync runs on TCP port 873 so be sure to open this port on your firewall.

If you don't have root access rsync can be run from a different port, just make sure you have access to the rsync binary(install it into your account if you need to) and that the port is open on the firewall.


Example Scenario
Your doing your web development on a machine you have access to, this will be the rsync server. You have a web host which is hosting your files, this will be the rsync client.


Setting up the rsync server:
If your going to run the rsync server as an xinetd service make your changes in the /etc/rsyncd.conf file otherwise follow these directions. Assuming your doing your web development under a normal user account called "user" then:
0. Create a new directory under your user account to hold the rsync configuration files:
mkdir /home/user/rsync
1. Change directory to this new rsync directory:
2. Create a rsyncd.motd file. This is a banner file which is displayed everytime an rsync client connects to your server. Even something as simple as:
echo `whoami`@`hostname` > rsyncd.motd
This stores your username @ your hostname as the banner.
3. Create a rsyncd.secrets file. In this file you create a list of new usernames and passwords(they have nothing to do with the existing accounts already on your machine) which are allowed to connect to your rsync server. For example:
rsyncuser:tranfers5
4. Create a rsyncd.conf file. Lets assume all your web pages are stored in your home account under /home/user/webdev then an example rsyncd.conf would look like:
motd file = /home/user/rsync/rsyncd.motd
log file = /home/user/rsync/rsyncd.log
pid file = /home/user/rsync/rsyncd.pid

[webdev]
comment = webdev directory
path = /home/user/webdev
use chroot = no
lock file = /home/user/rsync/rsync.lock
read only = yes
list = yes
uid = user
gid = user
auth users = rsyncuser
secrets file = /home/user/rsync/rsyncd.secrets
strict modes = false
hosts allow = 1.2.3.4
transfer logging = yes
timeout = 600
1.2.3.4 needs to be the IP address of your web host, i.e. the rsync client.
5. Find out the rsync server's Public IP address. Either the command or website should be able to help you:
For this example let's assume the Public IP address is:
18.19.251.43
6. Pick a TCP port above 1023 and open it up on your firewall if you need to. For this example I've chosen
TCP port 11213
7. Time to run the rsync server:
/usr/bin/rsync --daemon --address=18.19.251.43 --config=/home/user/rsync/rsyncd.conf --port=11213
8. Watch the log file, this is needed when you are setting up rsync for the first time as any errors will show up here:
tail -f /home/user/rsync/rsyncd.log
At the moment you should only see this line:
2009/06/07 04:30:54 [30841] rsyncd version 2.6.3 starting, listening on port 11213


Setting up the rsync client:
To use the rsync client at your web host, you will either need an rsync web frontend(ask your hoster to install one, there is a live demo of one here: http://backupmondemo.host56.com/) or have SSH access. The following is assuming you have SSH access.
0. Most web hosters require you to store your files in a public_html directory. Run the rsync binary like this:
[account1@webhoster ~]$ time rsync --verbose --archive --compress --stats --progress rsync://rsyncuser@18.19.251.43:11213/hlds /home/account1/public_html
1. If everything works you will see the banner you made earlier(the rsyncd.motd file) as well as a prompt for a password. Enter in the password for the rsyncuser user as stored in the rsyncd.secrets file. For this example enter in this password:
tranfers5
If something is wrong with your config files or you entered the wrong password you will see:
@ERROR
otherwise you will see the files being transferred and at the end some statistics.


To stop the rsync server:
0. From the web development machine, i.e. the rsync server enter in this command:
[user@webdevmachine ~]$ kill `cat /home/user/rsync/rsyncd.pid`


References:
This is only a quick guide to rsync. Read the manuals for rsync to get a greater understanding of the config file format as well as rsync:
[user@webdevmachine ~]$ man rsyncd.conf
[user@webdevmachine ~]$ man rsync


Notes:
Its also possible to do two way synchronisation with rsync.
It should also be possible to setup an rsync server at the web hosting end and use the rsync client on your web development machine and still move files from the web development machine to the web hosting. This might be a guide for another day.
 
Back
Top