Tag Archives: rsync

Using rsync to Synchronise Two Directories Securely

rsync is a utility that provides fast incremental file transfer. This allows us to keep directories and files synchronised between servers whilst only transferring the minimum required data. This is often employed during code deployment scenarios from one environment to another.

It is preferable to use the -e option to specify a remote shell to use, namely ssh. This will ensure that the file transfers are encrypted and performed over SSH.

In our deployment scenario there are two servers, venus and earth. Both are webservers, and have content at /var/www/example.com/htdocs. venus is our test server, and earth is production. We’d like to deploy our content from venus to earth. This deployment could be automated (think continuous integration), or even scripted for simple deployments, but for the purposes of this tutorial we’ll just manually issue the commands as required. This is being performed on CentOS 6.4 with SELinux enabled.

I’ll start off by creating some simple “content” on venus:

On earth, I’ll just create the empty content directory:

Create a user/group on both servers that will be used for deployment. deploy@venus will allow interactive logins to push content to deploy@earth. deploy@earth will only allow the specific remote-side rsync command to run. This will be defined later in ~/.ssh/authorized_keys. For now, just create the users and groups:

Set ownership on our deployment directories. Do this on both servers:

It is obvious that any permissions set here should not interfere with the way your web server actually serves its content. You may need to set up ample group permissions for the deploy group, then add the webserver user to that group.

The first step in setting up our file syncronisation is to configure password-less SSH login from deploy@venus to deploy@earth. Having no passphrase on the SSH key we’re about to create is the reason for limiting command access via ~/.ssh/authorized_keys.

Start with key generation as deploy@venus:

As you can see, a 2048-bit RSA key has been generated.

Next, create ~/.ssh on earth, and copy the public key to deploy@earth:.ssh/authorized_keys:

On earth, we now need to fix SELinux contexts and permissions on deploy@earth:

We can now check that password-less SSH is working:

Next, we can use rsync to perform a dry-run of our deployment with the --dry-run option. Specifying the -v option twice shows the remote command that will be executed:

The remote command can be seen above - rsync --server -vvnlogDtprCze.iLs . /var/www/example.com/htdocs. We can populate deploy@earth:.ssh/authorized_keys with this command against deploy@venus‘s public key in the file, limiting deploy@venus to ONLY be able to execute the remote rsync command. We need to drop one of the -v‘s and -n (--dry-run) from the remote command too. Modify deploy@earth:.ssh/authorized_keys as appropriate:

Note that I also add the from= configuration parameter so that only connections from venus are permitted. Now, the password-less SSH configuration is locked down so that any connections from deploy@venus can only run the remote rsync command.

Test from deploy@venus:

Running the command again should not transfer any changes (as there haven’t been any):

And we can now verify that earth has the files: