Jumlins TechBlog

The coding hell and the daily IT of Niklas Jumlin

July 28th, 2010 by Dennis

Having to synchronize some data among a Debian linux cluster, i settled on using csync2 for the job.
Here’s a short guide to set it up.

We are assuming two machines here, 01.cluster and 02.cluster. The 01.cluster is gonna be our “master” in this setup.

First on both machines install csync2 by executing:

apt-get install csync2

And if you don’t have inetd installed, do:

apt-get install openbsd-inetd

On each node we now need to generate a certificate for csync2 to communicate. We do it by the following commands, When asked to set a challenge password leave it empty, and leave the common name empty.:

openssl genrsa -out /etc/csync2_ssl_key.pem 1024
openssl req -new -key /etc/csync2_ssl_key.pem -out /etc/csync2_ssl_cert.csr
openssl x509 -req -days 600 -in /etc/csync2_ssl_cert.csr -signkey /etc/csync2_ssl_key.pem -out /etc/csync2_ssl_cert.pem

On the master we need to generate a preshared key for the nodes to communicate with:

csync2 -k /etc/csync2_ssl_cert.key

Note: You might experience a somewhat hang on the command, this is because the /dev/random pool isn’t filling up fast enough. To remedy this open a secondary connection to the server, and jab a bit around. Look in files, download some big file whatever makes the /dev/random entropy fill up

Now we need to set up the configuration file on the master, which is in /etc/csync2.cfg.

# Csync2 configuration example
group cluster
	host 01.cluster;
	host (02.cluster); # Slave host

	key /etc/csync2_ssl_cert.key;

	include /var/www;
	exclude /var/www/sessions;

	auto none;

Note: the hostname of all the machines needs to match the output of the hostname command.
The parentheses around 02.cluster is to make the synchronization only work in one direction. Now we need to copy the csync.cfg and the csync2_ssl_cert.key to the slave server(s).

After all this do a /etc/init.d/openbsd-inetd restart on all machines.
And run csync -x on the master to synchronize data on the slaves. Note data on the slave(s) WILL be overwritten/deleted

A logical step now would be to run csync -x from within a cron job. Which i will leave for a later post.

3 Responses to “ Setting up cluster synchronization with csync2 ”

  1. Ham3 says:

    Nice! This one is also good: http://docs.homelinux.org/doku.php?id=csync2_cluster_synchronization_tool , it works with xinetd

  2. Nick says:

    Thanks Dennis, this assisted me some-what; For those who do not have the luxury running apt-get and are on a CentOS or another RPM Based Build check out the automated scripts I came across over at: http://www.cloudedify.com/synchronising-files-in-cloud-with-csync2/

  3. Jan Day says:

    Typically, all other things being good, this will clear up a 611. If not, start focusing on a hostname resolution issue.

Leave a Reply