How to mirror FreeBSD with CVSup

Date published: July 22nd, 2011
Posted in FreeBSD | No Comments »

The following was done with FreeBSD 8.2-RELEASE.

CVSup is a highly efficient way of distributing files. It works similar to rsync, but was specially designed for use with CVS repositories.

Requirements:

• 5.4 GB disk space ( an additional 10-20% more free space is recommended)
• >= 2GB RAM
• A decent CPU as CVSup can be CPU intensive
• A fast disk subsystem (RAID highly recommended)
• Good network connection

Building your own CVSup mirror is made easy by the port net/cvsup-mirror.

# cd /usr/ports/net/cvsup-mirror

# make install clean

You will be prompted for a master site for you updates. Choose a mirror from the CVSup Sites list .

Follow the instructions  (Note: If you would also like to mirror the WWW data, this is a very easy way to do it.  Select ‘y’ when prompted):

I am going to ask you a few questions so that I can set up your
FreeBSD mirror configuration.  Every question has a [default]
answer.  To accept the default, just press ENTER.

At this point, I am just gathering information.  I will not touch
your system until you type “make install”.

Master site for your updates [cvsup-master.freebsd.org]? cvsup9.us.FreeBSD.org
How many hours between updates of your files [1]? [Enter]

Now you must decide which sets of files you wish to make available
from your mirror site.  You can choose any combination, and you
can put each set anywhere you want to on your disks.  Although each
set is optional, we strongly encourage every mirror site to carry
at least the main source repository.

Do you wish to mirror the main source repository [y]? [Enter]
Where would you like to put it [/home/ncvs]? [Enter]
Do you wish to mirror the installed World Wide Web data [y]? n
Do you wish to mirror the GNATS bug tracking database [y]? n
Do you wish to mirror the mailing list archive [y]? n

Now, a few questions so that I can set up your CVSup server properly.

For security reasons, both the CVSup client and server should run
under their own unique user and group IDs.  These IDs should have no
special access privileges.  Normally, the user:group “cvsupin:cvsupin”
is used for the client and “cvsup:cvsup” is used for the server, but
you can choose other names if you wish.  At “make install” time, I
will create the users and groups, if they don’t already exist.

Use unique user and group IDs for these.  Do not use “nobody”,
“nonroot”, or “nogroup”.

Unique unprivileged user ID for running the client [cvsupin]? [Enter]
Unique unprivileged group ID for running the client [cvsupin]? [Enter]
Unique unprivileged user ID for running the server [cvsup]? [Enter]
Unique unprivileged group ID for running the server [cvsup]? [Enter]

The CVSup server does its logging via syslog.  At “make install”
time, I will set up the logging for you, if necessary.  I will use
the “!program” feature of syslog to keep your CVSup log messages
separate from the messages of your other daemons.

Syslog facility for the server log [daemon]? [Enter]

You can control the load on your machine by limiting the number of
clients that the CVSup server will serve at once.  CVSup won’t load
your network especially heavily, but it is more CPU and disk
intensive than most other file server software.

Maximum simultaneous client connections [8]? 10

Later, it will prompt you again with more questions:

You need a group “cvsup”.
Would you like me to create it [y]? [Enter]

You need a user “cvsup”.
Would you like me to create it [y]? [Enter]

You need a group “cvsupin”.
Would you like me to create it [y]? [Enter]

You need a user “cvsupin”.
Would you like me to create it [y]? [Enter]

Would you like me to create cvsupin’s home directory (/home/cvsupin) [y]?

The port should now be installed and ready for configuration.

First, comment out the line added to /etc/crontab. You can adjust the time for the update to script to run if you desire:

# vi /etc/crontab

There should be a line that looks similar to this:

#6    *    *    *    *    root    /usr/local/etc/cvsup/update.sh

If you would like to further restrict access you can configure your cvsupd.access file:

# vi /usr/local/etc/cvsup/cvsupd.access

Below is what should be there originally:

-0.0.0.0/0      10      # Limit total connections
-0.0.0.0/0/32   1       # Allow only 1 connection from each host
+0.0.0.0/0              # If we reach this rule, we let the client in

Now you should be ready run your first update. This will take some time as it downloads the entire repository.

# /usr/local/etc/cvsup/update.sh

Once that is done, uncomment the line in /etc/crontab.

Add a line to /etc/rc.conf for cvsupd:

# vi /etc/rc.conf

cvsupd_enable=”YES”

Then start cvsupd:

# /usr/local/etc/rc.d/cvsupd start

Your CVSup mirror should now be working!

Other notes:

CVSup requires incoming connections on port 5999 so add a firewall rule if necessary.

If you encounter trouble, check /var/log/cvsup.log and /var/log/cvsupd.log .

— Rob Lampe