HOWTO: Setting up CruiseControl.rb on Slicehost

Continuous Integration is a key tool for collborative development, and CruiseControl.rb is the tool of choice for many Ruby and Rails teams, including us at Logical Reality.

Unfortunately, setting up CC.rb for a team can be a relatively frustrating experience: this guide (the first of a series of HOWTOs by LRD) will walk you through every step of setting up a team instance of CruiseControl.rb on a low-cost server from Slicehost.

Step 1: Lease a Ubuntu Slicehost account

I recommend a 384 slice or a 512 slice, as 256MB or RAM is pretty light for anything involving a Rails application.   Our CI server runs on a 512 slice; if you are running it on a smaller slice please let us know how it performs.

I used Ubuntu 9.10 (Karmic) for this post.

Step 2: Create a working user

Slicehost configures slices with an active root account - definitely a Ubuntu no-no - and no user account. Ick! Let's start by creating a user account with sudo access to do everything from. Log in as root using the information Slicehost sends you, run this (replace 'usename' with whatever name you like) and fill in the information it asks for:

 # adduser username

Then edit /etc/sudoers and add this line to the bottom of the file:

username    ALL=(ALL) ALL

Log out, and log back in as the user you've now configured, to make sure it work.

Step 3: Installing packages and gems

Reset your timezone:

sudo dpkg-reconfigure tzdata

Install a whole bunch of packages you'll want for running Rails applications and hosting CruiseControl:

sudo aptitude install locate emacs git-core ruby build-essential \
libopenssl-ruby ruby1.8-dev irb  apache2 apache2-mpm-prefork \
apache2-prefork-dev sqlite3 rubygems mysql-server mysql-client

Go grab a cup of coffee while those install. The mysql install will ask you to set a root password. Do so, and write it down for later use. When all the installs are done, come back and install the ruby gems you'll be needing:

sudo gem install sqlite3-ruby passenger mysql metric_fu reek roodi

Step 4: Assorted server configuration

Add this line to the bottom of your ~/.profile to put your gems in your path:

PATH="$PATH:/var/lib/gems/1.8/bin/"

And source it:

. ~/.profile

Some assorted config: set up the passenger module for Apache, set your hostname, and make /etc/hosts readable. (For some bizarre reason, /etc/hosts was only readable by root on my slice, and that has a tendency to break things down the road).

sudo /var/lib/gems/1.8/bin/passenger-install-apache2-module
sudo emacs /etc/hostname  # set it to "your.hostname.com"
sudo /bin/hostname -F /etc/hostname
sudo chmod a+r /etc/hosts

Step 5: Configure Passenger and Apache

We'll run CruiseControl.rb with Apache and Passenger. Start by enabling the Passenger module. The command below will walk you through a super-easy configuration:

sudo /var/lib/gems/1.8/bin/passenger-install-apache2-module

When the command completes, it will give you three lines to paste into your apache config, they should look pretty much like these below. Put these lines at the top of /etc/apache2/apache2.conf. I included the hostname I set in the previous step as ServerName.

LoadModule passenger_module /var/lib/gems/1.8/gems/passenger-2.2.8/ext/apache2/mod_passenger.so
PassengerRoot /var/lib/gems/1.8/gems/passenger-2.2.8
PassengerRuby /usr/bin/ruby1.8   
 
ServerName your.hostname.com

To set up the application itself, edit /etc/apache2/sites-available/default to look like this:

<VirtualHost *:80>
        ServerAdmin administrator@your-email-domain.com
        DocumentRoot /u/apps/cruisecontrol/public
        RailsEnv production
        RailsBaseURI /
        ServerName <IP Address from Slicehost>
        ServerAlias your.hostname.com
        SetEnv PATH /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/var/lib/gems/1.8/bin/
 
        ErrorLog /var/log/apache2/error.log
 
        # Possible values include: debug, info, notice, warn, error, crit,
        # alert, emerg.
        LogLevel warn
 
        CustomLog /var/log/apache2/access.log combined
</VirtualHost>

Make a home for the app. (I use /u/apps/ as a convention for apps in apache. Use whatever you like, but make sure your DocumentRoot in your config file above matches.)

sudo mkdir -p /u/apps

Step 6: Download and install CruiseControl.rb

Download cruisecontrol.rb from RubyForge (Check for the current version first; it was 1.4.0 when I installed), and give ownership to the web user www-data:

cd /u/apps
sudo wget http://rubyforge.org/frs/download.php/59598/cruisecontrol-1.4.0.tgz
sudo tar -zxf cruisecontrol-1.4.0.tgz     
sudo mv cruisecontrol-1.4.0 cruisecontrol   
sudo chown -R :www-data cruisecontrol

Give environment.rb to the web user; this prevents an Errno::EACCES accessing environment.rb from Passenger (see discussion at this forum post).

sudo chown www-data:www-data config/environment.rb

Turn off the built-in htaccess, it will break Passenger:

sudo mv public/.htaccess public/.htaccess-disabled
cd config  
sudo cp site_config.rb_example site_config.rb

Step 7: Setting up the user environment

CruiseControl.rb prefers, by default, to put project builds in the running user's ~/.cruise directory. This is unfortunate because the standard user for running Apache, www-data, doesn't have a user directory! There are ways to override this, but I've found that they cause significant problems down the line.

An example of the problem is letting CC.rb check out your source code. If you authenticate access to GitHub or another code repository with SSH, CC.rb — running as www-data — won't be able access your repo since www-data doesn't have a ~/.ssh directory to put the keys in!

After much hacking, I came to the unhappy conclusion that the best solution is simply to let CruiseControl.rb have its way and give user www-data a home directory. Boo, hiss, but here we go:

sudo /etc/init.d/apache2 stop   
sudo usermod -d /home/www-data www-data      
sudo usermod -s /bin/bash www-data
sudo /etc/init.d/apache2 start

If you give www-data standard config files as well, then you can set the PATH so that user www-data can find your gems, and you can set up ssh keys so that CruiseControl.rb can securely check out projects from GitHub or whatever source code repository you're using:

sudo cp -r /etc/skel /home/www-data 
sudo chown www-data:www-data /home/www-data
sudo su www-data               
cd
mkdir ~/.ssh  
cd ~/.ssh
ssh-keygen -t rsa
cat id_rsa.pub

Add this line to the bottom of ~/www-data/.profile:

PATH="$PATH:/var/lib/gems/1.8/bin/"

Re-start Apache:

sudo /etc/init.d/apache2 restart

At this point, you should be able to load CruiseControl.rb in a web browser at the IP address given to you by Slicehost, or at the domain name if you've set up DNS and it's resolving. Congratulations, you have CC.rb up and running! One last thing to configure.

Running CruiseControl.rb will have created a configuration directory . ~www-data/.cruise. You'll want to edit ~www-data/.cruise/site_config.rb to set two options. Uncomment and set appropriate values for this line:

 Configuration.email_from = 'cruisecontrolrb@mydomain.com'
 Configuration.dashboard_url = 'http://my.cruisecontrolrb.host/'

Okay, it's time to get a project installed!

Step 8: Setting up your first project

I'll use Logical Reality's open-source project, Convection, as an example project for CruiseControl.rb. This works best if you run it as user 'www-data'.

The command for adding a new project is really simple:

cd /u/apps/cruisecontrol 
sudo su www-data
./cruise add Convection -r git://github.com/LRDesign/Convection.git -s git

This will set up the build in ~www-data/.cruise/projects/Convection.

Create a test database for the application. For Convection, I'm going to use mysql, and prefix my database name with 'ci' for Continuous Integration.

mysqladmin -u root -p create ci_convection

We don't want to put a functioning database.yml in our GitHub repository, but at the same time we want CruiseControl.rb to be able to build and test the app without help from the user. For all our Rails projects, we use a custom rake task that generates a database.yml from command-line arguments, then rebuilds the database, run the specs, and generate output with metric_fu. For an example of how to do this, look at our integration.rake and ERB database.yml template from Convection.

To configure CruiseControl.rb to run Convection this, we need to add that task to the configuration file for this project. Edit ~www-data/.cruise/projects/Convection/cruise_config.rb so that it looks like this:

 
Project.configure do |project|
 
  # Send email notifications about broken and fixed builds to email1@your.site, email2@your.site (default: send to nobody)
  project.email_notifier.emails = ['sysadmin@lrdesign.com', 'judson@lrdesign.com']
 
  # Set email 'from' field
  project.email_notifier.from = 'sysadmin@lrdesign.com'
 
  # Build the project by invoking rake task 'custom'
  # project.rake_task = 'custom'
 
  # Build the project by invoking shell script "build_my_app.sh". Keep in mind that when the script is invoked,
  # current working directory is <em>[cruise&nbsp;data]</em>/projects/your_project/work, so if you do not keep build_my_app.sh
  # in version control, it should be '../build_my_app.sh' instead
  # project.build_command = 'build_my_app.sh'
  project.build_command = 'rake ci:run[localhost,root,<YOUR_MYSQL_ROOT_PASSWORD>,ci_convection] --trace RAILS_ENV=test'
 
  # Ping Subversion for new revisions every 5 minutes (default: 30 seconds)
  # project.scheduler.polling_interval = 5.minutes
end

Step 9: There is no step nine!

Okay, so it's not the simplest thing in the world to set up. But if you've done everything above correctly, you should have a running server your team can use for continuous integration. If you've included metric_fu in your build task, you should get both test output and a wealth of useful code metrics.

Did this sequence work for you? Did I omit a step or misspell a command? Let me know in comments, and I'll update/correct the post.