HOWTO: Setting up CruiseControl.rb on Slicehost

Continuous Integration is a key tool for collborative development, and CruiseControl.rb is the tool of choice for many Ruby and Rails teams, including us at Logical Reality.

Unfortunately, setting up CC.rb for a team can be a relatively frustrating experience: this guide (the first of a series of HOWTOs by LRD) will walk you through every step of setting up a team instance of CruiseControl.rb on a low-cost server from Slicehost.

Step 1: Lease a Ubuntu Slicehost account

I recommend a 384 slice or a 512 slice, as 256MB or RAM is pretty light for anything involving a Rails application.   Our CI server runs on a 512 slice; if you are running it on a smaller slice please let us know how it performs.

I used Ubuntu 9.10 (Karmic) for this post.

Step 2: Create a working user

Slicehost configures slices with an active root account - definitely a Ubuntu no-no - and no user account. Ick! Let's start by creating a user account with sudo access to do everything from. Log in as root using the information Slicehost sends you, run this (replace 'usename' with whatever name you like) and fill in the information it asks for:

 # adduser username

Then edit /etc/sudoers and add this line to the bottom of the file:

username    ALL=(ALL) ALL

Log out, and log back in as the user you've now configured, to make sure it work.

Step 3: Installing packages and gems

Reset your timezone:

sudo dpkg-reconfigure tzdata

Install a whole bunch of packages you'll want for running Rails applications and hosting CruiseControl:

sudo aptitude install locate emacs git-core ruby build-essential \
libopenssl-ruby ruby1.8-dev irb  apache2 apache2-mpm-prefork \
apache2-prefork-dev sqlite3 rubygems mysql-server mysql-client

Go grab a cup of coffee while those install. The mysql install will ask you to set a root password. Do so, and write it down for later use. When all the installs are done, come back and install the ruby gems you'll be needing:

sudo gem install sqlite3-ruby passenger mysql metric_fu reek roodi

Step 4: Assorted server configuration

Add this line to the bottom of your ~/.profile to put your gems in your path:

PATH="$PATH:/var/lib/gems/1.8/bin/"

And source it:

. ~/.profile

Some assorted config: set up the passenger module for Apache, set your hostname, and make /etc/hosts readable. (For some bizarre reason, /etc/hosts was only readable by root on my slice, and that has a tendency to break things down the road).

sudo /var/lib/gems/1.8/bin/passenger-install-apache2-module
sudo emacs /etc/hostname  # set it to "your.hostname.com"
sudo /bin/hostname -F /etc/hostname
sudo chmod a+r /etc/hosts

Step 5: Configure Passenger and Apache

We'll run CruiseControl.rb with Apache and Passenger. Start by enabling the Passenger module. The command below will walk you through a super-easy configuration:

sudo /var/lib/gems/1.8/bin/passenger-install-apache2-module

When the command completes, it will give you three lines to paste into your apache config, they should look pretty much like these below. Put these lines at the top of /etc/apache2/apache2.conf. I included the hostname I set in the previous step as ServerName.

LoadModule passenger_module /var/lib/gems/1.8/gems/passenger-2.2.8/ext/apache2/mod_passenger.so
PassengerRoot /var/lib/gems/1.8/gems/passenger-2.2.8
PassengerRuby /usr/bin/ruby1.8   
 
ServerName your.hostname.com

To set up the application itself, edit /etc/apache2/sites-available/default to look like this:

<VirtualHost *:80>
        ServerAdmin administrator@your-email-domain.com
        DocumentRoot /u/apps/cruisecontrol/public
        RailsEnv production
        RailsBaseURI /
        ServerName <IP Address from Slicehost>
        ServerAlias your.hostname.com
        SetEnv PATH /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/var/lib/gems/1.8/bin/
 
        ErrorLog /var/log/apache2/error.log
 
        # Possible values include: debug, info, notice, warn, error, crit,
        # alert, emerg.
        LogLevel warn
 
        CustomLog /var/log/apache2/access.log combined
</VirtualHost>

Make a home for the app. (I use /u/apps/ as a convention for apps in apache. Use whatever you like, but make sure your DocumentRoot in your config file above matches.)

sudo mkdir -p /u/apps

Step 6: Download and install CruiseControl.rb

Download cruisecontrol.rb from RubyForge (Check for the current version first; it was 1.4.0 when I installed), and give ownership to the web user www-data:

cd /u/apps
sudo wget http://rubyforge.org/frs/download.php/59598/cruisecontrol-1.4.0.tgz
sudo tar -zxf cruisecontrol-1.4.0.tgz     
sudo mv cruisecontrol-1.4.0 cruisecontrol   
sudo chown -R :www-data cruisecontrol

Give environment.rb to the web user; this prevents an Errno::EACCES accessing environment.rb from Passenger (see discussion at this forum post).

sudo chown www-data:www-data config/environment.rb

Turn off the built-in htaccess, it will break Passenger:

sudo mv public/.htaccess public/.htaccess-disabled
cd config  
sudo cp site_config.rb_example site_config.rb

Step 7: Setting up the user environment

CruiseControl.rb prefers, by default, to put project builds in the running user's ~/.cruise directory. This is unfortunate because the standard user for running Apache, www-data, doesn't have a user directory! There are ways to override this, but I've found that they cause significant problems down the line.

An example of the problem is letting CC.rb check out your source code. If you authenticate access to GitHub or another code repository with SSH, CC.rb — running as www-data — won't be able access your repo since www-data doesn't have a ~/.ssh directory to put the keys in!

After much hacking, I came to the unhappy conclusion that the best solution is simply to let CruiseControl.rb have its way and give user www-data a home directory. Boo, hiss, but here we go:

sudo /etc/init.d/apache2 stop   
sudo usermod -d /home/www-data www-data      
sudo usermod -s /bin/bash www-data
sudo /etc/init.d/apache2 start

If you give www-data standard config files as well, then you can set the PATH so that user www-data can find your gems, and you can set up ssh keys so that CruiseControl.rb can securely check out projects from GitHub or whatever source code repository you're using:

sudo cp -r /etc/skel /home/www-data 
sudo chown www-data:www-data /home/www-data
sudo su www-data               
cd
mkdir ~/.ssh  
cd ~/.ssh
ssh-keygen -t rsa
cat id_rsa.pub

Add this line to the bottom of ~/www-data/.profile:

PATH="$PATH:/var/lib/gems/1.8/bin/"

Re-start Apache:

sudo /etc/init.d/apache2 restart

At this point, you should be able to load CruiseControl.rb in a web browser at the IP address given to you by Slicehost, or at the domain name if you've set up DNS and it's resolving. Congratulations, you have CC.rb up and running! One last thing to configure.

Running CruiseControl.rb will have created a configuration directory . ~www-data/.cruise. You'll want to edit ~www-data/.cruise/site_config.rb to set two options. Uncomment and set appropriate values for this line:

 Configuration.email_from = 'cruisecontrolrb@mydomain.com'
 Configuration.dashboard_url = 'http://my.cruisecontrolrb.host/'

Okay, it's time to get a project installed!

Step 8: Setting up your first project

I'll use Logical Reality's open-source project, Convection, as an example project for CruiseControl.rb. This works best if you run it as user 'www-data'.

The command for adding a new project is really simple:

cd /u/apps/cruisecontrol 
sudo su www-data
./cruise add Convection -r git://github.com/LRDesign/Convection.git -s git

This will set up the build in ~www-data/.cruise/projects/Convection.

Create a test database for the application. For Convection, I'm going to use mysql, and prefix my database name with 'ci' for Continuous Integration.

mysqladmin -u root -p create ci_convection

We don't want to put a functioning database.yml in our GitHub repository, but at the same time we want CruiseControl.rb to be able to build and test the app without help from the user. For all our Rails projects, we use a custom rake task that generates a database.yml from command-line arguments, then rebuilds the database, run the specs, and generate output with metric_fu. For an example of how to do this, look at our integration.rake and ERB database.yml template from Convection.

To configure CruiseControl.rb to run Convection this, we need to add that task to the configuration file for this project. Edit ~www-data/.cruise/projects/Convection/cruise_config.rb so that it looks like this:

 
Project.configure do |project|
 
  # Send email notifications about broken and fixed builds to email1@your.site, email2@your.site (default: send to nobody)
  project.email_notifier.emails = ['sysadmin@lrdesign.com', 'judson@lrdesign.com']
 
  # Set email 'from' field
  project.email_notifier.from = 'sysadmin@lrdesign.com'
 
  # Build the project by invoking rake task 'custom'
  # project.rake_task = 'custom'
 
  # Build the project by invoking shell script "build_my_app.sh". Keep in mind that when the script is invoked,
  # current working directory is <em>[cruise&nbsp;data]</em>/projects/your_project/work, so if you do not keep build_my_app.sh
  # in version control, it should be '../build_my_app.sh' instead
  # project.build_command = 'build_my_app.sh'
  project.build_command = 'rake ci:run[localhost,root,<YOUR_MYSQL_ROOT_PASSWORD>,ci_convection] --trace RAILS_ENV=test'
 
  # Ping Subversion for new revisions every 5 minutes (default: 30 seconds)
  # project.scheduler.polling_interval = 5.minutes
end

Step 9: There is no step nine!

Okay, so it's not the simplest thing in the world to set up. But if you've done everything above correctly, you should have a running server your team can use for continuous integration. If you've included metric_fu in your build task, you should get both test output and a wealth of useful code metrics.

Did this sequence work for you? Did I omit a step or misspell a command? Let me know in comments, and I'll update/correct the post.

New LRDesign logo and site!

We've finished our redesign project, and the first version of the new look is up in time for LA Rubyconf!

There's plenty more to do, but we're very happy to have a refreshed look. In can be hard, as a web development company with active clients, to find time to work on our own website!

RailsTutorial.org launched

The new Ruby on Rails Tutorial book and website by Michael Hartl has launched at RailsTutorial.org.   Hartl is the author of RailsSpace and cofounder of the Insoshi Ruby on Rails social networking platform.

Logical Reality did the logo and layout design work for Rails Tutorial.

Back up after hacking incident

This blog, along with a dozen or so other CMS-driven sites I maintain, was compromised by a hacker recently. I've finally gotten this one back up and am working on the others.

When ‘rake spec’ and ‘spec spec’ produce different results

AKA adventures in class loading.

A couple of days ago I did some significant work in authorization in one of my apps, involving creating a Groups class with an HABTM relationship to Person, so I could assign roles to people a group at a time. It all worked out great, and I pushed the product to GitHub. The next day, my collaborator wrote in that my recent contribution broke 119 specs.

I pulled and retested the code, and everything worked perfectly. WTF? After a bit of investigation, I discovered that the specs worked great when I ran 'autotest' or 'spec spec', but that 119 specs broke when I ran the exact same spec suite with 'rake spec'! Double WTF.

Setting constants at class loading

Ultimately, I tracked the problem down to this line and method, in Person.rb:
class Person << ActiveRecord::Base
ADMIN_GROUP = Group.find_by_name('Admin')
def admin?
groups.include? ADMIN_GROUP
end
end


I consider a person an administrator if they are a member of this group, and I was loading it as a constant at the class level in order to avoid having to query the database again every time Person#admin? is called. This worked just fine for me, both in the application, and every time I ran Person#admin?.

But, remarkably, ADMIN_GROUP does not get initialized correctly when I run the tests via rake. I found this via the ruby debugger, running in this particular spec in spec/models/person_spec.rb:


describe Person do
it "should load an admin user from fixture" do
debugger
people(:admin).should be_admin
end
end


When I run the specs and evaluate Person::ADMIN_USER, I get very different results depending on which spec runner I'm using:

Running 'spec spec/models/person_spec.rb':


[11:17:54 CITAlumni]$ spec spec/models/person_spec.rb
spec/models/person_spec.rb:64
people(:admin).should be_admin
(rdb:1) eval Person::ADMIN_GROUP
#


Running 'rake spec SPEC=spec/models/person_spec.rb':


[11:17:13 CITAlumni (48c51f1...)]$ rake spec SPEC=spec/models/person_spec.rb
(in /Users/evan/Development/Ruby/CITAlumni)
FF.....spec/models/person_spec.rb:64
people(:admin).should be_admin
(rdb:1) eval Person::ADMIN_GROUP
nil


How very interesting ... when I use rake, that constant initializes to nil. At some point, I'll actually get around to figuring out why this is so different when the specs are run via rake. In the meantime, the fix was easy:

The Solution

The fix was just to refactor ADMIN_GROUP as a class method with a memoized instance variable. This will at least limit DB queries for the admin group to one per page load; not quite as good as a single DB hit when the class is first loaded, but still a major improvement over querying for the admin group every time Person#admin? is called. I moved it to the Group class at the same time, which was probably the right place for it in the first place:


#app/models/group.rb:
class Group < ActiveRecord::Base
def self.admin_group
@admin_group ||= self.find_by_name('Admin')
end
end

#app/models/person.rb
class Group < ActiveRecord::Base
def admin?
groups.include? Group.admin_group
end
end


And this worked just fine in all environments, solving the problem with 'rake spec'.

The Moral

Be careful with depending on behavior that occurs only during the loading of classes, as it can be environment-dependent!

If anyone out there with uber Ruby skills knows exactly why running specs via rake prevents that class variable from loading correctly, please enlighten us in comments!

Rails fixture strings that are all numbers

I ran into this one today: If you need to specify a string in a YAML file (fixtures or the like) but that string is all digits, put it in quotes.

The problem YAML file looked like this:

spec/fixtures/people.yml (broken)


one:
funky_database_id: 0000012345

two:
funky_database_id: 0000012346


The trouble with this is that yaml interprets those values as integers, not strings, and Person#funky_database_id is a string column. So Ruby conveniently loads the value as an integer and runs to_s on it before inserting. Worse, because these start with 0, they get translated from octal. So people(:one).funky_database_id comes out "5349". Definitely not what I wanted.

This works as expected:

spec/fixtures/people.yml (fixed)


one:
funky_database_id: "0000012345"

two:
funky_database_id: "0000012346"


New York Times article about UniThrive

One of my clients, UniThrive, was just written up in the New York Times. Go check it out!

An excerpt:

In the photo, the young person’s eyes are brown and kind-looking. She is in need of financial help. A new Web site that brings together the charitable minded and those in need has posted the details of her request.

This is not one of those arrangements where donors can sponsor a needy child or a sorghum farmer in the developing world. The person asking for help is a 21-year-old neurobiology major at Harvard, and she is requesting a loan from Harvard alumni.

Using link_to (or other helper methods) in a controller

This one was a big aggravator to me lately. I have one controller that needs to call link_to and url_for, which are normally helper methods you'd call from a view. However, in this case during certain modifications to a record, I actually need to append user-visible HTML links to a block of HTML stored in that object, or possibly another one.

Specifically, I needed to put annotations in the description of a work order object that said, for example "this work order was escalated from Problem Report 293. This was done in a create action that redirected at the end and never rendered a view, so I really did need to generate that link in the controller. And for consistency with the rest of the application, I wanted to generate the link with link_to(@task).

Now, ActionView::Helpers::UrlHelper is not loaded in a Rails controller, even if you've put helper :all in application.rb (application_controller.rb in newer versions). So, when I tried to use link_to in the controller, I got an error:

NoMethodError: undefined method `link_to' for #
/Users/evan/Development/Ruby/eclipticdb/app/helpers/tasks_helper.rb:64:in `task_link'
/Users/evan/Development/Ruby/eclipticdb/app/controllers/tasks_controller.rb:103:in `escalate'
... etc ...


The first fix - but with a problem

A year ago, I fixed this just by adding include ActionView::Helpers::UrlHelper at the top of that controller. This worked great ... for a while.

Lately, I've been rewriting this application into a RESTful style - it had previously been a controller/action style application. In the process, I started linking things with resource paths and polymorphic paths ... a lot of link_to @task and edit_polymorphic_path(@task) sorts of bits. And these started breaking. I began seeing this mysterious error:

Error:

You have a nil object when you didn't expect it!
The error occurred while evaluating nil.url_for

vendor/rails/actionpack/lib/action_view/helpers/url_helper.rb:71:in `send'
vendor/rails/actionpack/lib/action_view/helpers/url_helper.rb:71:in `url_for'

This one was a real bitch to debug, I have to say. The line in question that was failing in url_helper.rb said this: url = @controller.send(:url_for, options). Clearly, @controller was nil ... which was very bizarre, because I never interact with that instance variable anywhere.

I thrashed around trying to find the cause of this error for quite some time. Eventually I realized that the link_to method was only failing when called from a view in TasksController, and not from any other controller. And then I realized that TasksController was the one where, a year ago, I'd put include ActionView::Helpers::UrlHelper at the top. Somehow, including that helper in the controller was nullifying @controller when those helper method we called from within the view. I removed the include and my polymorphic and resource links all started working again.

Now back to the original problem!

Of course, that then left me back with the problem I'd had a year ago ... needing to use link_to from within the controller and having no way to do it. After a fair bit of googling around I found this post from Neeraj, which had an interesting approach -- but a commenter had suggested a much easier solution:

[sourcecode language='ror']self.class.helpers.link_to[/sourcecode]

I'm not certain where one would find this in the docs, but it does seem to have solved my problem for now. Onward and upward!

New Project Launch: UniThive.org

Where have I been the last few months?

Busy building and launching UniThrive.org!

UniThrive.org is a fantastic new non-profit startup that seeks to help reduce the cost of higher education by networking college students with alumni, and facilitating direct, zero-interest loans between alumni and students to defray tuition costs.

Technologically, UniThrive is a Rails application that began as a fork of the open-source social networking application Insoshi. Since forking Insoshi, we've nearly doubled the size of the code.

Today, UniThrive is in a live beta test available to students and alumni of Harvard University. Take a look, and check out the UniThrive Blog!

Single Table Inheritance and RESTful Routes

I'm converting an old, controller/action/id style Rails application to a more RESTful way of doing things, and ran into a brief roadblock: one of my main tables uses single table inheritance to generate three subclasses of items. I never actually use the superclass "task", I only use the three subclasses "action item", "work order", and "problem report".

So, I ran into this little challenge: all three STI subclasses use the same controller, "tasks", because they all have essentially the same behavior and differ only in minor details. But, when I do a resources map:

map.resources :tasks

Then I get errors in much of my code when I say things like redirect_to @task, because if that task happens to be an ActionItem, it's trying to call action_item_path(@task), which doesn't exist.

I googled around a bit to no result. Striking out on my own, it turns out the answer is as simple as mapping each resource independently, and just overriding the controller in map.resources:

In config/routes.rb

map.resources :tasks
map.resources :action_items, :controller => 'tasks'
map.resources :work_orders, :controller => 'tasks'
map.resources :problem_reports, :controller => 'tasks'


Now, redirect_to @task works just fine regardless of which subclass @task happens to be.