When ‘rake spec’ and ‘spec spec’ produce different results

August 9, 2009

AKA adventures in class loading.

A couple of days ago I did some significant work in authorization in one of my apps, involving creating a Groups class with an HABTM relationship to Person, so I could assign roles to people a group at a time. It all worked out great, and I pushed the product to GitHub. The next day, my collaborator wrote in that my recent contribution broke 119 specs.

I pulled and retested the code, and everything worked perfectly. WTF? After a bit of investigation, I discovered that the specs worked great when I ran 'autotest' or 'spec spec', but that 119 specs broke when I ran the exact same spec suite with 'rake spec'! Double WTF.

Setting constants at class loading

Ultimately, I tracked the problem down to this line and method, in Person.rb:
class Person << ActiveRecord::Base
ADMIN_GROUP = Group.find_by_name('Admin')
def admin?
groups.include? ADMIN_GROUP
end
end


I consider a person an administrator if they are a member of this group, and I was loading it as a constant at the class level in order to avoid having to query the database again every time Person#admin? is called. This worked just fine for me, both in the application, and every time I ran Person#admin?.

But, remarkably, ADMIN_GROUP does not get initialized correctly when I run the tests via rake. I found this via the ruby debugger, running in this particular spec in spec/models/person_spec.rb:


describe Person do
it "should load an admin user from fixture" do
debugger
people(:admin).should be_admin
end
end


When I run the specs and evaluate Person::ADMIN_USER, I get very different results depending on which spec runner I'm using:

Running 'spec spec/models/person_spec.rb':


[11:17:54 CITAlumni]$ spec spec/models/person_spec.rb
spec/models/person_spec.rb:64
people(:admin).should be_admin
(rdb:1) eval Person::ADMIN_GROUP
#


Running 'rake spec SPEC=spec/models/person_spec.rb':


[11:17:13 CITAlumni (48c51f1...)]$ rake spec SPEC=spec/models/person_spec.rb
(in /Users/evan/Development/Ruby/CITAlumni)
FF.....spec/models/person_spec.rb:64
people(:admin).should be_admin
(rdb:1) eval Person::ADMIN_GROUP
nil


How very interesting ... when I use rake, that constant initializes to nil. At some point, I'll actually get around to figuring out why this is so different when the specs are run via rake. In the meantime, the fix was easy:

The Solution

The fix was just to refactor ADMIN_GROUP as a class method with a memoized instance variable. This will at least limit DB queries for the admin group to one per page load; not quite as good as a single DB hit when the class is first loaded, but still a major improvement over querying for the admin group every time Person#admin? is called. I moved it to the Group class at the same time, which was probably the right place for it in the first place:


#app/models/group.rb:
class Group < ActiveRecord::Base
def self.admin_group
@admin_group ||= self.find_by_name('Admin')
end
end

#app/models/person.rb
class Group < ActiveRecord::Base
def admin?
groups.include? Group.admin_group
end
end


And this worked just fine in all environments, solving the problem with 'rake spec'.

The Moral

Be careful with depending on behavior that occurs only during the loading of classes, as it can be environment-dependent!

If anyone out there with uber Ruby skills knows exactly why running specs via rake prevents that class variable from loading correctly, please enlighten us in comments!