On the road to faster specs

Running large spec or test suites can be a bane of Rails developers. I've certainly stopped using autotest since half of our projects started exceeding 5 minutes of spec runtime. After seeing (three times!) presentations by Aman Gupta, I had spent some time with perftools trying to figure out what the heck was making my specs take so long. It's not just that more specs take more time to run: if you clock individual specs you will see identical examples run slower in a larger project. I'd seen that rspec runs can spend upwards of 60% of their time in the garbage collector, but not pursued it further than that.

A couple days ago, Jamis at the 37Signals blog took this idea further, dug into ActiveSupport::TestCase, and generated this wonderful blog post that explains his findings and how to get a 40% or more speedup in Test::Unit. His solution involves reducing the frequency of garbage collection and forcing ActiveSupport::TestCase to destroy instance variables it doesn't need anymore).

It's great, but if you do exactly what he says it won't quite work in RSpec - and RSpec users should get to enjoy this new development, too! While RSpec makes use of ActiveSupport::TestCase, it has a different set of internal instance variables, and Jamis' code will end up erasing the variables that store your actual examples. If you drop in Jamis' code to spec_helper.rb you'll see this error:

vendor/rails/activesupport/lib/active_support/whiny_nil.rb:52:in `method_missing':
 undefined method `description' for nil:NilClass (NoMethodError)

All that's needed to make RSpec happy is a little tweak to Jamis' code that protects a different set of instance variables from being unset. Just drop this blob of code at the bottom of your spec_helper.rb - I saw a 43% speed increase in one project's spec suite. (Note that if you are still using fixtures, you might need to add @loaded_fixtures and/or @fixture_cache to @@reserved_ivars; at LRD we long since abandoned fixtures in favor of factories, so I haven't tested this on spec suites with fixtures).

class ActiveSupport::TestCase
  setup :begin_gc_deferment
  teardown :reconsider_gc_deferment
  teardown :scrub_instance_variables
 
  @@reserved_ivars = %w(@_implementation @_result @_proxy  @_assigns_hash_proxy @_backtrace)
  DEFERRED_GC_THRESHOLD = (ENV['DEFER_GC'] || 1.0).to_f
 
  @@last_gc_run = Time.now
 
  def begin_gc_deferment
    GC.disable if DEFERRED_GC_THRESHOLD > 0
  end
 
  def reconsider_gc_deferment
    if DEFERRED_GC_THRESHOLD > 0 && Time.now - @@last_gc_run >= DEFERRED_GC_THRESHOLD
      GC.enable
      GC.start
      GC.disable
 
      @@last_gc_run = Time.now
    end
  end
 
  def scrub_instance_variables
    (instance_variables - @@reserved_ivars).each do |ivar|
      instance_variable_set(ivar, nil)
    end
  end
end

(Most of this code is Jamis', and I'm not taking credit for his fantastic work.)

RSpec already does a much better job of handling instance variables than Test::Unit, so the scrubbing didn't produce a big speedup for me (only about 5%). But the GC deferment did indeed give me a 43% speed improvement in the spec suite for my biggest project; run time dropped from 7m38s to 4m23s ... what a difference!