Spec::Runner.configure do |config|
config.prepend_before(:each, :type => :controller) do
...
end
end

Now looks more like:

RSpec::configure do |config|
  config.before(:each, :line => 153) do
  end
end

One significant and interesting change is the way that before hooks are processed.  Specifically, the #before, #after, and #around methods are now part of the Hooks module, which is included in both ExampleGroup and in Configuration, so you call configure.before in exactly the same way as you do within a describe block.  Normally, you pass :each or :all, which sets the scope under which the hook will be called, but Hooks inspects the arguments for filtering metadata regardless of where you call it - I don't know that you'd want filter within an ExampleGroup, but you could...

Unfortunately, as cool as the metadata filtering capabilities are they aren't, as far as I can tell, very well documented.  The process of extracting the metadata lives in it's own :nodoc: limbo, and the attachment of metadata to a particular example is scattered throughout the RSpec code.  This, then, is an attempt to pick that apart.

Extracting Filters

When you call Hooks#before, for example (#after and #around work fundamentally the same way), the args are examined and two things are extracted:

A scope, which is the :each, :all, or :suite specification.

A metadata filter hash.  Normally, you call #before(:each, {:hash => [:of, :metadata]}), but you can instead do something like before(:all, :symbol) which will result in a metadata filter like {:symbol => true}

Again, probably if you need to add metadata inside of a describe block, you are Doing Something Wrong, but maybe there's a good reason.  The extreme (excessive?) flexibility of RSpec metadata and filtering does open up a lot of interesting possibilities.

Filter Matching

The metadata filter is used to decide if the hook should be run for a particular example block that it might apply to.  As such, it's a remarkably powerful filtering system, although there's a lot of assumptions about it's format that you need to bear in mind.

The actual mechanics of the metadata filtering happen in RSpec::Core::Metadata#apply? and #apply_condition - there's a long chain of delegation and extra-meta-programming that leads there.

The upshot is that your metadata filter will be compared to the metadata on the example key/value pair by pair, like this:

• A regular expression in the filter will match against the appropriate value for the example.
• If you pass :line_number => 17, Rspec will check to see if the example includes line 17, much like running rspec filename_spec.rb:17
• Any other Fixnum will be compared with == to the value in the metadata
• Anything else gets compared with == to the value in the metadata, after both values have been converted to a string.
• A proc like {|value| ... } will get the value of the key, and can return true for a match.

Filters can nest Hashes, which will be compared to nested Hashes in the metadata.  In other words, if you want to be able to match for metadata like

{...,  :example_group => {..., :full_description => "A very long winded example of the group", ...}, ...}

You can do something like:

before(:each, {:example_group =>{:full_description => /long winded/}})

it "should do something useful, someday", :pending => "Not this day, though"

Which is much faster than using the pending method call inside the block, and can be applied to a describe block to make the whole thing pending - especially handy when you have a before block inside that is causing problems.

In the same token, the example given in RSpec 2 documentation and announcement posts has been doing something like:

it "should not be taking this looooong", :slow => true

Since metadata can also be used to filter examples, you could use this to pull out the examples that take forever from your all-the-time specs, and run them only before a push, for instance.

What Metadata Does RSpec Give Us?

Probably the best way to figure that out is this very pragmatic approach.

A Useful Trick

Very useful for experimenting with metadata is that the proc form of the metadata has a special case: if the proc takes two arguments, the whole metadata hash will get passed into the proc, so you can inspect it at leisure.  The snippet looks like:

require 'pp'
before(:each, :bogus => proc{|val, all| pp all}) {}

{
:execution_result=>{:started_at=>Tue Jun 07 14:13:46 -0700 2011},
:type=>:controller,
:full_description=>"UserSessionsController should be authorized",
:description=>"should be authorized",
:example_group=>
  {
  :full_description=>"UserSessionsController",
  :file_path=> "spec/controllers/user_sessions_controller_spec.rb",
  :describes=>UserSessionsController,
  :description=>"UserSessionsController",
  :block=> #
,
  :line_number=>3,
  :caller=> [ ... the whole backtrace of the group ... ]
  },
:caller=> [ ... the backtrace of the example ...]
}

One of the cool-but-problematic things about metadata in RSpec is that it get's added and updated all over the codebase, and constantly over the lifecycle of an example run and extensions (like Rspec-Rails) add their own fields and values, so it's very hard to have formal documentation for what you can match.  Also, somewhat troubling, is that none of these fields are an explicit part of the RSpec API, and so might change with very little notice.  It seems like the best way to manage working with the metadata is with the above pragmatic approach.

An example method added to forms:

Since I nearly always want <input> and <label> tags at the same time, I created a labeled_input method that lets me say this (in HAML):

= form_for(@book) do |f|
    = f.labeled_input :title
    = f.labeled_input :author
    = f.labeled_input :price

to get:

<form action="/books/new">
  <div class="labeled_input">
    <label for="book_title">Title:</label><input id="book_title" name="book[title]" type="text" />
  </div>
  <div class="labeled_input">
    <label for="book_author">Author:</label><input id="book_author" name="book[author]" type="text" />
  </div>
  <div class="labeled_input">
    <label for="book_price">Price:</label>
    <input id="book_price" name="book[price]" type="text" />
  </div>
</form>

Combined with some default CSS code in our application template that aligns the <label>s and <input>s in columns, this saves us a couple of hours setting up clean-looking forms on every new project, while significantly shortening and prettifying our view templates. (Markup Haiku, just like HAML intended.)

Implementing the extension in Rails 3

The code that handles form_for in Rails 3 is rather dense and incomprehensible and takes a while to pore through. Here's the short version to understanding it so you can add your own methods to FormBuilder properly. Since we dug through it, hopefully this will save others some time. The only Rails file you care about for this purpose is actionpack-3.0.x/lib/action_view/helpers/form_helper.rb.

• module ActionView::Helpers::FormHelper defines a bunch of helpers, like label, text_field, etc. that define helpers you use outside of a form_for. For example, text_field(@user, :title) calls this version of the helper.
• class ActionView::Helpers::FormBuilder is what's used to define the helpers you run inside a form_for. It works automatically via metaprogramming ... when loaded, it finds each helper in FormHelper (except for a few) and defines a similarly named method in FormBuilder. form_for(@user) { |f| f.text_field(:title) calls this version of the helper, which basically just calls the FormHelper version but passes the FormBuilder's @object_name as an additional first argument. In version 3.0.7, this metaprogramming happens on lines 1131-1141 of form_helper.rb.
• As a result, if you were to write a new helper in ActionView::Helpers::FormHelper that uses the same argument structure as the pre-built ones, you'd automatically get both kinds of helper. However, if you're writing your own plugin or gem and injecting new helpers, this won't happen because by the time you inject your method FormBuilder will have already done its metaprogramming (it happens when the file is loaded).
• The solution to this is that your gem needs to do the second half - defining the FormBuilder version of the helper - itself. I'll put an example below.
• Most of the helper methods work by instantiating InstanceTag, a local one-size-fits-all class to emit a form tag, and then calling the appropriate method for the kind of tag that's wanted, like to_text_field_tag. It's very confusing why the Rails team decided to do one class for InstanceTag and a bunch of different methods, rather than make subclasses of InstanceTag for each kind of tag they want; an odd OOP decision, but that's what we've got.
• InstanceTag itself has only one line: it includes InstanceTagMethods, a model that defines all the methods for the class, and which isn't used elsewhere.

So to implement a FormBuilder method yourself that you can use inside a form_for, the best way is to inject your method inside FormHelper, and then call that from a method you inject into FormBuilder. This gives you both versions of the method, in the same structure that Rails defines them. You could do this either in a helper file directly in your application, or in a gem (like we have) so you can reuse your form helpers in more than one projects.

An example implementation.

Here's a simplified construction of the labeled_input method we use at LRD. This one just emits a label and a text field and wraps them in a <div>.

Start by defining the helper:

module LRD
  module FormHelper
    def labeled_input(object_name, method, options = {})
      input = text_field(object_name, method, options)
      label = label(object_name, method, options)
      content_tag(:div, (label+input), { :class => 'labeled_input' }
    end
  end
end
ActionView::Helpers::FormHelper.send(:include, LRD::FormHelper)

This will successfully define labeled_input that you can use outside of a form_for.

Now add the FormBuilder version:

To get it working inside of a form_for, you need to add a similar method to ActionView::Helpers::FormBuilder. As mentioned above, Rails does this automatically for its own FormHelper methods using a metaprogramming approach. But since that has already happened by the time your code can inject into FormHelper, you have to do it yourself. The solution we used is to make our own FormBuilder module that manually defines the labeled_input method in the same format that FormBuilder would have done, and then auto-include that into FormBuilder when our own FormHelper module gets included. Add this stuff to the above code block:

# Inside LRD::FormHelper, add this method:
def self.included(arg)
  ActionView::Helpers::FormBuilder.send(:include, LRD::FormBuilder)
end
 
module LRD::FormBuilder
  # ActionPack's metaprogramming would have done this for us, if FormHelper#labeled_input 
  # had been defined  at load.   Instead we define it ourselves here.
  def labeled_input(method, options = {})
    @template.labeled_input(@object_name, method, objectify_options(options))
  end
end

In practice, our labeled_input method is much more complex; it handles other input types, can add instructional comments/notes to the field, and can accept a block if you want to put something other than an <input> where the text field normally goes. This guide should get you started to writing your own form_for methods quickly, but if you want to see how to do more complex things, check out the full version.

Adding more input types or other tags.

If you wanted to add an entire different tag or input type (as opposed to combining different ones, the way labeled_input does), you would probably start by building a module that you inserted into InstanceTag or InstanceTagMethods. It should define a method like MyInstanceTagModule#to_some_funky_tag() in parallel with to_input_field_tag().

Testing it with rSpec 2

Another challenge we faced was writing specs for labeled_input's behavior. It's a bit of a trick because we needed to instantiate ActionView and render some templates to check the output, but rspec-rails is written with the assumption that you will be loading an entire rails project and all the rails gems. If you want to spec just a view helper, you need to load a bunch of rspec-rails's files one by one, and then manually include RSpec::Rails::ViewExampleGroup into RSpec's configuration. We may write a separate post on this process in the future, but in the meantime, take a look at lrd_view_tools' spec_helper file and example spec for labeled_input to get the sense of it. ]]> http://blog.lrdesign.com/2011/04/extending-form_for-in-rails-3-with-your-own-methods/feed/ 0 http://blog.lrdesign.com/2011/01/on-the-road-to-faster-specs/ http://blog.lrdesign.com/2011/01/on-the-road-to-faster-specs/#comments Fri, 21 Jan 2011 23:39:41 +0000 Evan http://blog.lrdesign.com/?p=234 Running large spec or test suites can be a bane of Rails developers. I've certainly stopped using autotest since half of our projects started exceeding 5 minutes of spec runtime. After seeing (three times!) presentations by Aman Gupta, I had spent some time with perftools trying to figure out what the heck was making my specs take so long. It's not just that more specs take more time to run: if you clock individual specs you will see identical examples run slower in a larger project. I'd seen that rspec runs can spend upwards of 60% of their time in the garbage collector, but not pursued it further than that.

A couple days ago, Jamis at the 37Signals blog took this idea further, dug into ActiveSupport::TestCase, and generated this wonderful blog post that explains his findings and how to get a 40% or more speedup in Test::Unit. His solution involves reducing the frequency of garbage collection and forcing ActiveSupport::TestCase to destroy instance variables it doesn't need anymore).

It's great, but if you do exactly what he says it won't quite work in RSpec - and RSpec users should get to enjoy this new development, too! While RSpec makes use of ActiveSupport::TestCase, it has a different set of internal instance variables, and Jamis' code will end up erasing the variables that store your actual examples. If you drop in Jamis' code to spec_helper.rb you'll see this error:

vendor/rails/activesupport/lib/active_support/whiny_nil.rb:52:in `method_missing':
 undefined method `description' for nil:NilClass (NoMethodError)

All that's needed to make RSpec happy is a little tweak to Jamis' code that protects a different set of instance variables from being unset. Just drop this blob of code at the bottom of your spec_helper.rb - I saw a 43% speed increase in one project's spec suite. (Note that if you are still using fixtures, you might need to add @loaded_fixtures and/or @fixture_cache to @@reserved_ivars; at LRD we long since abandoned fixtures in favor of factories, so I haven't tested this on spec suites with fixtures).

class ActiveSupport::TestCase
  setup :begin_gc_deferment
  teardown :reconsider_gc_deferment
  teardown :scrub_instance_variables
 
  @@reserved_ivars = %w(@_implementation @_result @_proxy  @_assigns_hash_proxy @_backtrace)
  DEFERRED_GC_THRESHOLD = (ENV['DEFER_GC'] || 1.0).to_f
 
  @@last_gc_run = Time.now
 
  def begin_gc_deferment
    GC.disable if DEFERRED_GC_THRESHOLD > 0
  end
 
  def reconsider_gc_deferment
    if DEFERRED_GC_THRESHOLD > 0 && Time.now - @@last_gc_run >= DEFERRED_GC_THRESHOLD
      GC.enable
      GC.start
      GC.disable
 
      @@last_gc_run = Time.now
    end
  end
 
  def scrub_instance_variables
    (instance_variables - @@reserved_ivars).each do |ivar|
      instance_variable_set(ivar, nil)
    end
  end
end

(Most of this code is Jamis', and I'm not taking credit for his fantastic work.)

RSpec already does a much better job of handling instance variables than Test::Unit, so the scrubbing didn't produce a big speedup for me (only about 5%). But the GC deferment did indeed give me a 43% speed improvement in the spec suite for my biggest project; run time dropped from 7m38s to 4m23s ... what a difference!

Lots of file exchange services exist, for example SendBigFiles.com etc. However, all of these services are hosted on someone else's hardware, and most of them share files by transferring URLs -- usually via email -- without good access control or authorization schemes.

We built Convection because a client needed to transfer files with other companies, but they needed to host the system themselves because the contracts they hold with their own clients require them not to store data on services that they don't control. The specifications Convection was built around were:

1. Hosted on our own server.
2. Downloads require a login, and files cannot be shared by email.
3. Users must log in to download files or see available files.
4. User accounts can be grouped, groups can be managed.
5. Files can be shared with an entire group.
6. Files uploaded by users default to minimal permission - visible only to the uploader and to admins.
7. All communications over SSL. (we made this optional)

Installing and hosting Convection

To run Convection, you will need a webserver capable of running a Ruby on Rails application, and a database. Setting such a thing up is beyond the scope of this post. If you have a Dreamhost account, you can set up a Rails-capable domain with a couple of clicks in their web panel. In addition to the server, you will need to set up a database (we have only tested MySQL, but Convection should work with any SQL database for which Rails/ActiveRecord has a supported adapter, including PostgreSQL and Oracle), and initialize the database with these two commands:

  > rake db:migrate RAILS_ENV=production
  > rake db:seed RAILS_ENV=production

This will generate the tables necessary for Convection to run, and create a pair of initial demo users "admin" and "user", both with password "foobar".

If you are setting up a server yourself, there are plenty of guides to deploying Rails on the web. Much of our own guide to deploying CruiseControl.rb can be used to set up any Rails application on Slicehost or any other Ubuntu Linux hosting provider.

Let me know if you're trying to deploy Convection and having trouble: if we know people are using it we may put effort into making it easier to deploy and install, and write a more thorough guide.

A few other links that may help you with deploying a Rails application, depending on your environment:

1. Using Phusion Passenger to Deploy a Rails Application on Apache
2. Deploying Rails Applications (book)

If you Google around you may find plenty of other links relevant to your particular environment.

Configuring Convection

If you log into your running Convection application as an administrator (initial user "admin", password "foobar"), an Admin Tools utility will appear in the right hand column. From here, you can access tools for creating users, and groups, and the general site configuration.

In general site config, you can set your site name and logo, set whether or not the site requires SSL access (Note: your server must already support SSL!) outgoing email and email notification preferences, add Google analytics, and an assortment of other site configuration operations that are mostly self-explanatory.

Upload progress bar: experimental feature.

If your site hosts large uploads that take a while to transfer, you can try our experimental tools to provide an upload progress bar to the user. This tool will only work if your site is served by Apache, and requires installing and configuring an optional module for Apache.

To enable this tool, follow the instructions in the README file and associated links, and turn on the progress bar setting in site preferences.

Helping us improve Convection

Convection is currently in version 1.1.4 and has been in production in two places (that we know of) for about five months as of June 8, 2010.

Please let us know if you are using Convection and enjoy it (or don't). Feel free to request features or alterations, but Convection is open source, so also please consider contributing if you have ideas!

We're using the use_db plugin, and all it takes to make testing transactions happen around multiple DBs is:

In: spec/spec_helper.rb

require 'override_test_callbacks'

My concern comes from the fact that this is a direct and unfiltered monkeypatch on ActiveRecord::TestFixtures. So it relies on use_transactional_fixtures (which could certainly be used without using actual fixures, granted), and if the test transaction code moves within Rails, that's another integration to worry about. Or if we add a spec that doesn't wind up making ActiveRecord::TestFixtures load... Or if we decide to use something other than use_db...

So instead I'm using:

Spec::Runner.configure do |config|
  config.prepend_before do
    (UseDbPlugin.all_use_dbs - [ActiveRecord::Base]).each do |db|
      db.connection.increment_open_transactions
      db.connection.transaction_joinable = false
      db.connection.begin_db_transaction
    end
  end
 
  config.append_after do
    (UseDbPlugin.all_use_dbs - [ActiveRecord::Base]).reverse.each do |db|
      db.connection.rollback_db_transaction
      db.connection.decrement_open_transactions
    end
  end
end

If we weren't already using transactional fixtures, I might pull out the - [ActiveRecord::Base]. And if we were to change off of use_db, there's one place to change the transaction code. Finally, there's much less dependence on the innards of ActiveRecord - only it's published API.

MyModel.new(:field_1 => 'foo', :field_2 => 'bar')
MyModel.new('field_1' => 'foo', 'field_2' => 'bar')

It's convenient, often, to not have to worry about whether your keys are symbols are strings since they get converted around a bit when you pass parameters. The downside of this, however, is that it will happily accept BOTH without complaining, and will quietly default to the symbol key regardless of the order you specify them in:

>> model = MyModel.new(:field_1 => 'foo', 'field_1' => 'bar'); nil; 
>> mymodel.field_1
=> 'foo'
>> model = MyModel.new('field_1' => 'foo', :field_1 => 'bar'); nil;
>> mymodel.field_1
=> 'bar'

Okay, so that's kinda sloppy. Bad ActiveRecord! No Biscuit!

This can cause serious confusion for the unwary. When ActionController hands us a params hash, it always has String keys, like this:

>> eval params
=>  { 'article' => { 'title' => 'Awesome blog post', 'body' => 'I will make you smart' } }

But most of us, canonically, specify params and default AR values with symbols, like this:

   post :article => {:title => 'Awesome blog post', :body => 'I will make you smart'}

So we get used to thinking about them as symbols.

This means we can make mistakes like this one I made recently. Consider this block of code for a shopping cart model that pre-fills some fields for an associated Payment by pulling the address from the user's profile, to save the user re-typing their address:

class ShoppingCart < ActiveRecord::Base
  has_one :payment
 
  def build_default_payment(options = {}) 
    #prepopulate the billing address from the profile and merge
    #with params passed into options
    build_payment(prepopulated_fields.merge!(options)    
  end
 
  def prepopulated_fields
    if (addr = self.person.address)
      {
        :billing_address_1 => addr.line_1,
        :billing_address_2 => addr.line_2,
        :city => addr.city,
        :state => addr.state,
        :zip => addr.zipcode
      }
    else
      {}
    end
  end
end

Looks great, right? And if the user's address has a nil field (like no city, or no line_1), it will get overwritten by the hash merge.

Except not. I specified symbol keys in prepopulated_fields, but the hash getting passed to build_default_payment's 'options' argument has string keys, because it's coming from params. So the merge doesn't overwrite the value for :line_1, it simply adds a new key 'line_1'. So, if a user has a profile address but hadn't entered a line_1 (just city and state), and then manually entered line_1 in the payment form to submit, the Payment build during the create action was getting this hash:

build_payment({
   :line_1 => nil,
   :city => 'Pasadena',
   :state =>'CA',
   :zipcode => '91106'
   'line_1' => '100 Main St.'.
})

ActiveRecord was respecting the :line_1 => nil from the profile, and not the 'line_1' => '100 Main St.' from params. This meant that the user couldn't make payment! The payment had validates_inclusion_of line_1, and even though it was typed into the form it was getting ignored because of the nil from his profile address. Very frustrating for a user to manually type in a billing address and get back "Address Line 1 can't be blank." on every submit!

Nasty ... this one took a while to figure out. Beware of this little foible of ActiveRecord!

Logical Reality did the logo and layout design work for Rails Tutorial.

Specifically, I needed to put annotations in the description of a work order object that said, for example "this work order was escalated from Problem Report 293. This was done in a create action that redirected at the end and never rendered a view, so I really did need to generate that link in the controller. And for consistency with the rest of the application, I wanted to generate the link with link_to(@task).

Now, ActionView::Helpers::UrlHelper is not loaded in a Rails controller, even if you've put helper :all in application.rb (application_controller.rb in newer versions). So, when I tried to use link_to in the controller, I got an error:

NoMethodError: undefined method `link_to' for #
/Users/evan/Development/Ruby/eclipticdb/app/helpers/tasks_helper.rb:64:in `task_link'
/Users/evan/Development/Ruby/eclipticdb/app/controllers/tasks_controller.rb:103:in `escalate'
... etc ...


The first fix - but with a problem

A year ago, I fixed this just by adding include ActionView::Helpers::UrlHelper at the top of that controller. This worked great ... for a while.

Lately, I've been rewriting this application into a RESTful style - it had previously been a controller/action style application. In the process, I started linking things with resource paths and polymorphic paths ... a lot of link_to @task and edit_polymorphic_path(@task) sorts of bits. And these started breaking. I began seeing this mysterious error:

Error:

You have a nil object when you didn't expect it!
The error occurred while evaluating nil.url_for

vendor/rails/actionpack/lib/action_view/helpers/url_helper.rb:71:in `send'
vendor/rails/actionpack/lib/action_view/helpers/url_helper.rb:71:in `url_for'

This one was a real bitch to debug, I have to say. The line in question that was failing in url_helper.rb said this: url = @controller.send(:url_for, options). Clearly, @controller was nil ... which was very bizarre, because I never interact with that instance variable anywhere.

I thrashed around trying to find the cause of this error for quite some time. Eventually I realized that the link_to method was only failing when called from a view in TasksController, and not from any other controller. And then I realized that TasksController was the one where, a year ago, I'd put include ActionView::Helpers::UrlHelper at the top. Somehow, including that helper in the controller was nullifying @controller when those helper method we called from within the view. I removed the include and my polymorphic and resource links all started working again.

Now back to the original problem!

Of course, that then left me back with the problem I'd had a year ago ... needing to use link_to from within the controller and having no way to do it. After a fair bit of googling around I found this post from Neeraj, which had an interesting approach -- but a commenter had suggested a much easier solution:

[sourcecode language='ror']self.class.helpers.link_to[/sourcecode]

I'm not certain where one would find this in the docs, but it does seem to have solved my problem for now. Onward and upward!

So, I ran into this little challenge: all three STI subclasses use the same controller, "tasks", because they all have essentially the same behavior and differ only in minor details. But, when I do a resources map:

map.resources :tasks

Then I get errors in much of my code when I say things like redirect_to @task, because if that task happens to be an ActionItem, it's trying to call action_item_path(@task), which doesn't exist.

I googled around a bit to no result. Striking out on my own, it turns out the answer is as simple as mapping each resource independently, and just overriding the controller in map.resources:

In config/routes.rb

map.resources :tasks
map.resources :action_items, :controller => 'tasks'
map.resources :work_orders, :controller => 'tasks'
map.resources :problem_reports, :controller => 'tasks'


Now, redirect_to @task works just fine regardless of which subclass @task happens to be.

I'm following Chapweske's approach of blocking mass assignment by default in all models, by putting this line in an initializer:

ActiveRecord::Base.send(:attr_accessible, nil)

This had the expected side effect of breaking several zillion tests, because tests frequently use things like Model.build() and Model.create!() to generate on-demand fixtures during testing. Hartl has a great bit of code that creates unsafe_build() and unsafe_create() methods in ActiveRecord. You can use these methods instead of build() and create() to function as expected in your tests.

This works great, except that I also use the mass-assignment method update_attributes! in my tests and specs frequently, particularly when I want to spec the effect a change on one model has on an associated models' methods. So, I expanded on Hartl's helper code a bit, to give myself the necessary methods. In case it helps anyone else:

/lib/initializers/unsafe_build_and_create.rb

class ActiveRecord::Base

def unsafe_attributes=(attrs)
attrs.each do |k, v|
send("#{k}=", v)
end
end
end