LRBlog » Development

RSpec 2.0 and before/after hooks

Judson — Tue, 07 Jun 2011 21:42:58 +0000

As of RSpec 2, the configuration interface for RSpec changed dramatically. What used to look like:

Spec::Runner.configure do |config|
config.prepend_before(:each, :type =&gt; :controller) do
...
end
end

Now looks more like:

RSpec::configure do |config|
  config.before(:each, :line =&gt; 153) do
  end
end

One significant and interesting change is the way that before hooks are processed. Specifically, the #before, #after, and #around methods are now part of the Hooks module, which is included in both ExampleGroup and in Configuration, so you call configure.before in exactly the same way as you do within a describe block. Normally, you pass :each or :all, which sets the scope under which the hook will be called, but Hooks inspects the arguments for filtering metadata regardless of where you call it - I don't know that you'd want filter within an ExampleGroup, but you could...

Unfortunately, as cool as the metadata filtering capabilities are they aren't, as far as I can tell, very well documented. The process of extracting the metadata lives in it's own :nodoc: limbo, and the attachment of metadata to a particular example is scattered throughout the RSpec code. This, then, is an attempt to pick that apart.

Extracting Filters

When you call Hooks#before, for example (#after and #around work fundamentally the same way), the args are examined and two things are extracted:

A scope, which is the :each, :all, or :suite specification.

A metadata filter hash. Normally, you call #before(:each, {:hash => [:of, :metadata]}), but you can instead do something like before(:all, :symbol) which will result in a metadata filter like {:symbol => true}

Again, probably if you need to add metadata inside of a describe block, you are Doing Something Wrong, but maybe there's a good reason. The extreme (excessive?) flexibility of RSpec metadata and filtering does open up a lot of interesting possibilities.

Filter Matching

The metadata filter is used to decide if the hook should be run for a particular example block that it might apply to. As such, it's a remarkably powerful filtering system, although there's a lot of assumptions about it's format that you need to bear in mind.

The actual mechanics of the metadata filtering happen in RSpec::Core::Metadata#apply? and #apply_condition - there's a long chain of delegation and extra-meta-programming that leads there.

The upshot is that your metadata filter will be compared to the metadata on the example key/value pair by pair, like this:

A regular expression in the filter will match against the appropriate value for the example.
If you pass :line_number => 17, Rspec will check to see if the example includes line 17, much like running rspec filename_spec.rb:17
Any other Fixnum will be compared with == to the value in the metadata
Anything else gets compared with == to the value in the metadata, after both values have been converted to a string.
A proc like {|value| ... } will get the value of the key, and can return true for a match.

Filters can nest Hashes, which will be compared to nested Hashes in the metadata. In other words, if you want to be able to match for metadata like

{...,  :example_group =&gt; {..., :full_description =&gt; "A very long winded example of the group", ...}, ...}

You can do something like:

before(:each, {:example_group =&gt;{:full_description =&gt; /long winded/}})

RSpec attaches some metadata to examples and groups, but you can also explicitly add metadata to groups and examples as they're defined. One useful example of that is

it "should do something useful, someday", :pending =&gt; "Not this day, though"

Which is much faster than using the pending method call inside the block, and can be applied to a describe block to make the whole thing pending - especially handy when you have a before block inside that is causing problems.

In the same token, the example given in RSpec 2 documentation and announcement posts has been doing something like:

it "should not be taking this looooong", :slow =&gt; true

Since metadata can also be used to filter examples, you could use this to pull out the examples that take forever from your all-the-time specs, and run them only before a push, for instance.

What Metadata Does RSpec Give Us?

Probably the best way to figure that out is this very pragmatic approach.

A Useful Trick

Very useful for experimenting with metadata is that the proc form of the metadata has a special case: if the proc takes two arguments, the whole metadata hash will get passed into the proc, so you can inspect it at leisure. The snippet looks like:

require 'pp'
before(:each, :bogus =&gt; proc{|val, all| pp all}) {}

From a Rails controller spec:

{
:execution_result=&gt;{:started_at=&gt;Tue Jun 07 14:13:46 -0700 2011},
:type=&gt;:controller,
:full_description=&gt;"UserSessionsController should be authorized",
:description=&gt;"should be authorized",
:example_group=&gt;
  {
  :full_description=&gt;"UserSessionsController",
  :file_path=&gt; "spec/controllers/user_sessions_controller_spec.rb",
  :describes=&gt;UserSessionsController,
  :description=&gt;"UserSessionsController",
  :block=&gt; #
,
  :line_number=&gt;3,
  :caller=&gt; [ ... the whole backtrace of the group ... ]
  },
:caller=&gt; [ ... the backtrace of the example ...]
}

One of the cool-but-problematic things about metadata in RSpec is that it get's added and updated all over the codebase, and constantly over the lifecycle of an example run and extensions (like Rspec-Rails) add their own fields and values, so it's very hard to have formal documentation for what you can match. Also, somewhat troubling, is that none of these fields are an explicit part of the RSpec API, and so might change with very little notice. It seems like the best way to manage working with the metadata is with the above pragmatic approach.

Extending form_for in Rails 3 with your own methods

Evan — Mon, 25 Apr 2011 19:58:12 +0000

At LRDesign, we have a bunch of internal tools to make laying out Rails views more consistent. I recently upgraded and improved some of ours for Rails 3, and published them as a gem. (The published / open source ones are available at https://github.com/LRDesign/lrd_view_tools, if you're interested). One of the handy techniques we figured out (poring through the Rails code) is how to correctly add a method to FormBuilder so that you can properly use it inside a form_for block.

An example method added to forms:

Since I nearly always want and tags at the same time, I created a labeled_input method that lets me say this (in HAML):

= form_for(@book) do |f|
    = f.labeled_input :title
    = f.labeled_input :author
    = f.labeled_input :price

to get:


  
    Title:
  
  
    Author:
  
  
    Price:

Combined with some default CSS code in our application template that aligns the s and s in columns, this saves us a couple of hours setting up clean-looking forms on every new project, while significantly shortening and prettifying our view templates. (Markup Haiku, just like HAML intended.)

Implementing the extension in Rails 3

The code that handles form_for in Rails 3 is rather dense and incomprehensible and takes a while to pore through. Here's the short version to understanding it so you can add your own methods to FormBuilder properly. Since we dug through it, hopefully this will save others some time. The only Rails file you care about for this purpose is actionpack-3.0.x/lib/action_view/helpers/form_helper.rb.

module ActionView::Helpers::FormHelper defines a bunch of helpers, like label, text_field, etc. that define helpers you use outside of a form_for. For example, text_field(@user, :title) calls this version of the helper.
class ActionView::Helpers::FormBuilder is what's used to define the helpers you run inside a form_for. It works automatically via metaprogramming ... when loaded, it finds each helper in FormHelper (except for a few) and defines a similarly named method in FormBuilder. form_for(@user) { |f| f.text_field(:title) calls this version of the helper, which basically just calls the FormHelper version but passes the FormBuilder's @object_name as an additional first argument. In version 3.0.7, this metaprogramming happens on lines 1131-1141 of form_helper.rb.
As a result, if you were to write a new helper in ActionView::Helpers::FormHelper that uses the same argument structure as the pre-built ones, you'd automatically get both kinds of helper. However, if you're writing your own plugin or gem and injecting new helpers, this won't happen because by the time you inject your method FormBuilder will have already done its metaprogramming (it happens when the file is loaded).
The solution to this is that your gem needs to do the second half - defining the FormBuilder version of the helper - itself. I'll put an example below.
Most of the helper methods work by instantiating InstanceTag, a local one-size-fits-all class to emit a form tag, and then calling the appropriate method for the kind of tag that's wanted, like to_text_field_tag. It's very confusing why the Rails team decided to do one class for InstanceTag and a bunch of different methods, rather than make subclasses of InstanceTag for each kind of tag they want; an odd OOP decision, but that's what we've got.
InstanceTag itself has only one line: it includes InstanceTagMethods, a model that defines all the methods for the class, and which isn't used elsewhere.

So to implement a FormBuilder method yourself that you can use inside a form_for, the best way is to inject your method inside FormHelper, and then call that from a method you inject into FormBuilder. This gives you both versions of the method, in the same structure that Rails defines them. You could do this either in a helper file directly in your application, or in a gem (like we have) so you can reuse your form helpers in more than one projects.

An example implementation.

Here's a simplified construction of the labeled_input method we use at LRD. This one just emits a label and a text field and wraps them in a

Start by defining the helper:

module LRD
  module FormHelper
    def labeled_input(object_name, method, options = {})
      input = text_field(object_name, method, options)
      label = label(object_name, method, options)
      content_tag(:div, (label+input), { :class =&gt; 'labeled_input' }
    end
  end
end
ActionView::Helpers::FormHelper.send(:include, LRD::FormHelper)

This will successfully define labeled_input that you can use outside of a form_for.

Now add the FormBuilder version:

To get it working inside of a form_for, you need to add a similar method to ActionView::Helpers::FormBuilder. As mentioned above, Rails does this automatically for its own FormHelper methods using a metaprogramming approach. But since that has already happened by the time your code can inject into FormHelper, you have to do it yourself. The solution we used is to make our own FormBuilder module that manually defines the labeled_input method in the same format that FormBuilder would have done, and then auto-include that into FormBuilder when our own FormHelper module gets included. Add this stuff to the above code block:

# Inside LRD::FormHelper, add this method:
def self.included(arg)
  ActionView::Helpers::FormBuilder.send(:include, LRD::FormBuilder)
end
 
module LRD::FormBuilder
  # ActionPack's metaprogramming would have done this for us, if FormHelper#labeled_input 
  # had been defined  at load.   Instead we define it ourselves here.
  def labeled_input(method, options = {})
    @template.labeled_input(@object_name, method, objectify_options(options))
  end
end

In practice, our labeled_input method is much more complex; it handles other input types, can add instructional comments/notes to the field, and can accept a block if you want to put something other than an where the text field normally goes. This guide should get you started to writing your own form_for methods quickly, but if you want to see how to do more complex things, check out the full version.

Adding more input types or other tags.

If you wanted to add an entire different tag or input type (as opposed to combining different ones, the way labeled_input does), you would probably start by building a module that you inserted into InstanceTag or InstanceTagMethods. It should define a method like MyInstanceTagModule#to_some_funky_tag() in parallel with to_input_field_tag().

Testing it with rSpec 2

Another challenge we faced was writing specs for labeled_input's behavior. It's a bit of a trick because we needed to instantiate ActionView and render some templates to check the output, but rspec-rails is written with the assumption that you will be loading an entire rails project and all the rails gems. If you want to spec just a view helper, you need to load a bunch of rspec-rails's files one by one, and then manually include RSpec::Rails::ViewExampleGroup into RSpec's configuration. We may write a separate post on this process in the future, but in the meantime, take a look at lrd_view_tools' spec_helper file and example spec for labeled_input to get the sense of it.

On the road to faster specs

Evan — Fri, 21 Jan 2011 23:39:41 +0000

Running large spec or test suites can be a bane of Rails developers. I've certainly stopped using autotest since half of our projects started exceeding 5 minutes of spec runtime. After seeing (three times!) presentations by Aman Gupta, I had spent some time with perftools trying to figure out what the heck was making my specs take so long. It's not just that more specs take more time to run: if you clock individual specs you will see identical examples run slower in a larger project. I'd seen that rspec runs can spend upwards of 60% of their time in the garbage collector, but not pursued it further than that.

A couple days ago, Jamis at the 37Signals blog took this idea further, dug into ActiveSupport::TestCase, and generated this wonderful blog post that explains his findings and how to get a 40% or more speedup in Test::Unit. His solution involves reducing the frequency of garbage collection and forcing ActiveSupport::TestCase to destroy instance variables it doesn't need anymore).

It's great, but if you do exactly what he says it won't quite work in RSpec - and RSpec users should get to enjoy this new development, too! While RSpec makes use of ActiveSupport::TestCase, it has a different set of internal instance variables, and Jamis' code will end up erasing the variables that store your actual examples. If you drop in Jamis' code to spec_helper.rb you'll see this error:

vendor/rails/activesupport/lib/active_support/whiny_nil.rb:52:in `method_missing':
 undefined method `description' for nil:NilClass (NoMethodError)

All that's needed to make RSpec happy is a little tweak to Jamis' code that protects a different set of instance variables from being unset. Just drop this blob of code at the bottom of your spec_helper.rb - I saw a 43% speed increase in one project's spec suite. (Note that if you are still using fixtures, you might need to add @loaded_fixtures and/or @fixture_cache to @@reserved_ivars; at LRD we long since abandoned fixtures in favor of factories, so I haven't tested this on spec suites with fixtures).

class ActiveSupport::TestCase
  setup :begin_gc_deferment
  teardown :reconsider_gc_deferment
  teardown :scrub_instance_variables
 
  @@reserved_ivars = %w(@_implementation @_result @_proxy  @_assigns_hash_proxy @_backtrace)
  DEFERRED_GC_THRESHOLD = (ENV['DEFER_GC'] || 1.0).to_f
 
  @@last_gc_run = Time.now
 
  def begin_gc_deferment
    GC.disable if DEFERRED_GC_THRESHOLD > 0
  end
 
  def reconsider_gc_deferment
    if DEFERRED_GC_THRESHOLD > 0 && Time.now - @@last_gc_run >= DEFERRED_GC_THRESHOLD
      GC.enable
      GC.start
      GC.disable
 
      @@last_gc_run = Time.now
    end
  end
 
  def scrub_instance_variables
    (instance_variables - @@reserved_ivars).each do |ivar|
      instance_variable_set(ivar, nil)
    end
  end
end

(Most of this code is Jamis', and I'm not taking credit for his fantastic work.)

RSpec already does a much better job of handling instance variables than Test::Unit, so the scrubbing didn't produce a big speedup for me (only about 5%). But the GC deferment did indeed give me a 43% speed improvement in the spec suite for my biggest project; run time dropped from 7m38s to 4m23s ... what a difference!

NinjaScript: Javascript so unobtrusive, you’ll never it see coming

Judson — Wed, 13 Oct 2010 23:19:24 +0000

We're happy to announce NinjaScript: a jQuery plugin for unobtrusive scripting.
NinjaScript provides:

CSS-like language for web page behavior
Define rich behaviors that include both event handlers and transformations.
Durable behaviors that survive DOM alteration, with performance comparable to jQuery's live() method.
Handy built-in behaviors for AJAX.

Motivations

Unobtrusive Javascript is one of the coming movements in web design for a reason. Separation of concerns is generally a good thing, and the idea of separating behavior from semantics is pretty obvious once you think about it. If nothing else, it makes it much easier to think about how you structure your site. Just build it out as if this were still 1998 and you couldn't trust a browser to open an alert box, much less submit AJAX, then come back and mark everything up.

On the other hand, one hears a lot about "Unobtrusive Is Hard" and how a graceful degrade takes twice as long, etc etc. At the same time, software exists to encapsulate skills. Could be you'll be seeing a Rails plugin from LRD soon to convert the big Rails helpers into degrading versions.

Therefore: NinjaScript. Unobtrusive Javascript in a tidy package so that you can get on with your day.

What's it look like?

Here's a very simple example. Suppose you have an existing

that POSTs and reloads the page, and you would like it to submit to the same URL, but via AJAX.

If you have NinjaScript loaded, this all you need:

$.behavior({
  '#coolness': $.ninja.submits_as_ajax() 
})

In addition to the pre-defined behaviors like submits_as_ajax(), you can build your own rich behaviors, specifying both transforms (alterations to an element that will be applied as soon as the element appears in the doms) and event handlers at the same time:

$.behavior({
  '.date_entry': { transform: function(elem){ $(elem).datepicker() }},
  '#work_unit_select_all': { click: selectAllWorkUnits },
 
  '#timeclock form.edit_work_unit':    $.ninja.ajax_submission({
    busy_element: function(elem){ return $('#timeclock')}
  }),
 
  '#messages .flash': {
    transform: function(elem) {
      $(elem).delay(10000).slideUp(600, function(){$(elem).remove()})
    }
  },
  '#timeclock input#work_unit_hours': {
    click: function(evnt, elem) {
      $(elem).val(hours_format(task_elapsed))
    }
  }
});

That's direct from an LRD project.

Basically, it's meant to look like a stylesheet - as much as possible within jQuery. This snippet:

adds a datapicker to the date entry fields
binds a click handler (defined elsewhere) to allow for selecting all work units
makes a form into an AJAX submitter - complete with busy overlay
makes the #messages list decay - items live for 10 seconds and then go away
sets up automatic calculation of hours for certain fields

Pretty simple, but powerful.

What you're seeing there are CSS-style selectors (strictly speaking: jQuery selectors) used to pick elements, and behaviors applied to the elements. $.ninja.ajax_submission() is a prepackaged behavior, which are pretty easy to write.

The ad hoc behavior applied to '#messages .flash' defines a transformation. Transformations are basically the code you'd throw into a document.ready block, pre-sorted to go to their respective elements, with the added bonus that they'll survive later modifications of the DOM.

Behaviors can also define event handlers by adding an events clause, with the events they respond to as keys. In other words:

$.behavior({
  '.fun': {
    events: {
      click: function(ev, elem){ $(elem).sing_and_dance(); }
      mouseover: function(ev, elem) { $(elem).shiver_in_anticipation(); }
      //yes: mouseover.
    }
  }
});

What the app needs to do

To start, pretend that there is no AJAX. Build everything with full round trip HTTP.

Next, come back and make sure that your app responds to requires for javascript with scripts to do whatever you need them to do. Replace elements, usually.

Finally, add behaviors to your pages with NinjaScript. For AJAX, you don't need to change the HTML at all. All straightforward forms and GET links can be converted into to AJAX forms just by specifying the submits_as_ajax() behavior as shown in the top example above. Since you wrote them without AJAX originally, they will continue to work and degrade gracefully without AJAX.

How it works

The short answer is: rebinding. Event delegation is well and good, and if that's all you need, you probably can look elsewhere. My advice is to stick around, though. You get plenty of goodness from NinjaScript, without too much pain.

There are some problems with bubbly delegation though. Event delegation doesn't solve the problem of modifying elements. You can't delegate watermarking. And you can't (easily) store data on an element-by-element basis while you're delegating.

NinjaScript builds and applies behavior objects to all the elements selected in the $.behavior block. When they're applied, behaviors modify their host with their transform function (adding tooltips, changing no-input forms into links, pulling input labels in as watermarks, etc.) and apply event handlers directly to the element. They also mark the element as having been enriched with behavior, so that we don't try to re-apply behavior.

Now, when the DOM is modified, the collection of behaviors is told to reapply all the behavior objects. Any elements already enriched get skipped, since we know they were already enriched. One nifty consequence is that elements that weren't around for the initial application get found this time and get behaviors applied.

How do we know the DOM was modified? Believe it or not, there are events that a lot of browsers generate when the DOM is changed, and we listen for those. Plus, when a NinjaScript behavior does an AJAX call, it assumes that the resulting javascript execution changed the page, and it fires it's own event based on that.

Consequences of adopting NinjaScript

Most noticeably, NinjaScript event handlers are a little different from normal event handlers. We assume that events shouldn't bubble and shouldn't fire their normal behavior - you can save a little code not worrying about suppressing those. NinjaScript handlers also are called in the context of the behavior object they're attached to. This means that "this" is not the element receiving the event; "this" is the behavior object that is unique to the element. You can stash information about the behavior in there during the transform step, or maintain state for the element between events. The event handler receives not only the event record (with the original target, etc.) but also the element it's attached to as arguments. All in all, changing a standard event handler over to a NinjaScript handler isn't terribly difficult.

You should also be aware that NinjaScript really does need to know when the DOM changes. Everywhere but IE, you should be okay without doing anything - any DOMNodeInserted or DOMSubtreeModified that reaches the root node should trigger rebinding. To be safe, call $.ninja.tools.fire_mutation_event() and everything should be fine.

There exists a (small) set of utility functions at $.ninja.tools - right now there's only:
$.ninja.tools.perform_ajax_submission(form_or_anchor) - Submits form data over AJAX, evals the response and triggers a rebind.

Directions for the Future

NinjaScript really wants more stock behaviors. Already on the TODO list are:

make_watermark
Editable table rows - it'd be nice to be able to have AJAX checkboxes and draggable order
Fading messages - complete with a backlog and roll back - "Wait, what was that?"
Undoable edits - there's likely a lot of backend support this needs.

Behaviors should be mergable. At the moment, the application of two behaviors to the same element is undefined, in that their order isn't predictable.

Convection: self-hosted secure file exchange in Rails

Evan — Wed, 09 Jun 2010 02:54:03 +0000

Introducing Convection, an open-source (MIT License) project of Logical Reality Design. Need to swap files with clients or collaborators, but don't want to (or can't) trust those files to Amazon or sendbigfiles.com? Want fine-grained control over which users can see which files? Try Convection.

Lots of file exchange services exist, for example SendBigFiles.com etc. However, all of these services are hosted on someone else's hardware, and most of them share files by transferring URLs -- usually via email -- without good access control or authorization schemes.

We built Convection because a client needed to transfer files with other companies, but they needed to host the system themselves because the contracts they hold with their own clients require them not to store data on services that they don't control. The specifications Convection was built around were:

Hosted on our own server.
Downloads require a login, and files cannot be shared by email.
Users must log in to download files or see available files.
User accounts can be grouped, groups can be managed.
Files can be shared with an entire group.
Files uploaded by users default to minimal permission - visible only to the uploader and to admins.
All communications over SSL. (we made this optional)

Installing and hosting Convection

To run Convection, you will need a webserver capable of running a Ruby on Rails application, and a database. Setting such a thing up is beyond the scope of this post. If you have a Dreamhost account, you can set up a Rails-capable domain with a couple of clicks in their web panel. In addition to the server, you will need to set up a database (we have only tested MySQL, but Convection should work with any SQL database for which Rails/ActiveRecord has a supported adapter, including PostgreSQL and Oracle), and initialize the database with these two commands:

  > rake db:migrate RAILS_ENV=production
  > rake db:seed RAILS_ENV=production

This will generate the tables necessary for Convection to run, and create a pair of initial demo users "admin" and "user", both with password "foobar".

If you are setting up a server yourself, there are plenty of guides to deploying Rails on the web. Much of our own guide to deploying CruiseControl.rb can be used to set up any Rails application on Slicehost or any other Ubuntu Linux hosting provider.

Let me know if you're trying to deploy Convection and having trouble: if we know people are using it we may put effort into making it easier to deploy and install, and write a more thorough guide.

A few other links that may help you with deploying a Rails application, depending on your environment:

If you Google around you may find plenty of other links relevant to your particular environment.

Configuring Convection

If you log into your running Convection application as an administrator (initial user "admin", password "foobar"), an Admin Tools utility will appear in the right hand column. From here, you can access tools for creating users, and groups, and the general site configuration.

In general site config, you can set your site name and logo, set whether or not the site requires SSL access (Note: your server must already support SSL!) outgoing email and email notification preferences, add Google analytics, and an assortment of other site configuration operations that are mostly self-explanatory.

Upload progress bar: experimental feature.

If your site hosts large uploads that take a while to transfer, you can try our experimental tools to provide an upload progress bar to the user. This tool will only work if your site is served by Apache, and requires installing and configuring an optional module for Apache.

To enable this tool, follow the instructions in the README file and associated links, and turn on the progress bar setting in site preferences.

Helping us improve Convection

Convection is currently in version 1.1.4 and has been in production in two places (that we know of) for about five months as of June 8, 2010.

Please let us know if you are using Convection and enjoy it (or don't). Feel free to request features or alterations, but Convection is open source, so also please consider contributing if you have ideas!

Transactional Testing for Multiple Databases in ActiveRecord

Judson — Fri, 19 Mar 2010 06:36:27 +0000

We've been working on an app that needs to stand astride two databases - one local DB for the app itself, and another with restrictive policies about modifications that is nonetheless authoritative on many subjects. There's a fair amount of tricky interaction between the two, and testing has been a delightful challenge.

We're using the use_db plugin, and all it takes to make testing transactions happen around multiple DBs is:

In: spec/spec_helper.rb

require 'override_test_callbacks'

My concern comes from the fact that this is a direct and unfiltered monkeypatch on ActiveRecord::TestFixtures. So it relies on use_transactional_fixtures (which could certainly be used without using actual fixures, granted), and if the test transaction code moves within Rails, that's another integration to worry about. Or if we add a spec that doesn't wind up making ActiveRecord::TestFixtures load... Or if we decide to use something other than use_db...

So instead I'm using:

Spec::Runner.configure do |config|
  config.prepend_before do
    (UseDbPlugin.all_use_dbs - [ActiveRecord::Base]).each do |db|
      db.connection.increment_open_transactions
      db.connection.transaction_joinable = false
      db.connection.begin_db_transaction
    end
  end
 
  config.append_after do
    (UseDbPlugin.all_use_dbs - [ActiveRecord::Base]).reverse.each do |db|
      db.connection.rollback_db_transaction
      db.connection.decrement_open_transactions
    end
  end
end

If we weren't already using transactional fixtures, I might pull out the - [ActiveRecord::Base]. And if we were to change off of use_db, there's one place to change the transaction code. Finally, there's much less dependence on the innards of ActiveRecord - only it's published API.

Danger: ActiveRecord, param hashes, and symbol keys

Evan — Wed, 10 Mar 2010 22:20:17 +0000

Here's a little foible of ActiveRecord that cost me over an hour today. AR accepts both symbol keys and string keys when specifying attributes. Both of these are valid ways of mass assigning attributes to a Rails model:

MyModel.new(:field_1 => 'foo', :field_2 => 'bar')
MyModel.new('field_1' => 'foo', 'field_2' => 'bar')

It's convenient, often, to not have to worry about whether your keys are symbols are strings since they get converted around a bit when you pass parameters. The downside of this, however, is that it will happily accept BOTH without complaining, and will quietly default to the symbol key regardless of the order you specify them in:

>> model = MyModel.new(:field_1 => 'foo', 'field_1' => 'bar'); nil; 
>> mymodel.field_1
=> 'foo'
>> model = MyModel.new('field_1' => 'foo', :field_1 => 'bar'); nil;
>> mymodel.field_1
=> 'bar'

Okay, so that's kinda sloppy. Bad ActiveRecord! No Biscuit!

This can cause serious confusion for the unwary. When ActionController hands us a params hash, it always has String keys, like this:

>> eval params
=>  { 'article' => { 'title' => 'Awesome blog post', 'body' => 'I will make you smart' } }

But most of us, canonically, specify params and default AR values with symbols, like this:

   post :article => {:title => 'Awesome blog post', :body => 'I will make you smart'}

So we get used to thinking about them as symbols.

This means we can make mistakes like this one I made recently. Consider this block of code for a shopping cart model that pre-fills some fields for an associated Payment by pulling the address from the user's profile, to save the user re-typing their address:

class ShoppingCart < ActiveRecord::Base
  has_one :payment
 
  def build_default_payment(options = {}) 
    #prepopulate the billing address from the profile and merge
    #with params passed into options
    build_payment(prepopulated_fields.merge!(options)    
  end
 
  def prepopulated_fields
    if (addr = self.person.address)
      {
        :billing_address_1 => addr.line_1,
        :billing_address_2 => addr.line_2,
        :city => addr.city,
        :state => addr.state,
        :zip => addr.zipcode
      }
    else
      {}
    end
  end
end

Looks great, right? And if the user's address has a nil field (like no city, or no line_1), it will get overwritten by the hash merge.

Except not. I specified symbol keys in prepopulated_fields, but the hash getting passed to build_default_payment's 'options' argument has string keys, because it's coming from params. So the merge doesn't overwrite the value for :line_1, it simply adds a new key 'line_1'. So, if a user has a profile address but hadn't entered a line_1 (just city and state), and then manually entered line_1 in the payment form to submit, the Payment build during the create action was getting this hash:

build_payment({
   :line_1 => nil,
   :city => 'Pasadena',
   :state =>'CA',
   :zipcode => '91106'
   'line_1' => '100 Main St.'.
})

ActiveRecord was respecting the :line_1 => nil from the profile, and not the 'line_1' => '100 Main St.' from params. This meant that the user couldn't make payment! The payment had validates_inclusion_of line_1, and even though it was typed into the form it was getting ignored because of the nil from his profile address. Very frustrating for a user to manually type in a billing address and get back "Address Line 1 can't be blank." on every submit!

Nasty ... this one took a while to figure out. Beware of this little foible of ActiveRecord!

HOWTO: Setting up CruiseControl.rb on Slicehost

Evan — Sun, 21 Feb 2010 00:47:46 +0000

Continuous Integration is a key tool for collborative development, and CruiseControl.rb is the tool of choice for many Ruby and Rails teams, including us at Logical Reality.

Unfortunately, setting up CC.rb for a team can be a relatively frustrating experience: this guide (the first of a series of HOWTOs by LRD) will walk you through every step of setting up a team instance of CruiseControl.rb on a low-cost server from Slicehost.

Step 1: Lease a Ubuntu Slicehost account

I recommend a 384 slice or a 512 slice, as 256MB or RAM is pretty light for anything involving a Rails application. Our CI server runs on a 512 slice; if you are running it on a smaller slice please let us know how it performs.

I used Ubuntu 9.10 (Karmic) for this post.

Step 2: Create a working user

Slicehost configures slices with an active root account - definitely a Ubuntu no-no - and no user account. Ick! Let's start by creating a user account with sudo access to do everything from. Log in as root using the information Slicehost sends you, run this (replace 'usename' with whatever name you like) and fill in the information it asks for:

 # adduser username

Then edit /etc/sudoers and add this line to the bottom of the file:

username    ALL=(ALL) ALL

Log out, and log back in as the user you've now configured, to make sure it work.

Step 3: Installing packages and gems

Reset your timezone:

sudo dpkg-reconfigure tzdata

Install a whole bunch of packages you'll want for running Rails applications and hosting CruiseControl:

sudo aptitude install locate emacs git-core ruby build-essential \
libopenssl-ruby ruby1.8-dev irb  apache2 apache2-mpm-prefork \
apache2-prefork-dev sqlite3 rubygems mysql-server mysql-client

Go grab a cup of coffee while those install. The mysql install will ask you to set a root password. Do so, and write it down for later use. When all the installs are done, come back and install the ruby gems you'll be needing:

sudo gem install sqlite3-ruby passenger mysql metric_fu reek roodi

Step 4: Assorted server configuration

Add this line to the bottom of your ~/.profile to put your gems in your path:

PATH="$PATH:/var/lib/gems/1.8/bin/"

And source it:

. ~/.profile

Some assorted config: set up the passenger module for Apache, set your hostname, and make /etc/hosts readable. (For some bizarre reason, /etc/hosts was only readable by root on my slice, and that has a tendency to break things down the road).

sudo /var/lib/gems/1.8/bin/passenger-install-apache2-module
sudo emacs /etc/hostname  # set it to "your.hostname.com"
sudo /bin/hostname -F /etc/hostname
sudo chmod a+r /etc/hosts

Step 5: Configure Passenger and Apache

We'll run CruiseControl.rb with Apache and Passenger. Start by enabling the Passenger module. The command below will walk you through a super-easy configuration:

sudo /var/lib/gems/1.8/bin/passenger-install-apache2-module

When the command completes, it will give you three lines to paste into your apache config, they should look pretty much like these below. Put these lines at the top of /etc/apache2/apache2.conf. I included the hostname I set in the previous step as ServerName.

LoadModule passenger_module /var/lib/gems/1.8/gems/passenger-2.2.8/ext/apache2/mod_passenger.so
PassengerRoot /var/lib/gems/1.8/gems/passenger-2.2.8
PassengerRuby /usr/bin/ruby1.8   
 
ServerName your.hostname.com

To set up the application itself, edit /etc/apache2/sites-available/default to look like this:

<VirtualHost *:80>
        ServerAdmin administrator@your-email-domain.com
        DocumentRoot /u/apps/cruisecontrol/public
        RailsEnv production
        RailsBaseURI /
        ServerName 
        ServerAlias your.hostname.com
        SetEnv PATH /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/var/lib/gems/1.8/bin/
 
        ErrorLog /var/log/apache2/error.log
 
        # Possible values include: debug, info, notice, warn, error, crit,
        # alert, emerg.
        LogLevel warn
 
        CustomLog /var/log/apache2/access.log combined
VirtualHost>

Make a home for the app. (I use /u/apps/ as a convention for apps in apache. Use whatever you like, but make sure your DocumentRoot in your config file above matches.)

sudo mkdir -p /u/apps

Step 6: Download and install CruiseControl.rb

Download cruisecontrol.rb from RubyForge (Check for the current version first; it was 1.4.0 when I installed), and give ownership to the web user www-data:

cd /u/apps
sudo wget http://rubyforge.org/frs/download.php/59598/cruisecontrol-1.4.0.tgz
sudo tar -zxf cruisecontrol-1.4.0.tgz     
sudo mv cruisecontrol-1.4.0 cruisecontrol   
sudo chown -R :www-data cruisecontrol

Give environment.rb to the web user; this prevents an Errno::EACCES accessing environment.rb from Passenger (see discussion at this forum post).

sudo chown www-data:www-data config/environment.rb

Turn off the built-in htaccess, it will break Passenger:

sudo mv public/.htaccess public/.htaccess-disabled
cd config  
sudo cp site_config.rb_example site_config.rb

Step 7: Setting up the user environment

CruiseControl.rb prefers, by default, to put project builds in the running user's ~/.cruise directory. This is unfortunate because the standard user for running Apache, www-data, doesn't have a user directory! There are ways to override this, but I've found that they cause significant problems down the line.

An example of the problem is letting CC.rb check out your source code. If you authenticate access to GitHub or another code repository with SSH, CC.rb — running as www-data — won't be able access your repo since www-data doesn't have a ~/.ssh directory to put the keys in!

After much hacking, I came to the unhappy conclusion that the best solution is simply to let CruiseControl.rb have its way and give user www-data a home directory. Boo, hiss, but here we go:

sudo /etc/init.d/apache2 stop   
sudo usermod -d /home/www-data www-data      
sudo usermod -s /bin/bash www-data
sudo /etc/init.d/apache2 start

If you give www-data standard config files as well, then you can set the PATH so that user www-data can find your gems, and you can set up ssh keys so that CruiseControl.rb can securely check out projects from GitHub or whatever source code repository you're using:

sudo cp -r /etc/skel /home/www-data 
sudo chown www-data:www-data /home/www-data
sudo su www-data               
cd
mkdir ~/.ssh  
cd ~/.ssh
ssh-keygen -t rsa
cat id_rsa.pub

Add this line to the bottom of ~/www-data/.profile:

PATH="$PATH:/var/lib/gems/1.8/bin/"

Re-start Apache:

sudo /etc/init.d/apache2 restart

At this point, you should be able to load CruiseControl.rb in a web browser at the IP address given to you by Slicehost, or at the domain name if you've set up DNS and it's resolving. Congratulations, you have CC.rb up and running! One last thing to configure.

Running CruiseControl.rb will have created a configuration directory . ~www-data/.cruise. You'll want to edit ~www-data/.cruise/site_config.rb to set two options. Uncomment and set appropriate values for this line:

 Configuration.email_from = 'cruisecontrolrb@mydomain.com'
 Configuration.dashboard_url = 'http://my.cruisecontrolrb.host/'

Okay, it's time to get a project installed!

Step 8: Setting up your first project

I'll use Logical Reality's open-source project, Convection, as an example project for CruiseControl.rb. This works best if you run it as user 'www-data'.

The command for adding a new project is really simple:

cd /u/apps/cruisecontrol 
sudo su www-data
./cruise add Convection -r git://github.com/LRDesign/Convection.git -s git

This will set up the build in ~www-data/.cruise/projects/Convection.

Create a test database for the application. For Convection, I'm going to use mysql, and prefix my database name with 'ci' for Continuous Integration.

mysqladmin -u root -p create ci_convection

We don't want to put a functioning database.yml in our GitHub repository, but at the same time we want CruiseControl.rb to be able to build and test the app without help from the user. For all our Rails projects, we use a custom rake task that generates a database.yml from command-line arguments, then rebuilds the database, run the specs, and generate output with metric_fu. For an example of how to do this, look at our integration.rake and ERB database.yml template from Convection.

To configure CruiseControl.rb to run Convection this, we need to add that task to the configuration file for this project. Edit ~www-data/.cruise/projects/Convection/cruise_config.rb so that it looks like this:

 
Project.configure do |project|
 
  # Send email notifications about broken and fixed builds to email1@your.site, email2@your.site (default: send to nobody)
  project.email_notifier.emails = ['sysadmin@lrdesign.com', 'judson@lrdesign.com']
 
  # Set email 'from' field
  project.email_notifier.from = 'sysadmin@lrdesign.com'
 
  # Build the project by invoking rake task 'custom'
  # project.rake_task = 'custom'
 
  # Build the project by invoking shell script "build_my_app.sh". Keep in mind that when the script is invoked,
  # current working directory is [cruise data]/projects/your_project/work, so if you do not keep build_my_app.sh
  # in version control, it should be '../build_my_app.sh' instead
  # project.build_command = 'build_my_app.sh'
  project.build_command = 'rake ci:run[localhost,root,,ci_convection] --trace RAILS_ENV=test'
 
  # Ping Subversion for new revisions every 5 minutes (default: 30 seconds)
  # project.scheduler.polling_interval = 5.minutes
end

Step 9: There is no step nine!

Okay, so it's not the simplest thing in the world to set up. But if you've done everything above correctly, you should have a running server your team can use for continuous integration. If you've included metric_fu in your build task, you should get both test output and a wealth of useful code metrics.

Did this sequence work for you? Did I omit a step or misspell a command? Let me know in comments, and I'll update/correct the post.

RailsTutorial.org launched

Evan — Mon, 14 Dec 2009 18:56:28 +0000

The new Ruby on Rails Tutorial book and website by Michael Hartl has launched at RailsTutorial.org. Hartl is the author of RailsSpace and cofounder of the Insoshi Ruby on Rails social networking platform.

Logical Reality did the logo and layout design work for Rails Tutorial.

Using link_to (or other helper methods) in a controller

Evan — Wed, 06 May 2009 23:40:10 +0000

This one was a big aggravator to me lately. I have one controller that needs to call link_to and url_for, which are normally helper methods you'd call from a view. However, in this case during certain modifications to a record, I actually need to append user-visible HTML links to a block of HTML stored in that object, or possibly another one.

Specifically, I needed to put annotations in the description of a work order object that said, for example "this work order was escalated from Problem Report 293. This was done in a create action that redirected at the end and never rendered a view, so I really did need to generate that link in the controller. And for consistency with the rest of the application, I wanted to generate the link with link_to(@task).

Now, ActionView::Helpers::UrlHelper is not loaded in a Rails controller, even if you've put helper :all in application.rb (application_controller.rb in newer versions). So, when I tried to use link_to in the controller, I got an error:

NoMethodError: undefined method `link_to' for # /Users/evan/Development/Ruby/eclipticdb/app/helpers/tasks_helper.rb:64:in `task_link' /Users/evan/Development/Ruby/eclipticdb/app/controllers/tasks_controller.rb:103:in `escalate' ... etc ...

The first fix - but with a problem

A year ago, I fixed this just by adding include ActionView::Helpers::UrlHelper at the top of that controller. This worked great ... for a while.

Lately, I've been rewriting this application into a RESTful style - it had previously been a controller/action style application. In the process, I started linking things with resource paths and polymorphic paths ... a lot of link_to @task and edit_polymorphic_path(@task) sorts of bits. And these started breaking. I began seeing this mysterious error:

Error:

You have a nil object when you didn't expect it! The error occurred while evaluating nil.url_for


... some code here that calls a link_to ...
Trace of template inclusion: /tasks/_task_panel.html.erb, /tasks/_task_tabbed_panel.html.erb, /tasks/index.html.erb
RAILS_ROOT: /Users/evan/Development/Ruby/eclipticdb

Application Trace | Framework Trace | Full Trace

vendor/rails/actionpack/lib/action_view/helpers/url_helper.rb:71:in `send' vendor/rails/actionpack/lib/action_view/helpers/url_helper.rb:71:in `url_for'

This one was a real bitch to debug, I have to say. The line in question that was failing in url_helper.rb said this: url = @controller.send(:url_for, options). Clearly, @controller was nil ... which was very bizarre, because I never interact with that instance variable anywhere.

I thrashed around trying to find the cause of this error for quite some time. Eventually I realized that the link_to method was only failing when called from a view in TasksController, and not from any other controller. And then I realized that TasksController was the one where, a year ago, I'd put include ActionView::Helpers::UrlHelper at the top. Somehow, including that helper in the controller was nullifying @controller when those helper method we called from within the view. I removed the include and my polymorphic and resource links all started working again.

Now back to the original problem!

Of course, that then left me back with the problem I'd had a year ago ... needing to use link_to from within the controller and having no way to do it. After a fair bit of googling around I found this post from Neeraj, which had an interesting approach -- but a commenter had suggested a much easier solution:

[sourcecode language='ror']self.class.helpers.link_to[/sourcecode]

I'm not certain where one would find this in the docs, but it does seem to have solved my problem for now. Onward and upward!