Cloning your remote database to local with Capistrano

Handy file: Capistrano recipe for cloning your remote (production) mysql database to your local box, and vice versa.

One of the more useful recipes for Capistrano that I've come across is this one by netzpirat. It's designed to give you simple commands that will dump the mysql database on your production server to a file, scp the file back to your development workstation, and import it into your local mysql database in a single operation. Very sexy, very powerful. (It also has commands for doing the reverse, and for uploading/downloading files as well - useful for things like paperclip attachments.)

Unfortunately, for my purposes, it failed right out of the box in two different ways:

It only pulls your database name, db login, and password from your database.yml, and then (on the remote server) tries to connect to the database on a local socket. Most of my databases are hosted on remote boxes.

It assumes your production database credentials exist in your local database.yml, which they really shouldn't in any decent deployment. Especially not if you are hosting your code in an outside repository.

Slightly improved version

So, I created this improved version, which pulls the remote DB credentials from the database.yml file on the remote server (assuming you've put them in #{shared_path}/config/database.yml), and which will also use the hostname, if any, from your database.yml when it executes mysqldump.

Let me know if you have any trouble with it and I'll correct any errors in the gist.

Transactional Testing for Multiple Databases in ActiveRecord

We've been working on an app that needs to stand astride two databases - one local DB for the app itself, and another with restrictive policies about modifications that is nonetheless authoritative on many subjects. There's a fair amount of tricky interaction between the two, and testing has been a delightful challenge.

We're using the use_db plugin, and all it takes to make testing transactions happen around multiple DBs is:

In: spec/spec_helper.rb

require 'override_test_callbacks'

My concern comes from the fact that this is a direct and unfiltered monkeypatch on ActiveRecord::TestFixtures. So it relies on use_transactional_fixtures (which could certainly be used without using actual fixures, granted), and if the test transaction code moves within Rails, that's another integration to worry about. Or if we add a spec that doesn't wind up making ActiveRecord::TestFixtures load... Or if we decide to use something other than use_db...

So instead I'm using:

Spec::Runner.configure do |config|
  config.prepend_before do
    (UseDbPlugin.all_use_dbs - [ActiveRecord::Base]).each do |db|
      db.connection.increment_open_transactions
      db.connection.transaction_joinable = false
      db.connection.begin_db_transaction
    end
  end
 
  config.append_after do
    (UseDbPlugin.all_use_dbs - [ActiveRecord::Base]).reverse.each do |db|
      db.connection.rollback_db_transaction
      db.connection.decrement_open_transactions
    end
  end
end

If we weren't already using transactional fixtures, I might pull out the - [ActiveRecord::Base]. And if we were to change off of use_db, there's one place to change the transaction code. Finally, there's much less dependence on the innards of ActiveRecord - only it's published API.

Danger: ActiveRecord, param hashes, and symbol keys

Here's a little foible of ActiveRecord that cost me over an hour today. AR accepts both symbol keys and string keys when specifying attributes. Both of these are valid ways of mass assigning attributes to a Rails model:

MyModel.new(:field_1 => 'foo', :field_2 => 'bar')
MyModel.new('field_1' => 'foo', 'field_2' => 'bar')

It's convenient, often, to not have to worry about whether your keys are symbols are strings since they get converted around a bit when you pass parameters. The downside of this, however, is that it will happily accept BOTH without complaining, and will quietly default to the symbol key regardless of the order you specify them in:

>> model = MyModel.new(:field_1 => 'foo', 'field_1' => 'bar'); nil; 
>> mymodel.field_1
=> 'foo'
>> model = MyModel.new('field_1' => 'foo', :field_1 => 'bar'); nil;
>> mymodel.field_1
=> 'bar'

Okay, so that's kinda sloppy. Bad ActiveRecord! No Biscuit!

This can cause serious confusion for the unwary. When ActionController hands us a params hash, it always has String keys, like this:

>> eval params
=>  { 'article' => { 'title' => 'Awesome blog post', 'body' => 'I will make you smart' } }

But most of us, canonically, specify params and default AR values with symbols, like this:

   post :article => {:title => 'Awesome blog post', :body => 'I will make you smart'}

So we get used to thinking about them as symbols.

This means we can make mistakes like this one I made recently. Consider this block of code for a shopping cart model that pre-fills some fields for an associated Payment by pulling the address from the user's profile, to save the user re-typing their address:

class ShoppingCart < ActiveRecord::Base
  has_one :payment
 
  def build_default_payment(options = {}) 
    #prepopulate the billing address from the profile and merge
    #with params passed into options
    build_payment(prepopulated_fields.merge!(options)    
  end
 
  def prepopulated_fields
    if (addr = self.person.address)
      {
        :billing_address_1 => addr.line_1,
        :billing_address_2 => addr.line_2,
        :city => addr.city,
        :state => addr.state,
        :zip => addr.zipcode
      }
    else
      {}
    end
  end
end

Looks great, right? And if the user's address has a nil field (like no city, or no line_1), it will get overwritten by the hash merge.

Except not. I specified symbol keys in prepopulated_fields, but the hash getting passed to build_default_payment's 'options' argument has string keys, because it's coming from params. So the merge doesn't overwrite the value for :line_1, it simply adds a new key 'line_1'. So, if a user has a profile address but hadn't entered a line_1 (just city and state), and then manually entered line_1 in the payment form to submit, the Payment build during the create action was getting this hash:

build_payment({
   :line_1 => nil,
   :city => 'Pasadena',
   :state =>'CA',
   :zipcode => '91106'
   'line_1' => '100 Main St.'.
})

ActiveRecord was respecting the :line_1 => nil from the profile, and not the 'line_1' => '100 Main St.' from params. This meant that the user couldn't make payment! The payment had validates_inclusion_of line_1, and even though it was typed into the form it was getting ignored because of the nil from his profile address. Very frustrating for a user to manually type in a billing address and get back "Address Line 1 can't be blank." on every submit!

Nasty ... this one took a while to figure out. Beware of this little foible of ActiveRecord!