Thoughts on the Github Hack

Over the weekend, a young coder demonstrated a security vulnerability in github.com - one with wide-reaching implications. An early demonstration is at: https://github.com/rails/rails/issues/5239.

Our friend went on to make several updates to github as he experimented/demonstrated the vulnerabilities, got his account suspended, reinstated and set off a firestorm of criticism every which way.

I was ready to put it all into the "someone is wrong" pile, until I ran across this pull request on the Rails core: https://github.com/rails/rails/pull/4062.  That's some mid-90's Microsoft style arrogance right there, and on the off chance that anyone is having trouble seeing it, I figured I might add my breath to the maelstrom.

First of all: it was not right to hack github.  It's not okay to ignore the intent of security, no matter how weak the enforcement.  Much better to have pointed out the vulnerability to github, although for sure the fame wouldn't have been as bright (which is why I'm pointedly not referring to him by name in this post.)  Given github's track record, I think there's a pretty good chance they would have come clean, admitted the fault, as well as crediting its reporter.  But that's complete supposition.

That said, I don't think it's legitimate to consider Github a blameless victim.  The flaw in Rails that was exploited is well known, and well reported, and easy, if irritating, to fix.

The technical aside here is pretty simple.  In a file in config/initializers add:

ActiveRecord::Base.__send__(:attr_accessible, nil)

Then you need to white-list mass-assignable attributes in your models:

attr_accessible :name, :body, :whatever

And keep an eye on your logs for

WARNING: Can't mass-assign protected attributes: :blah

Which is a sign that you might need to add an entry for :blah into the respective model.

All pretty simple.  There are a couple of other notes, like "don't allow reference fields (e.g. :person_id) to be mass assigned" but that's the meat of it.  Put the initializer in your generator (that's much harder) and you never have to think about it again.

So, Github didn't put a simple, well-reported fix into their code.  Is that so bad?  I think so.  Github not only invited the developer community to trust them with the products of their labor, pretty much ousting SourceForge from that position in the process and firming up a development environment choice for open source work (i.e. "use git for version control"), it also invites developers to trust them with secrets.  Specifically, the secret contents of client repositories.  Heck, they get you to pay for the privilege.   So, in short, Github is taking money to keep secrets.  And by not covering a known security hole in a default Rails deploy, they were failing to uphold the trust of their paying customers.

I think Github was letting us down pretty badly.  I think an overzealous coder did a bad thing to bring that to light, but you can't argue that Github should be surprised or is blameless.  Two bad things, no one is blameless.

But the last straw for me was reading the Rails core teams' replies to a pull request to set the default for whitelisting attributes in Rails 4.0 to 'true.'  (After previous discussion concluded that making the change for 3.2 would be "too disruptive.")  That having to do attr_accessible for every model was "a lot of paperwork" - the final commend is @dhh's "I don't like this. -1"  Which is to say: we would rather put unsanitized data into the database than do the bare minimum of manual review.  And that's pretty lame.

RSpec 2.0 and before/after hooks

As of RSpec 2, the configuration interface for RSpec changed dramatically.  What used to look like:

Spec::Runner.configure do |config|
config.prepend_before(:each, :type => :controller) do
...
end
end

Now looks more like:

RSpec::configure do |config|
  config.before(:each, :line => 153) do
  end
end

One significant and interesting change is the way that before hooks are processed.  Specifically, the #before, #after, and #around methods are now part of the Hooks module, which is included in both ExampleGroup and in Configuration, so you call configure.before in exactly the same way as you do within a describe block.  Normally, you pass :each or :all, which sets the scope under which the hook will be called, but Hooks inspects the arguments for filtering metadata regardless of where you call it - I don't know that you'd want filter within an ExampleGroup, but you could...

Unfortunately, as cool as the metadata filtering capabilities are they aren't, as far as I can tell, very well documented.  The process of extracting the metadata lives in it's own :nodoc: limbo, and the attachment of metadata to a particular example is scattered throughout the RSpec code.  This, then, is an attempt to pick that apart.

Extracting Filters

When you call Hooks#before, for example (#after and #around work fundamentally the same way), the args are examined and two things are extracted:

A scope, which is the :each, :all, or :suite specification.

A metadata filter hash.  Normally, you call #before(:each, {:hash => [:of, :metadata]}), but you can instead do something like before(:all, :symbol) which will result in a metadata filter like {:symbol => true}

Again, probably if you need to add metadata inside of a describe block, you are Doing Something Wrong, but maybe there's a good reason.  The extreme (excessive?) flexibility of RSpec metadata and filtering does open up a lot of interesting possibilities.

Filter Matching

The metadata filter is used to decide if the hook should be run for a particular example block that it might apply to.  As such, it's a remarkably powerful filtering system, although there's a lot of assumptions about it's format that you need to bear in mind.

The actual mechanics of the metadata filtering happen in RSpec::Core::Metadata#apply? and #apply_condition - there's a long chain of delegation and extra-meta-programming that leads there.

The upshot is that your metadata filter will be compared to the metadata on the example key/value pair by pair, like this:

• A regular expression in the filter will match against the appropriate value for the example.
• If you pass :line_number => 17, Rspec will check to see if the example includes line 17, much like running rspec filename_spec.rb:17
• Any other Fixnum will be compared with == to the value in the metadata
• Anything else gets compared with == to the value in the metadata, after both values have been converted to a string.
• A proc like {|value| ... } will get the value of the key, and can return true for a match.

Filters can nest Hashes, which will be compared to nested Hashes in the metadata.  In other words, if you want to be able to match for metadata like

{...,  :example_group => {..., :full_description => "A very long winded example of the group", ...}, ...}

You can do something like:

before(:each, {:example_group =>{:full_description => /long winded/}})

it "should do something useful, someday", :pending => "Not this day, though"

Which is much faster than using the pending method call inside the block, and can be applied to a describe block to make the whole thing pending - especially handy when you have a before block inside that is causing problems.

In the same token, the example given in RSpec 2 documentation and announcement posts has been doing something like:

it "should not be taking this looooong", :slow => true

Since metadata can also be used to filter examples, you could use this to pull out the examples that take forever from your all-the-time specs, and run them only before a push, for instance.

What Metadata Does RSpec Give Us?

Probably the best way to figure that out is this very pragmatic approach.

A Useful Trick

Very useful for experimenting with metadata is that the proc form of the metadata has a special case: if the proc takes two arguments, the whole metadata hash will get passed into the proc, so you can inspect it at leisure.  The snippet looks like:

require 'pp'
before(:each, :bogus => proc{|val, all| pp all}) {}

{
:execution_result=>{:started_at=>Tue Jun 07 14:13:46 -0700 2011},
:type=>:controller,
:full_description=>"UserSessionsController should be authorized",
:description=>"should be authorized",
:example_group=>
  {
  :full_description=>"UserSessionsController",
  :file_path=> "spec/controllers/user_sessions_controller_spec.rb",
  :describes=>UserSessionsController,
  :description=>"UserSessionsController",
  :block=> #
,
  :line_number=>3,
  :caller=> [ ... the whole backtrace of the group ... ]
  },
:caller=> [ ... the backtrace of the example ...]
}

One of the cool-but-problematic things about metadata in RSpec is that it get's added and updated all over the codebase, and constantly over the lifecycle of an example run and extensions (like Rspec-Rails) add their own fields and values, so it's very hard to have formal documentation for what you can match.  Also, somewhat troubling, is that none of these fields are an explicit part of the RSpec API, and so might change with very little notice.  It seems like the best way to manage working with the metadata is with the above pragmatic approach.

Extending form_for in Rails 3 with your own methods

At LRDesign, we have a bunch of internal tools to make laying out Rails views more consistent. I recently upgraded and improved some of ours for Rails 3, and published them as a gem. (The published / open source ones are available at https://github.com/LRDesign/lrd_view_tools, if you're interested). One of the handy techniques we figured out (poring through the Rails code) is how to correctly add a method to FormBuilder so that you can properly use it inside a form_for block.

An example method added to forms:

Since I nearly always want <input> and <label> tags at the same time, I created a labeled_input method that lets me say this (in HAML):

= form_for(@book) do |f|
    = f.labeled_input :title
    = f.labeled_input :author
    = f.labeled_input :price

to get:

<form action="/books/new">
  <div class="labeled_input">
    <label for="book_title">Title:</label><input id="book_title" name="book[title]" type="text" />
  </div>
  <div class="labeled_input">
    <label for="book_author">Author:</label><input id="book_author" name="book[author]" type="text" />
  </div>
  <div class="labeled_input">
    <label for="book_price">Price:</label>
    <input id="book_price" name="book[price]" type="text" />
  </div>
</form>

Combined with some default CSS code in our application template that aligns the <label>s and <input>s in columns, this saves us a couple of hours setting up clean-looking forms on every new project, while significantly shortening and prettifying our view templates. (Markup Haiku, just like HAML intended.)

Implementing the extension in Rails 3

The code that handles form_for in Rails 3 is rather dense and incomprehensible and takes a while to pore through. Here's the short version to understanding it so you can add your own methods to FormBuilder properly. Since we dug through it, hopefully this will save others some time. The only Rails file you care about for this purpose is actionpack-3.0.x/lib/action_view/helpers/form_helper.rb.

• module ActionView::Helpers::FormHelper defines a bunch of helpers, like label, text_field, etc. that define helpers you use outside of a form_for. For example, text_field(@user, :title) calls this version of the helper.
• class ActionView::Helpers::FormBuilder is what's used to define the helpers you run inside a form_for. It works automatically via metaprogramming ... when loaded, it finds each helper in FormHelper (except for a few) and defines a similarly named method in FormBuilder. form_for(@user) { |f| f.text_field(:title) calls this version of the helper, which basically just calls the FormHelper version but passes the FormBuilder's @object_name as an additional first argument. In version 3.0.7, this metaprogramming happens on lines 1131-1141 of form_helper.rb.
• As a result, if you were to write a new helper in ActionView::Helpers::FormHelper that uses the same argument structure as the pre-built ones, you'd automatically get both kinds of helper. However, if you're writing your own plugin or gem and injecting new helpers, this won't happen because by the time you inject your method FormBuilder will have already done its metaprogramming (it happens when the file is loaded).
• The solution to this is that your gem needs to do the second half - defining the FormBuilder version of the helper - itself. I'll put an example below.
• Most of the helper methods work by instantiating InstanceTag, a local one-size-fits-all class to emit a form tag, and then calling the appropriate method for the kind of tag that's wanted, like to_text_field_tag. It's very confusing why the Rails team decided to do one class for InstanceTag and a bunch of different methods, rather than make subclasses of InstanceTag for each kind of tag they want; an odd OOP decision, but that's what we've got.
• InstanceTag itself has only one line: it includes InstanceTagMethods, a model that defines all the methods for the class, and which isn't used elsewhere.

So to implement a FormBuilder method yourself that you can use inside a form_for, the best way is to inject your method inside FormHelper, and then call that from a method you inject into FormBuilder. This gives you both versions of the method, in the same structure that Rails defines them. You could do this either in a helper file directly in your application, or in a gem (like we have) so you can reuse your form helpers in more than one projects.

An example implementation.

Here's a simplified construction of the labeled_input method we use at LRD. This one just emits a label and a text field and wraps them in a <div>.

Start by defining the helper:

module LRD
  module FormHelper
    def labeled_input(object_name, method, options = {})
      input = text_field(object_name, method, options)
      label = label(object_name, method, options)
      content_tag(:div, (label+input), { :class => 'labeled_input' }
    end
  end
end
ActionView::Helpers::FormHelper.send(:include, LRD::FormHelper)

This will successfully define labeled_input that you can use outside of a form_for.

Now add the FormBuilder version:

To get it working inside of a form_for, you need to add a similar method to ActionView::Helpers::FormBuilder. As mentioned above, Rails does this automatically for its own FormHelper methods using a metaprogramming approach. But since that has already happened by the time your code can inject into FormHelper, you have to do it yourself. The solution we used is to make our own FormBuilder module that manually defines the labeled_input method in the same format that FormBuilder would have done, and then auto-include that into FormBuilder when our own FormHelper module gets included. Add this stuff to the above code block:

# Inside LRD::FormHelper, add this method:
def self.included(arg)
  ActionView::Helpers::FormBuilder.send(:include, LRD::FormBuilder)
end
 
module LRD::FormBuilder
  # ActionPack's metaprogramming would have done this for us, if FormHelper#labeled_input 
  # had been defined  at load.   Instead we define it ourselves here.
  def labeled_input(method, options = {})
    @template.labeled_input(@object_name, method, objectify_options(options))
  end
end

In practice, our labeled_input method is much more complex; it handles other input types, can add instructional comments/notes to the field, and can accept a block if you want to put something other than an <input> where the text field normally goes. This guide should get you started to writing your own form_for methods quickly, but if you want to see how to do more complex things, check out the full version.

Adding more input types or other tags.

If you wanted to add an entire different tag or input type (as opposed to combining different ones, the way labeled_input does), you would probably start by building a module that you inserted into InstanceTag or InstanceTagMethods. It should define a method like MyInstanceTagModule#to_some_funky_tag() in parallel with to_input_field_tag().

Testing it with rSpec 2

Another challenge we faced was writing specs for labeled_input's behavior. It's a bit of a trick because we needed to instantiate ActionView and render some templates to check the output, but rspec-rails is written with the assumption that you will be loading an entire rails project and all the rails gems. If you want to spec just a view helper, you need to load a bunch of rspec-rails's files one by one, and then manually include RSpec::Rails::ViewExampleGroup into RSpec's configuration. We may write a separate post on this process in the future, but in the meantime, take a look at lrd_view_tools' spec_helper file and example spec for labeled_input to get the sense of it.

A Security Insight

I may just be re-discovering an idea that's widely known, but I realized recently why cryptography is so important to computer security.  When there's no there there, the only physical constraint is time, and cryptography is a way to leverage that physical constraint into a security constraint.

Arguably, the fundamental problem of security (analogous to the fundamental problem of economics being scarcity) is constraining bad actors.

Good actors function in a society where their constraints are moral, ethical and legal.  We find that we can accomplish great things when we rely on these constraints, but the trade off is that we depend on them as universal constraints.  We can have a powerful dynamic economy, so long as no one commits fraud, or insider trading, etc.  Realistically, we can tolerate a certain amount of violation, but everyone needs to know the rules and at least make some attempt to play by them.

The weakness of these constraints is that they are volitional: we are only constrained by them by conscious acceptance.  As an upside, they're cheap to implement.  (Enforcement being a completely separate matter.)

Enter the bad actor.  Mallory (as he is sometimes known) recognizes that, if he's willing to disregard the damage done by violating the volitional constraints that everyone else agrees to, he can make huge gains for himself.

Recognizing that Mallory and his ilk will not accept volitional constraints, we remove the choice in the constrain - Mallory cannot help being a part of the physical universal and bound to laws of physics.  So, we enforce constraints by their application.  We take advantage of the fact that solid objects cannot co-locate (well before we understood why this would be so) by building walls and vaults and fences and keeps.  We use the fact that matter has continued existence to make receipts and licenses and ledgers.  We use the fact that matter cannot be created from nothing (and therefore cannot be perfectly duplicated) with signatures, seals, coinage, keys...

But on a computer, none of those things are true.  There is only data (not even a separate 'code' as much as it's attempted), which is only a pattern, and it exists in the province of a machine designed to manipulate patterns.  The duplication and removal of a pattern is trivial, and to move it from one place to another not much more difficult.

The most remaining physical constraints (that I can think of) are time, connection, and the properties of the hardware the computer system runs on (e.g. power consumption, waste heat, MTBF of components.)  We use connection all the time: firewalls and airgaps, black and white machines, etc.  The trouble is that it is cumbersome to manage connection perfectly, and it tends to be poorly scoped to the action under constraint.  The properties of the hardware are difficult to leverage, since it is often feasible to circumvent them as a constraint - Mallory moves processing to his hardware and throws money at it.

But time, if properly exploited, can be a very effective constraint.  We all have the same amount of time, and cryptographic algorithms can leverage a few seconds of my weak machine's time into more time on a super computer than is likely to be left in the universe.

Something that's interesting about all this is that we haven't removed the volitional aspect from the constraints entirely - merely moved it around, to a smaller and more visible place maybe.  Castles are conquered with the aid of a traitor, accounting systems foiled by colluding participants, and users give their passwords away for chocolate.  As our security systems become more complex and removed from the things they protect, it becomes harder to associate the new volitional constraints they require with the moral, ethical or legal constraints that good actors voluntarily accept.

On the road to faster specs

Running large spec or test suites can be a bane of Rails developers. I've certainly stopped using autotest since half of our projects started exceeding 5 minutes of spec runtime. After seeing (three times!) presentations by Aman Gupta, I had spent some time with perftools trying to figure out what the heck was making my specs take so long. It's not just that more specs take more time to run: if you clock individual specs you will see identical examples run slower in a larger project. I'd seen that rspec runs can spend upwards of 60% of their time in the garbage collector, but not pursued it further than that.

A couple days ago, Jamis at the 37Signals blog took this idea further, dug into ActiveSupport::TestCase, and generated this wonderful blog post that explains his findings and how to get a 40% or more speedup in Test::Unit. His solution involves reducing the frequency of garbage collection and forcing ActiveSupport::TestCase to destroy instance variables it doesn't need anymore).

It's great, but if you do exactly what he says it won't quite work in RSpec - and RSpec users should get to enjoy this new development, too! While RSpec makes use of ActiveSupport::TestCase, it has a different set of internal instance variables, and Jamis' code will end up erasing the variables that store your actual examples. If you drop in Jamis' code to spec_helper.rb you'll see this error:

vendor/rails/activesupport/lib/active_support/whiny_nil.rb:52:in `method_missing':
 undefined method `description' for nil:NilClass (NoMethodError)

All that's needed to make RSpec happy is a little tweak to Jamis' code that protects a different set of instance variables from being unset. Just drop this blob of code at the bottom of your spec_helper.rb - I saw a 43% speed increase in one project's spec suite. (Note that if you are still using fixtures, you might need to add @loaded_fixtures and/or @fixture_cache to @@reserved_ivars; at LRD we long since abandoned fixtures in favor of factories, so I haven't tested this on spec suites with fixtures).

class ActiveSupport::TestCase
  setup :begin_gc_deferment
  teardown :reconsider_gc_deferment
  teardown :scrub_instance_variables
 
  @@reserved_ivars = %w(@_implementation @_result @_proxy  @_assigns_hash_proxy @_backtrace)
  DEFERRED_GC_THRESHOLD = (ENV['DEFER_GC'] || 1.0).to_f
 
  @@last_gc_run = Time.now
 
  def begin_gc_deferment
    GC.disable if DEFERRED_GC_THRESHOLD > 0
  end
 
  def reconsider_gc_deferment
    if DEFERRED_GC_THRESHOLD > 0 && Time.now - @@last_gc_run >= DEFERRED_GC_THRESHOLD
      GC.enable
      GC.start
      GC.disable
 
      @@last_gc_run = Time.now
    end
  end
 
  def scrub_instance_variables
    (instance_variables - @@reserved_ivars).each do |ivar|
      instance_variable_set(ivar, nil)
    end
  end
end

(Most of this code is Jamis', and I'm not taking credit for his fantastic work.)

RSpec already does a much better job of handling instance variables than Test::Unit, so the scrubbing didn't produce a big speedup for me (only about 5%). But the GC deferment did indeed give me a 43% speed improvement in the spec suite for my biggest project; run time dropped from 7m38s to 4m23s ... what a difference!

NinjaScript: Javascript so unobtrusive, you’ll never it see coming

We're happy to announce NinjaScript: a jQuery plugin for unobtrusive scripting.
NinjaScript provides:

• CSS-like language for web page behavior
• Define rich behaviors that include both event handlers and transformations.
• Durable behaviors that survive DOM alteration, with performance comparable to jQuery's live() method.
• Handy built-in behaviors for AJAX.

Motivations

Unobtrusive Javascript is one of the coming movements in web design for a reason. Separation of concerns is generally a good thing, and the idea of separating behavior from semantics is pretty obvious once you think about it. If nothing else, it makes it much easier to think about how you structure your site. Just build it out as if this were still 1998 and you couldn't trust a browser to open an alert box, much less submit AJAX, then come back and mark everything up.

On the other hand, one hears a lot about "Unobtrusive Is Hard" and how a graceful degrade takes twice as long, etc etc. At the same time, software exists to encapsulate skills.  Could be you'll be seeing a Rails plugin from LRD soon to convert the big Rails helpers into degrading versions.

Therefore: NinjaScript. Unobtrusive Javascript in a tidy package so that you can get on with your day.

What's it look like?

Here's a very simple example. Suppose you have an existing

$.behavior({
  '#coolness': $.ninja.submits_as_ajax() 
})

$.behavior({
  '.date_entry': { transform: function(elem){ $(elem).datepicker() }},
  '#work_unit_select_all': { click: selectAllWorkUnits },
 
  '#timeclock form.edit_work_unit':    $.ninja.ajax_submission({
    busy_element: function(elem){ return $('#timeclock')}
  }),
 
  '#messages .flash': {
    transform: function(elem) {
      $(elem).delay(10000).slideUp(600, function(){$(elem).remove()})
    }
  },
  '#timeclock input#work_unit_hours': {
    click: function(evnt, elem) {
      $(elem).val(hours_format(task_elapsed))
    }
  }
});

$.behavior({
  '.fun': {
    events: {
      click: function(ev, elem){ $(elem).sing_and_dance(); }
      mouseover: function(ev, elem) { $(elem).shiver_in_anticipation(); }
      //yes: mouseover.
    }
  }
});

Logical Tabs: a Rails helper for persistent HTML tab interfaces

logical_tabs is a Rails plugin that assists with the creation of a tabbed panel interface. It has a number of advantages over other solutions, but the primary one is that the tabs persist: if you reload or revisit a page with a tabbed panel, it will remember which tab you last had open on that page. There is a demo project available that shows how to use it with both ERB and HAML templates.

logical_tabs works with both Prototype and jQuery. Here's a screenshot of it in use on a social networking site:

It's still a little rough and I have future plans for it, but I'm using it in production on three sites so I figured I'd give it an early release while I'm hacking at RailsConf 2010.

Why build a new tool, when others exist?

There are already other solutions for building tabbed interfaces, most notably the jQuery UI Tabs tool. Another solution, which I had used previously, is railstabbedpanel.

The jQuery UI tool still requires you to construct all the HTML yourself which can be tedious, and it doesn't work for the large number of Rails products out there using Prototype rather than jQuery. railstabbedpanel provides HTML helpers, but the rendering code is heavily dependent on the specifics of the ActionView rendering/capturing code and so kept breaking when Rails was upgraded, or when HAML was upgraded. I found it very difficult to keep it working as I upgraded my toolset. Also, RTP's javascript was not unobtrusive.

Moreover, neither solution persists tab selections across page loads. So, I wrote this plugin which provides all of these advantages:

• Simple helpers to make it easy to build semantic HTML structures for the tabbed panels.
• Helper rendering that is isolated from the internals of ActionView and shouldn't break when Rails or HAML are upgraded.
• Javascript for both Prototype and jQuery environments
• Persistence of tab selection across page loads - revisit a page, get the same tab.
• Unobtrusive javascript.

Installing logical_tabs

logical_tabs is currently in version 0.6 and only supports Rails 2.x. I have only tested it in Rails 2.3.x, but it should work in 2.2 and 2.1 ... let me know if it doesn't.

Install the plugin:

> script/plugin install git://github.com/LRDesign/logical_tabs.git

Copy in the appropriate javascript file:

If your project uses Prototype:

> cp vendor/plugins/logical_tabs/javascripts/logical_tabs_prototype.js public/javascripts

If your project uses jQuery:

> cp vendor/plugins/logical_tabs/javascripts/logical_tabs_jquery.js public/javascripts

Copy stylesheets (optional)

logical_tabs includes starter stylesheets in both css and sass, you can find them in vendor/plugins/logical_tabs/stylesheets; copy the appropriate file to your public/stylesheets directory and include it in your application, or create your own.

Using the view helpers

logical tabs gives you a view helper, create_tabbed_panel, which takes a block with an argument - that argument will be a TabbedPanel object, the code for which can be found in vendor/plugins/logical_tabs/lib/logical_tabs/tabbed_panel.rb. This object is responsible for tracking all the tabs and their content, generating appropriate element IDs and the like.

There is example code - both ERB and HAML - in the logical_tabs_demo demo project, but the basic usage looks like this. Individual tab/pane pairs are generated simultaneously and can take their content either as a block or as a :content => "content" option.

<% create_tabbed_panel do |tabbed_panel| %>
  <% tabbed_panel.add_tab("Tab One") do %>
    Content for Tab One
  <% end %>
  <% tabbed_panel.add_tab("Tab Two") do %>
    Content for Tab Two
  <% end %>
  <% tabbed_panel.add_tab("Tab Three", :content => "<p>Content for tab three</p>")  %>
<% end %>

When you close the block for create_tabbed_panel, the object will render the HTML for all tabs you have added. It will generate a div with class="tabbed_panel that contains two UL lists; one for the tabs and one for their associated content panels.

That's about it. If you have the correct javascript file for your environment installed, it should just work.

Other Options

logical_tabs generates a default format for the IDs and classes for generated; it uses this to track multiple panels on the same page, for example. However, you can override the generated IDs if you want them formatted in a specific way for custom javascript you've written:

<%= create_tabbed_panel(:base_id => "my_panel") do |panel| %>
   ...etc...

Will use "my_panel" as the ID of the outer div and as the prefix for the individual IDs of the sub-tabs.

logical_tabs always uses the ID of the outer div as the prefix for all internal components, to guarantee that you don't have duplicate IDs (i.e. two LIs with id='tab_1') if you have multiple panel structures on the page.

Tab Persistence

When using tab interfaces in the past, one of the most common complaints I heard from clients is that the panel would "forget" which tab they had selected when visiting a previous page. The two cases where this was most frequently a problem were:

• When the panels contained forms. If you submit a form but the data fails validation, you may be returned to the same view but with the wrong panel selected, so the user can't see the error messages or the color-coded input fields. This is extremely confusing.
• When the panels list items that the user may want to edit on another page. For example, one client has a tabbed panel in a project management system that lists a project's work orders, action items, and problem reports in separate tabs. If the user clicks to edit a problem report, when finished they may be redirected back to the project page, but now the list of work orders is the active tab - they can't see the list containing the item they just edited. Ouch.

The solution I chose was to store the tab selection in a cookie. logical_tabs stores the current tab selection for each tabbed panel structure (in case you have more than one) along with the url, and at each page load the JS it checks to see if there was previously a tab selected.

Caveats

In some ways, logical_tabs is still a bit of a weekend hack. In particular, I'm not thrilled with the ugly element ID formats, and the jQuery script is not a proper jQuery plugin, doesn't interface with jQueryUI, and isn't themeable with themeroller.

Most of my projects use Prototype, and Logical Reality is only recently beginning to switch to jQuery for new projects. So, the adaptation of the javascript for logical_tabs to jQuery was a very quick hack - my best attempt to clone the Prototype behavior with jQuery selectors. If you think it's ugly, it is, and that's why. :-)

Future Plans

• Clean up the jQuery and make it a properly-behaved jQuery UI plugin skinnable with themeroller.
• Rails 3 compatibility.

Convection: self-hosted secure file exchange in Rails

Introducing Convection, an open-source (MIT License) project of Logical Reality Design. Need to swap files with clients or collaborators, but don't want to (or can't) trust those files to Amazon or sendbigfiles.com? Want fine-grained control over which users can see which files? Try Convection.

Lots of file exchange services exist, for example SendBigFiles.com etc. However, all of these services are hosted on someone else's hardware, and most of them share files by transferring URLs -- usually via email -- without good access control or authorization schemes.

We built Convection because a client needed to transfer files with other companies, but they needed to host the system themselves because the contracts they hold with their own clients require them not to store data on services that they don't control. The specifications Convection was built around were:

1. Hosted on our own server.
2. Downloads require a login, and files cannot be shared by email.
3. Users must log in to download files or see available files.
4. User accounts can be grouped, groups can be managed.
5. Files can be shared with an entire group.
6. Files uploaded by users default to minimal permission - visible only to the uploader and to admins.
7. All communications over SSL. (we made this optional)

Installing and hosting Convection

To run Convection, you will need a webserver capable of running a Ruby on Rails application, and a database. Setting such a thing up is beyond the scope of this post. If you have a Dreamhost account, you can set up a Rails-capable domain with a couple of clicks in their web panel. In addition to the server, you will need to set up a database (we have only tested MySQL, but Convection should work with any SQL database for which Rails/ActiveRecord has a supported adapter, including PostgreSQL and Oracle), and initialize the database with these two commands:

  > rake db:migrate RAILS_ENV=production
  > rake db:seed RAILS_ENV=production

This will generate the tables necessary for Convection to run, and create a pair of initial demo users "admin" and "user", both with password "foobar".

If you are setting up a server yourself, there are plenty of guides to deploying Rails on the web. Much of our own guide to deploying CruiseControl.rb can be used to set up any Rails application on Slicehost or any other Ubuntu Linux hosting provider.

Let me know if you're trying to deploy Convection and having trouble: if we know people are using it we may put effort into making it easier to deploy and install, and write a more thorough guide.

A few other links that may help you with deploying a Rails application, depending on your environment:

1. Using Phusion Passenger to Deploy a Rails Application on Apache
2. Deploying Rails Applications (book)

If you Google around you may find plenty of other links relevant to your particular environment.

Configuring Convection

If you log into your running Convection application as an administrator (initial user "admin", password "foobar"), an Admin Tools utility will appear in the right hand column. From here, you can access tools for creating users, and groups, and the general site configuration.

In general site config, you can set your site name and logo, set whether or not the site requires SSL access (Note: your server must already support SSL!) outgoing email and email notification preferences, add Google analytics, and an assortment of other site configuration operations that are mostly self-explanatory.

Upload progress bar: experimental feature.

If your site hosts large uploads that take a while to transfer, you can try our experimental tools to provide an upload progress bar to the user. This tool will only work if your site is served by Apache, and requires installing and configuring an optional module for Apache.

To enable this tool, follow the instructions in the README file and associated links, and turn on the progress bar setting in site preferences.

Helping us improve Convection

Convection is currently in version 1.1.4 and has been in production in two places (that we know of) for about five months as of June 8, 2010.

Please let us know if you are using Convection and enjoy it (or don't). Feel free to request features or alterations, but Convection is open source, so also please consider contributing if you have ideas!

Cloning your remote database to local with Capistrano

Handy file: Capistrano recipe for cloning your remote (production) mysql database to your local box, and vice versa.

One of the more useful recipes for Capistrano that I've come across is this one by netzpirat. It's designed to give you simple commands that will dump the mysql database on your production server to a file, scp the file back to your development workstation, and import it into your local mysql database in a single operation. Very sexy, very powerful. (It also has commands for doing the reverse, and for uploading/downloading files as well - useful for things like paperclip attachments.)

Unfortunately, for my purposes, it failed right out of the box in two different ways:

It only pulls your database name, db login, and password from your database.yml, and then (on the remote server) tries to connect to the database on a local socket. Most of my databases are hosted on remote boxes.

It assumes your production database credentials exist in your local database.yml, which they really shouldn't in any decent deployment. Especially not if you are hosting your code in an outside repository.

Slightly improved version

So, I created this improved version, which pulls the remote DB credentials from the database.yml file on the remote server (assuming you've put them in #{shared_path}/config/database.yml), and which will also use the hostname, if any, from your database.yml when it executes mysqldump.

Let me know if you have any trouble with it and I'll correct any errors in the gist.

Transactional Testing for Multiple Databases in ActiveRecord

We've been working on an app that needs to stand astride two databases - one local DB for the app itself, and another with restrictive policies about modifications that is nonetheless authoritative on many subjects. There's a fair amount of tricky interaction between the two, and testing has been a delightful challenge.

We're using the use_db plugin, and all it takes to make testing transactions happen around multiple DBs is:

In: spec/spec_helper.rb

require 'override_test_callbacks'

My concern comes from the fact that this is a direct and unfiltered monkeypatch on ActiveRecord::TestFixtures. So it relies on use_transactional_fixtures (which could certainly be used without using actual fixures, granted), and if the test transaction code moves within Rails, that's another integration to worry about. Or if we add a spec that doesn't wind up making ActiveRecord::TestFixtures load... Or if we decide to use something other than use_db...

So instead I'm using:

Spec::Runner.configure do |config|
  config.prepend_before do
    (UseDbPlugin.all_use_dbs - [ActiveRecord::Base]).each do |db|
      db.connection.increment_open_transactions
      db.connection.transaction_joinable = false
      db.connection.begin_db_transaction
    end
  end
 
  config.append_after do
    (UseDbPlugin.all_use_dbs - [ActiveRecord::Base]).reverse.each do |db|
      db.connection.rollback_db_transaction
      db.connection.decrement_open_transactions
    end
  end
end

If we weren't already using transactional fixtures, I might pull out the - [ActiveRecord::Base]. And if we were to change off of use_db, there's one place to change the transaction code. Finally, there's much less dependence on the innards of ActiveRecord - only it's published API.