willbryant.net

MVC testing

Published Sun 11 November 2007 17:44 (+1300)
Tagged

This is part 3 of a miniseries of articles on Rails application testing.

We saw in the previous section that we generally structure our tests the same way our application is structured.

The goal of MVC testing is to test that each of the components of our application behave as desired. Testing parts individually means we can examine and interfere with internal details, and test out not just the “easy” situations but complex corner cases too.

Corner cases and specific internal actions are often difficult to produce from top-level tests, and if you do the tests are typically quite fragile – prone to breaking when something else is changed, which means wasted effort and confusion trying to keep everything in sync and figure out which problems are genuine problems.

When we get onto controllers in an article or two's time, we'll also see that splitting up the tests so we test components at their own layer makes everything run much faster – and usually takes less typing to write the tests too!

When to write tests

There is a strong consensus in the Rails community that MVC tests should always written as an integral part of the coding process, not by a separate QA team, and not after the application is supposedly “developed”.

Aside from making sure that the tests are kept up to date, writing the tests as you go helps show up & resolve confusion about how things should work when the code is being written, not when you're nearing launch.

And even more importantly it makes a big contribution to both individual and (especially) team productivity, because the developers don't waste time trying to deal with issues in broken untested code the slow way – in the browser and/or stepping through the whole app in the debugger.

I have seen a number of clients that didn't test as they went, and the result was almost always that their developers wasted far more time getting the release ready by manually debugging and testing issues than it would have taken to write (well-chosen, concise!) tests that would have found the same issues straight away.

Rails' testing frameworks are so quick to use that once you've gotten used to using them, it often takes no longer to write a test than it does to try it manually once – and then you've got it captured so you can make changes in and refactor the codebase rapidly without worrying what you might break.

Again, this is a big contribution to productivity. It's a popular misconception that the big benefits to writing test code come after months – I usually see payback on the tests I write within just days.

Yes, I know, you don't have time to write tests; you don't have time to manually debug and test blocking issues either, and that's honestly what you're going to end up doing instead!

Test::Unit

The standard Rails MVC testing platform is Test::Unit, which comes with it and which Rails' own tests are written in.

Test::Unit is certainly adequate, and remains popular simply because it's the traditional standard.

The biggest problem that I personally have with it is easy to see from a sample:

class InvoiceTest < Test::Unit::TestCase
  ...

  def test_invoice_payment
    @invoice.pay('Cheque 1312')
    assert @invoice.paid?
    assert_equal Time.now.to_date, @invoice.paid_on
    assert_equal 'Cheque 1312', @invoice.payment_details
    assert @invoice.valid?
  end
end

This isn't Ruby-style code; it's Java-style code implemented in Ruby (assert_equal??).

(And, although I don't see Test::Unit's authors describe it this way, I think that that's basically just what happened, because this stuff corresponds quite directly to the venerable Java unit testing granddaddy, JUnit.)

RSpec

The main contender, and as of November 2007 hot favourite amongst much of the Rails community, is RSpec.

It'll make RSpec devotees unhappy when I say that from a technical perspective, ie. looking just at the software itself, since RSpec does the same things in functional terms as Test::Unit does, the main difference between them is really syntax (but read on).

Certainly, this is much nicer:

describe Invoice do
  ...

  it "should record the date and deposit details when paid" do
    @invoice.pay('Cheque 1312')
    @invoice.paid?.should be_true
    @invoice.paid_on.should == Time.now.to_date
    @invoice.payment_details.should == 'Cheque 1312'
    @invoice.should be_valid
  end
end

It's much more readable (we like that in Ruby!) and more natural to write, and the Matcher-based system is nice.

There's also minor-but-nice bonuses such as the way that your specs are actually run in the order they're defined, rather than the arbitrary order they're run in by Test::Unit. I find this quicker to work with when you suddenly get a run of test failures, rather than having to hunt through the list.

More than just syntax

But RSpec's authors aren't just trying to improve the syntax you use to write tests (by the way, if you do want that, and for some reason aren't keen on RSpec, there are options – eg. you could split the difference and use test/spec or maybe Dust).

What RSpec's strongest proponents want you to do is to improve your development process by making specifying the required behavior the whole basic driver of your development process, not something you fill in when you write the code. This practice, Behaviour-Driven Development, incorporates Test-Driven Development, but goes further philosophically and emphasises working exclusively in terms of required behaviour.

I encourage you to learn more about BDD from its' proponents rather than me, as I'm personally just neutral to it and so can't really give you the “in”.

I don't have a problem with it, but I don't see the differences between test-as-you-go, TDD, and BDD as being as significant as some think, and I don't think there is one process that fits all projects – so you should have a look around yourself and make up your own mind.

Remember, whatever advantages your chosen practice has, ultimately what matters is how well your test code exposes helps you deliver what the customer actually needs – properly working, relevant software.

And remember

One thing that I do want to emphasise though is that even if you don't end up adopting BDD, you can certainly still think about using RSpec – even if its makers feel you're missing out on the best part, it's still up to you to choose the right tools for the job, and RSpec certainly has advantages on its own.

And in fact, on my projects, that's most often what we've done – we use RSpec a lot and like it, but we're mostly not BDD adherents. There is a happy middle ground!

Next: Model testing – For most apps, this is where you get the biggest pay rate of bugs squashed to time spent. What to test, what not to test, and some inflammatory comments about stubbing out too much.

More