Rails Engine Part II

Me = Mock and Stub Freak

No one at work has ever accused me of being underzealous in my use of mocks and stubs in my testing. Mocks are fake objects and stubs are fake behaviors employed in software tests. There are three reasons for my bullishness on using fake objects and messages in lieu of actual database entities and method calls.

First, speed. Mocks and stubs perform faster than real database objects and as a proponent of test driven development (TDD), slow tests can break flow state which then has a multiplier effect on retarding progress on a tough project.

Second, mocks and stubs can reveal code smells and encourage encapsulation. If a new method has many inputs, needs to know about methods on other classes, and has multiple side effects, well mocks and stubs are not going to fix that. But, they can help lead to more encapsulated solutions.

Finally, I think mocks and stubs are fun. They are fun because they allow me to work on a large and often alien codebase and still remain productive, as long as I can limit my attention to the one thing I need to change.

They are also fun because they encourage a type of test driven development known as the London School-TDD, which is a sort of top down approach to test driven development you can read about in this article by Justin Searls. As Turing School students, we have have been schooled in more of the Detroit style of TDD, which is a more bottom up approach that stresses the red, green, refactor cycle.

Applying Mocks and Stubs

To illustrate some of the reasons I like mocks and stubs so much, I'm going to refactor a test from one of my old Turing projects, called Rails Engine, which I write about more extensively here. Rails Engine is great for this task because there are many database models compared to other projects so there are plenty of tests that require the instantiation of multiple objects with complicated relationships to one another.

First I went back and ran my test suite ten times in a row to find a rough starting place from which we can improve. In this case the test suite on average takes 3.207 seconds to run. But imagine working on a larger side project, or even a small production app that is say, 10 times the size of the 120 tests I wrote for Rails Engine. A 30 second wait between test runs is not insignificant. Sure, you can run tests by directory, file, and even line, so some will see my work here as an exercise primarily in compulsivity.

Next let's look at a sample controller test:

require 'rails_helper'

# GET /api/v1/merchants/:id/favorite_customer returns the customer
# who has conducted the most total number of successful transactions.
describe "get to customers favorite merchant" do
  it "returns the merchant who has the most transactions with that customer" do
    customer = create(:customer)
    merchant_1 = create(:merchant)
    merchant_2 = create(:merchant)
    customer_1_invoices = create_list(:invoice, 2, customer_id: customer.id, merchant_id: merchant_1.id)
    customer_2_invoice = create(:invoice, customer_id: customer.id, merchant_id: merchant_2.id)
    create(:transaction, invoice_id: customer_2_invoice.id, result: "success")
    customer_1_invoices.each do |invoice|
      create(:transaction, invoice_id: invoice.id, result: "success")
    end

    get "/api/v1/customers/#{customer.id}/favorite_merchant"
    merchant = JSON.parse(response.body)

    expect(response).to be_success
    expect(merchant).to be_instance_of(Hash)
    expect(merchant["id"]).to eq(merchant_1.id)
  end
end

The above file is a test for a customers controller. It has one path, a get to /customers/:id/favorite_merchant, which returns a JSON version of the merchant with whom the given customer has done the most business. The above test across ten runs takes on average 2.148227 seconds to run.

Looking at the test one line at a time, we see that first we create a customer with Factory Girl. Next we create two merchants. Next we create two invoices between the customer and each of the merchants. Finally we create two transactions for the two invoices, which record things like payment type and status.

After our setup, we make a get request to our desired path. We parse the response body and make assertions about which of our two merchants should be returned. This is a great test, but it's totally out of place.

Since we have a method on our Customer object called "favorite_merchant", which we test thoroughly in our model test, we can trust at the controller level this method will continue to work as planned. At the controller we are only really interested in testing the route and that our favorite_merchant is called.

Let's refactor

require 'rails_helper'

describe "get to customers favorite merchant" do
  it "routes correctly" do
    customer = double
    allow(Customer).to receive(:find).with('1') { customer }
    expect(customer).to receive(:favorite_merchant).once

    get "/api/v1/customers/1/favorite_merchant"
  end
end

Taking this new test line by line we see first we are creating a customer. This time, though, we are just saying that a customer is a double, which means it is a mock object. This is part of Rspec's built in mocking and stubbing library. I actually use Rspec with Mocha for mocking at work, which I've grown to really like. 

Now we have a test double, which we store in the customer variable. Next we say allow Customer (the class, not the test double), to receive the message "find" with the string "1" as an argument and return customer (our test double). This line is an example of stubbing. We aren't actually calling Customer.find(1) in our test, rather we are replacing that behavior with the stubbed behavior described in our test. Instead of Customer.find going to the database and looking for the customer with an id of 1, it will just return our customer test double.

Next we expect that our customer (test double) will receive the message "favorite_merchant" one time. We don't have to worry about what this method does, or what various models it looks at because we only care about the route here, not the functionality which is tested on the model. Since we don't have to worry about the functionality, we don't need to set any of the models up in our test.

We kick everything off by making the get request to our desired route. This is confusing because in typical tests, setup steps come first followed by our expectations. With mocks and stubs you are writing what you expect to happen in the future and then call the action that starts everything off.

When we run our test everything should pass because the implementation is already done.

Benchmarking

Running this test 10 times, we see a pretty dramatic speed increase. File load time is not really going to change, so just looking at test run time, before we were at roughly .788 seconds. Now we are down to .705 seconds. On one test, this is obviously not a big deal, but over hundreds or thousands of tests, these speed gains add up and become quite significant. Again, just multiplying this by 1000 to simulate what we could hypothetically expect in a test suite with 1000 tests. That would give us a test suite that will run in 788 seconds, or 13 minutes. Our mocked and stubbed version would run in 11 minutes 45 seconds. That's 1 minute and 15 seconds of extra development time for every full test suite run! I'm only being half feciscious. 

Here is our Controller implementation

class Api::V1::Customers::MerchantsController < ApplicationController
  def show
    render json: Customer.find(params["id"]).favorite_merchant
  end
end

Does it not just make intuitive sense that for this one line method we should only have a few lines of test? Before we were testing this with something like 24 lines of code. We are down to 9 with our mocking and stubbing.

Part of the reason we test is so that in the future if a developer makes a change to the code, they'll know quickly if that change has broken some key piece of functionality. Let's mess with implementation to see what kind of errors our test will throw.

Let's start with tearing everything out:

class Api::V1::Customers::MerchantsController < ApplicationController
  def show
  end
end

Will give us the following when we run our mock and stub test:

Our test fails because we expected our test double to receive the method call favorite_merchant 1 time, but it never happened. That makes sense because our show method is completely empty.

How about if we just find our customer:

class Api::V1::Customers::MerchantsController < ApplicationController
  def show
    render json: Customer.find()
  end
end

In our above run, our test fails because we were expecting Customer (the class) to have "find" called on it with argument "1", but instead no argument was used in the find.

Now lets pass in the correct argument:

class Api::V1::Customers::MerchantsController < ApplicationController
  def show
    render json: Customer.find(params['id'])
  end
end

Now we are failing because our customer (test double) never has "favorite_merchant" called on it.

Going back to the original implementation (ok, I wrote a private customer method because it was bothering me), we get a passing test:

class Api::V1::Customers::MerchantsController < ApplicationController
  def show
    render json: customer.favorite_merchant
  end
  private

  def customer
    Customer.find(params['id'])
  end
end

It's not all Smiles Und SunshinE

There are definitely downsides to an over reliance on mocks and stubs. First, they can make tests more fragile and lead to false positives when test suites fail. Second, they can take longer to implement than a more straightforward Factory Girl style test.

That said, they are fun and when used responsibly, quite effective.