Guidelines for writing better specs

SINAPTIA's photo
SINAPTIA

As we’ve seen in one of the previous posts in this series, RSpec has a lot of cool features that can easily be misused and can make your test suite really painful, hard to work with, affect the team productivity and morale, enrage the management and ultimately lose confidence in your own code.

We’ve also analyzed the benefits of a more direct approach to testing using Minitest and the Four-phase testing pattern in the previous post

In this post, we will briefly tell you what we did in those projects where changing the testing stack was not viable. We will define a reduced and safe subset of the RSpec features and a set of guidelines on how to use them to avoid the problems we outlined.

TL;DR

This is the set of guidelines we came up with after a thorough analysis of the test suite, the problems it was facing and their root causes:

  • Use context to define the test scenario in natural language

  • Use context to group related tests together

  • Keep factories to their minimal expression (pass validations)

  • Don’t put test setups in factories

  • Don’t put test setups in contexts that have nested contexts

  • Don’t put its in contexts that have nested contexts

  • Use let! to define objects in the setup that the it will use to exercise and verify

  • Use before to define objects and relations that must be there but are not directly used in the following it

  • Define test setup in the context that has the its

  • Do not use let

  • Do not use shared examples

  • Do not use subject

  • Repetition in test setups is ok

  • If repetition is unbearable, extract the setup into a tailored factory method inside an instance method in the same spec file.

Read ahead if you would like to know the reasoning behind each of these guidelines

Main pain points

Let’s first revisit the main issues we faced in those projects. The root cause of those issues was the deep inheritance tree of test setups. This inheritance tree was enabled by 2 things:

  • RSpec’s ability to allow infinitely nested contexts

  • Naively optimizing code-writing: who wants to write again the same test setup that I wrote 15 lines above?

The two test anti-patterns that this produced were:

  • Global Shared Fixtures

  • Mystery Guests

Global shared fixtures

See: Standard Fixture

TL;DR: This means that the setup is shared among several test cases.

This brings three main problems:

  1. As it is global all the tests that depend on the fixture are coupled together.

  2. As there are many test cases depending on it, it will probably create more things than needed for each specific test

  3. There’s a lack of information categorization. As it holds details of many test setups, is hard to tell which part of it is important on a given test.

A global fixture usually lives in:

  1. Factories

  2. lets and befores near the top of specs with deeply nested contexts

Mystery Guest

The test reader is not able to see the cause and effect between fixture and verification logic because part of it is done outside the Test Method.

see: Mystery Guest

This means the test is running on top of out-of-band information that the dev needs to search, find, and reconstruct to fully understand what’s going on in the test.

Usually found in:

  1. Factories

  2. lets and befores near the top of specs with deeply nested contexts

  3. fixture files stored on disk

The Safe Toolset

As you can already imagine we used the conceptual structure of the Four-Phase Test Pattern:

  • Setup

  • Exercise

  • Verification

  • Teardown

So, the first step was to define a mapping between Rspec DSL and the four phases. Here’s what we came up with:

  • before, let and let! are the Setup phase

  • it will hold both the Exercise and Verification phases

  • after is Teardown

Surprisingly, there are no mappings for context or describe. This was the first revelation we had. The source of a lot of the problems we had is just not there anymore :). And so, the first question arose: if they are not an essential part of the test, why are they there? what is their mission?

The conclusions we arrived at are:

  1. They are mandatory: you need at least one describe as the entry point of the rspec tests.

  2. They are documentation, the description of the test case. Similar to the method name in Minitest::Unit, or the #test method in ActiveSupport::TestCase

  3. They are used in the test runner output, which is quite nice with nested contexts

So, taking all this into account, how can we use rspec and avoid the mentioned problems?

Guidelines

Here we will outline a few DOs and DON’Ts on how to use each of the methods in the safe subset and other tools and methods.

describe / context

Use describe and context as descriptions. The idea is to use these descriptions as the specification of a test’s setup. A code review can easily detect if there are discrepancies between test specification (what the dev wants to test) and implementation (what is actually being tested).

DOs:

  • use them to explicitly communicate to the reader what’s the test case and how the test setup should be built

  • use them to group related tests together (nesting contexts is ok if you respect the DON’Ts)

  • be verbose

DON’Ts:

  • Do not use lets or befores if the context has nested contexts

  • Do not use its if the context has nested contexts

let and let!

This one is one of the most interesting in my opinion. Here goes the controversial statement:

let and let! are just a DSL that hides away variable assignments. You don’t really need them to express a test case setup.

Do you agree?

let is even more problematic, as it only makes sense if you have a massive shared fixture at the top of the test file.

One can understand the reason behind let: rspec is trying to solve problem number 2 (creating more things than needed for each specific test, which makes the test brittle and slow). But, in my opinion, this would have been the first symptom showing us that something is not ok: The problem is not that variable assignment is not lazy/performant, the problem is the huge shared fixture at the top. Without it, you don’t need lazy variable assignments. A good old plain assignment that takes effect where it is defined is more than enough, straightforward, no second guessing when it is going to actually be defined.

Anyway, we didn’t remove them from the safe subset, as we found a communicative advantage in using them (see the next section before).

DOs:

  • Only use them inside the context that is the direct parent of its and have the exercise and verification phases

  • Always, always, use let!

DON’Ts:

  • Never use let

before

before and let! are both useful for defining stuff that belongs in the test setup. But how can we decide when to use one or the other? We use these simple rules for deciding:

  • Use let!s for pieces of setup that you need a handle to. That is: variables that are directly used in the its (exercise and verification phase).

  • Use befores for uninteresting things that need to be there but are not used directly in the test

The idea is to minimize the distracting bits of the test setup and make it evident if the test is fulfilling its description.

it

Finally! its hold the exercise and verification phases.

DOs:

  • Always make them the leaves of the contexts tree

  • use a blank line to visually separate the two phases

DON’Ts:

  • Don’t use them in the middle of the tree.

Final example

After all these points, arguments, DOs, and DON’Ts, how does a test using these guidelines look like? see for yourself:

describe "Creating a profile" do
  context "without providing location id" do
    context "when profile does not exist" do
      let!(:brand) { create(:brand) }
      let!(:user) { create_user(brand: brand) }
      let!(:service) { CreateProfileService.new(phone_number: "+155512345678") }
      let!(:default_location) { create_location(id: SecureRandom.uuid) }

      before do
        brand.locations = [default_location]
        brand.default_location = default_location
      end

      it "creates a profile for the default location" do
        service.call brand: brand, user: user

        expect(Profile.count).to be 1
        expect(Profile.last.location).to eq brand.default_location
      end
    end
    context "when profile exists with phone number within a Brand" do
      let!(:brand) { create(:brand) }
      let!(:user) { create(:user, brand: brand) }
      let!(:service) { CreateProfileService.new(phone_number: "+1 555 1234 5678") }

      before do
        create :profile, phone_number: "+1 555 1234 5678", brand: brand
      end

      it "does not create a profile" do
        result = service.call brand: brand, user: user

        expect(Profile.count).to eq 1
        expect(result.errors).to be_present
        expect(result.errors.first).to eq :profile_already_exists
      end
    end
  end

  context "providing a location id" do
    context "when the brand does not have a 3rd party integration" do
      #...
    end
    context "when the brand has a 3rd party integration" do
      #...
    end
  end
end

Doesn’t look too alien to my old-time RSpec eyes, what do you think?

Duplication is ok

This might be a little counterintuitive, but having repetition in the test’s setup or verification is ok.

Duplication causes less harm and waste of effort than the previous problems, it helps isolate each test, thus making debugging and finding problems easier when tests fail.

If duplication is not bearable or feels absolutely dumb, extract the repeated bits in carefully crafted Factory Methods (this is not a factory-bot factory, but an actual ruby method like in the factory method pattern).

For example, let’s assume all the test cases for a given service object need a User (that belongs to a Location which belongs to a Brand), and has a specific role on a Group inside that Location. But some of the tests care about the brand or location and Some don’t. But no test in this file cares about the group or the role. You can have a method defined inside this file as:

describe "Creating a user" do
  #...
  def create_user(brand: create(:brand), location: create(:location, brand: brand))
    create(:user).tap do |u|
      u.set_role :standard, create(:group, location: location)
    end
  end
end

This allows us to keep all the benefits of the above guidelines, and at the same time, clean up the little repetitive bits of complex test setups that are not interesting for the test cases at hand. But it is not the “global” unique way to create a User, is specifically tailored for the tests in this test file, thus the impact of the shared logic is contained.

Now, You might be thinking… Why not an actual factory? we’ll see in the next section

About factories

As mentioned before, using a full-featured factory library like factory_bot, gives us another way to obscure our test setups, move them out of the test context, and make them easily fall into a global shared fixture. Similarly to let and let! these libraries provide a DSL that replaces method calls and object instantiation. We still use them, because:

  • team’s culture

  • DSL is comfortable

But we follow these guidelines strictly:

  • Keep your factories to the minimal expression possible. Just make them pass validations.

  • If you are building a bunch of complex relations in your factories, stop and think if you are not moving test setup inside factories.

  • If you are, then:

    • Don’t put test setups inside factories

    • Use factory methods specially crafted for constructing such relations defined as instance methods in the spec file

The purpose of factories is not to define the needed relations in test setups but to hide uninteresting details for the current test about certain needed objects. Factories help to have those unimportant details defined with valid data and avoid polluting the information space of the test with a noisy setup.

For example:

describe "Creating a profile" do
  context "without providing location id" do
    context "when profile does not exist" do
      let!(:brand) { create(:brand) }
      let!(:user) { create_user(brand: brand) }
      let!(:service) { CreateProfileService.new(phone_number: "+155512345678") }
      let!(:default_location) { create_location(id: SecureRandom.uuid) }

      before do
        brand.locations = [default_location]
        brand.default_location = default_location
      end

      it "creates a profile for the default location" do
        service.call brand: brand, user: user

        expect(Profile.count).to be 1
        expect(Profile.last.location).to eq brand.default_location
      end
    end

Here the test doesn’t care about the brand’s or user’s name, but it does care about the relation between the two (a user in that brand). Without factories, this setup would look like:

  let(:brand) { Brand.create(name: "My Test Brand", address: "742 Evergreen Terrace, Springfield", type: :multi) }
  let(:user) { User.create(brand: brand, name: "My user name", password: "admin1234", password_confirmation: "admin1234", role: :admin, email: "admin@my-email.com") }
  let(:location) { Location.create brand: brand, name: "My Test Location", address: "742 Evergreen Terrace, Springfield, Again", another_uninteresting_field_for_this_test: true, but_needed_to_create_a_new_instance_of_this_class: true }
  let(:group) { Group.create name: :department, location: location}

  before { user.add_role :admin, group }
end

To figure out the relationship between the user and the brand, you would need to read all of this and connect the dots in your head. And to learn what are the interesting bits of setup, you need to read the whole test, which you’ll do anyway, but a little explicitness up front makes it way better.

Be kind to your reviewers. Be kind to your future self

About shared Examples

  • Don’t

Abstracting away the test’s exercise and verification phases ends up with highly-coupled and anemic tests. Other things benefit from an abstraction or a DRY-up refactoring. In general, tests don’t.

About subject

Avoid using subject when other ways to express which is the Subject Under Test are available. This falls along the lines of “prefer explicit vs implicit”.

Use more Ruby! <3

Ruby’s Object Orientation, methods, and classes are fantastic tools, super-expressive, and available everywhere, in rspec specs too. You don’t need to force yourself into the DSL. Use what it is good for, leave the rest out of your toolbox.

Always have a critical eye on the things you learned and don’t be afraid of revisiting your principles. You might learn or discover something new or unseen that better fits your team’s culture and brings benefits in the long run!

RubyRuby on Railssoftware developmentSoftware Engineeringsoftware architectureReactReact NativeREST APIapp developmentWeb Developmentweb application