Make tests read like a book 6


When all tests pass, life is great. And that should be the default case. We are working towards all tests passing, we are not trying to make mistakes.
But when we make mistakes we want to get rid of them asap. That means we need to understand the cause in order to fix it. If the test talks to me like that
"Expected false to be truthy." I get angry. But if the test says
"Expected [object DisplayObject] to have properties 'blendMode'." I get the feeling I know what’s going on and even better, it sounds like I know what I have to fix. That is where custom jasmine matchers come into play.

Be nice to yourself

It was really the above error message, that made me get frustrated and lead to matchers that just say what they expect. When I stumbled over tests like this

// BAD
expect('one million'.indexOf('two')===0)
  .toBeTruthy();

I first needed at least a second look to find out what the test is supposed to do, checking that a certain string starts with ‘one’.

Expected
    false
to be truthy.

But what is even worse, it makes a test result harder to put into context. Since the message does never come straight from the IDE, but mostly from the browser or (in the case of nodejs) on the console or even worse from jenkins (our CI tool). And if those tools tell me that false should be truthy I feel pretty much left in the dark. I have no clue where to look for the error and actually I get angry at the test (author – me? oops) for being lazy and not telling me what I wanted to know through testing. Of course, it worked for me at the time I wrote the test and I had the context and no problem to understand the test result, because I was in the flow. But I am not in that flow anymore and it’s my time that I steal by having been sloppy. And even if I am a one man show, a month later I won’t be in the flow anymore either.

Writing a test like

// GOOD
expect('one million')
  .toStartWith('two');

and get a resulting error message

Expected
    "one million"
to start with
    "two"

makes writing and especially reading (and fixing) tests a pleasure. The test case is not only a dumb unreadable verification of your code, but it is what tests are supposed to be and what BDD means:

  1. the description of the behavior
  2. a readable specification
  3. almost a documentation and last but not least
  4. a fun to use tool that helps you maintain your code with less pain.

Jasmine Matchers

In the following, I am going to show some example usages of some of the jasmine matchers, that we provide with jasmine-matchers, which you find on github, of course (it must be great being github and having people write “using github of course”, congrats!).

toBeArray, toBeNan, toBeNumber, toBeOfType

Some simple checks in a loosely typed language for a proper initialization or a return value is sometimes needed.

expect(new Sprite().filters)
  .toBeArray();
expect(value)
  .toBeNan();
expect(otherValue)
  .toBeNumber();

Which result in the nice error message like this

Expected
    value
to be array

If there is no explicit matcher, sometimes the following is used:

expect('a')
  .toBeOfType('number');

which properly reports

Expected
    "a"
to be of type
    "number"

Instead of the way you would do it, if you only had the standard jasmine matchers

// BAD
expect(typeof 'a' == 'number')
  .toBe(true);

which would only tell you

Expected
    false
to be 
    true

which helps little.

toBeCloseToOneOf

For our conversion from Flash to HTML5, we have some edge cases, where we want to make sure that certain values are alike, they don’t necessarily need to match. The concrete example here, was the textHeight/textWidth of fonts use by flash.text.TextField. As I learned the hard way too, font metrics are not set in stone. So I learned that the textWidth of “Y” is even different in the AIR runtime and in Flash’s web runtime. Just by one pixel, but still different. And not testable with a expect(x.textWidth).toBe(9), not even with expect(x.textWidth).toBeOneOf([9, 10]). The latter one might work for one letter tests, but as soon, as get to a longer string we can’t use that approach anymore. But we still want to make sure that the value is somewhere near.
So the following made it work, and gives pretty much security that we are doing it right.


function tenPercentOff(actual, expected) {
  return expected * 0.9 <= actual && expected * 1.1 >= actual;
}
it('should report correct textWidth', function() {
  expect(someText.textWidth)
    .toBeCloseToOneOf([23, 26], tenPercentOff);
});

In this case, the matcher can even include some intelligence for reporting, which in case of an error reports the following:

Expected 19 to be 'ten percent off' of one of [23, 26].

toContainOnce

Sometimes very specific matchers make sense, and since they are simple to write you grow a good library over time, one of them that might not be used that often, but states very well the intention of test is the following.
If checks if the given value is contained only once in a given array or string.

it('should return every package only once', function() {
  var actual = classGenerator.getAllPackageNames();
  expect(actual)
    .toContainOnce('flash.net');
});

toHaveProperties, toHaveOwnProperties

When working with objects, especially while we were implementing the AS3 library, we have come to need checks for certain object conditions. Not only to know if an object has a certain property, but also explicitly if it has been defined on this object and not any of it’s parents.

var obj = {x:0, y:undefined};
expect(obj)
  .toHaveProperties('x', 'y', 'z');

Expected 
    { x : 0, y : undefined } 
to have properties 
    'x', 'y', 'z'.

String matchers

When working with strings it becomes very handy to have specific string matchers available.

expect('abc')
  .toEndWith('c');
expect(['one', 'zwee', 'three'])
  .toEachEndWith('e');
// Explicit non-matcher check
expect(['one', 'zwei', 'three'])
  .not.toEachEndWith('e');
expect('abc')
  .toStartWith('a');
expect(['one', 'onetwo', 'onethree'])
  .toEachStartWith('o');
expect(['one', 'onetwo', 'three'])
  .toSomeStartWith('one');
expect('uxebu rox')
  .toContainOnce('uxebu'); // used as a string matcher

I guess by now, it’s clear that the reporting makes understanding failures easy, as you can see nicely in the following too:

Expected 
    [ 'two', 'three' ] 
to some start with 
    'one'.

Going beyond

By clever use of jasmine’s describe blocks when writing tests, I bet you can create a readable prosa documentation of the code you wrote. Even special cases, can be covered very well, things like

  • optimizations
  • bug fixes
  • enhancements
  • speed ups
  • even quirky behaviors

can be documented that way. And if you take TDD serious “documented” means, as usual: I write the test to ask for this behavior, see it fail and fix the test.
In the following code a test block describes additional optimizations, that have been done to speed up the code. I left out the actual tests, to show that the power of correct describe and it blocks can result in some pretty reasonable, readable test code.

describe('Optimizations', function() {
  describe('should not connect stage.on(pointerdown)', function() {
    it('in normal state', function() {});
    it('after second click', function() {});
  });
});

The good thing, even the expectation what the optimization is is well written down. What the implementation looks like is hidden behind – as it should be.

Conclusion

And being nice to yourself and making code maintainable for your future is definitely something that feels good. Additionally you are helping your team to benefit from the same, and hopefully you inspire them all to do the same.

Enjoy for the happiness and satisfaction that ‘this puzzle piece’ to better code brings!

References

The github hosted project home.
The best documentation of all the matchers, are the tests themselves.


About Wolfram Kriesing

Wolfram Kriesing has more than fourteen years professional experience in IT. The early involvement in web technologies provides him with deep knowledge and experience for designing and implementing stable and scalable architectures.

  • Jeff K

    I like to make quite extensive use of ‘describe’ and ‘it’ blocks. To be honest, I never thought of implementing better matchers as well, so something I must start doing!

    My describes end up telling a lot of the story on their own. I write a short sentence of what should happen. Wherever there’s a ‘when’or ‘and’ then I use a new ‘describe’ block. Most of my test set-up and execution goes into a ‘beforeEach’ and then the ‘it’s mostly just contain an expect:

    describe(“checkLocation()”, function () {
    describe(“when the container is fully visible”, function () {
    beforeEach(function () {
    // code to set container visible
    });

    describe(“and location is ‘top’”, function () {
    beforeEach(function () {
    // var result = …
    });
    it(“should not dock the container”, function () {
    // expect code to check the container. Use the result object
    });
    it(“should do something else…”, function () {});
    // … etc

  • http://blueskyonmars.com/ Kevin Dangoor

    Great article! Thanks!

    Could you add a license to jasmine-matchers.js? I realize that it’s only used in test code, but I’d still have an issue bringing code into our repository without a convenient license (BSD or MIT, not sure how our lawyers view LGPL or similar in such contexts).

  • Wolfram Kriesing

    hey Jeff, oh yeah I know exactly what you mean, I also like the tests to tell the story of what I expect the code to do.

  • Wolfram Kriesing

    thx Kevin, sure will do!

  • Wolfram Kriesing

    done

  • http://ericleads.com/ Eric Elliott

    That’s one of the reasons I’m a big fan of simpler test solutions like QUnit and Tape (by Substack). Assertions look like this:

    test(‘foo’, function () {
    equal(foo(), expected, ‘should return the expected output.’);
    });

    Since you have full control over the descriptive message, you can say exactly what you mean in plain english, rather than rely on your possibly contrived test case example to say something descriptive for you. The assertions are a lot fewer and easier to learn, and these kinds of tests tend to be both easier to read, and a lot less complicated.