Agile FAQs
  About   Slides   Home  

 
Managed Chaos
Naresh Jain's Random Thoughts on Software Development and Adventure Sports
     
`
 
RSS Feed
Recent Thoughts
Tags
Recent Comments

Single Assert Per Unit Test: Myth

You are telling me, if I were unit testing a List class and I wrote tests like the following with 2 assert statements in them, then I’m violating xUnit testing “best practices”.

@Test
public void addingElementIncreasesSizeOfTheList() {
	assertEquals(0, list.size()); 
       //or assertListSizeIs(0); - define your custom assert statement
	list.add("Element");
	assertEquals(1, list.size()); // assertListSizeIs(1);
}

If so, how and why?

I understand that if I wrote a test like the following its not very communicative:

@Test
public void removeElementBasedOnItsIndexAndReduceSizeByOne() {
	list.add("Element");
	assertEquals("Element", list.remove(0));
	assertEquals(0, list.size());
}

in this case, it might be better to write 2 test instead:

@Test
public void removeElementBasedOnItsIndex() {
	list.add("Element");
	assertEquals("Element", list.remove(0));
}
 
@Test
public void reducesSizeByOneOnRemovingAnElement() {
	list.add("Element");
	assertEquals(1, list.size());
	list.remove(0);
	assertEquals(0, list.size());
}

Notice our goal was better communication and one of the tests we ended up (incidentally) with had just one assert statement in it (which is better than the previous test which was asserting 2 different things).

Our goal is not one assert statement per test. Our goal is better communication/documentation. Using the one-assert-per-test heuristic helps to achieve better communication. But splitting tests into more tests or deleting an important assert statement from a test just to avoid multiple assert statements in a test is mindless dogma.

Conclusion: One-Assert-Per-Test really means test one aspect/behavior in each test (and communicate that clearly in your test name). It does not literally mean, one test should have only one assert statement in it. Test one aspect/behavior in one test and related aspect/behavior can be tested in another test. Sometimes we need more than one assert statement to assert one aspect/behavior and its helps to add those assert statements in the test for better communication.

So don’t be afraid, go ahead and add that second or third assert statement in your test. Be careful not to fill your test with many asserts, it becomes difficult to understand and debug.

UPDATE:
Folks, I’m sorry for using a naive code example. I agree the example does not do justice to the point I’m trying to make.

@Robert, I certainly like your test. Much better.

Here is another naive example, looking forward to brickbats ;)

Let’s assume I’m building a RomanDigit class and I need to make sure that RomanDigits are map friendly, .i.e. they work correctly when added to maps. Following is a test for the same:

@Test
public void isMapFriendly() throws Exception {
	RomanDigit one = new RomanDigit('I', 1);
	RomanDigit anotherOne = new RomanDigit('I', 1);
	Map romanDigits = new HashMap();
	romanDigits.put(one, "One");
	romanDigits.put(anotherOne, "AnotherOne");
	assertEquals(1, romanDigits.size());
	assertEquals("AnotherOne", romanDigits.get(one));
	assertEquals("AnotherOne", romanDigits.get(anotherOne));
}

Another example: When subtracting RomanDigits, only powers of ten can repeat.

@Test
public void onlyPowerOfTenCanRepeate() throws Exception {
	assertTrue("I is a power of Ten and can repeat", romanDigit('I').canRepeat());
	assertFalse("V is not a power of Ten and should not repeat", romanDigit('V').canRepeat());
}

Is there a better way to write these tests so that I only need one assert statement per test?

  • http://berndschiffer.blogspot.com/ Bernd Schiffer

    I like the Single Assert Per Unit Test pattern. And I’m not dogmatic about it. But in your two examples, I think the tests would be more clear with only one assertion per test.

    addingElementIncreasesSizeOfTheList don’t need to show an empty list’s size. sizeOfAnEmptyList shows that.

    reducesSizeByOneOnRemovingAnElement don’t need to show a list’s size after you’ve added an element to that list. addingElementIncreasesSizeOfTheList already shows that.

    In both cases IMO having only a single assertion per test makes the tests not only DRY, it makes them more focussed and less verbose.

  • http://twitter.com/bostonaholic Matthew Boston

    In your test, reducesSizeByOneOnRemovingAnElement(), you’re actually asserting _two_ aspects. I believe it shouldn’t have the first assert, which is asserting that adding an element increases the size by 1. If your list.add() method becomes compromised, this test could potentially fail at the first assert, which could be misleading to whomever is reviewing failed tests.

  • http://twitter.com/bgswan Brian Swan

    I totally agree that the one assert per test practice is unhelpfully dogmatic but I think the examples you give have arguably unnecessary asserts.

    In the test reducesSizeByOneOnRemovingAnElement() if we are confident in the behaviour of the list.add method then the first assert is unnecessary.

    The example I would use would assert the state of some other query methods on the list interface, e.g.

    @Test
    public void addAnElementToAList() {
    list.add(“Element”);
    assertEquals(1, list.size());
    assertFalse(list.isEmpty());
    }

  • http://openid-provider.appspot.com/robertdw Robert Watkins

    I’m not a dogmatic “one assert per test” guy either, but… your test arguably could be written as:

    public void addingElementIncreasesSizeOfTheList() {
    int sizeBeforeAddition = list.size();
    list.add(“Element”);
    assertEquals(1, list.size() – sizeBeforeAddition);
    }

    From a purist point of view, you were testing two aspects of the list – the additional aspect being that a new list is empty.

  • http://blogs.agilefaqs.com Naresh Jain

    Folks, I’m sorry for using a naive code example. I agree the example does not do justice to the point I’m trying to make.

    @Robert, I certainly like your test. Much better.

    Here is another naive example, looking forward to brickbats ;)

    Let’s assume I’m building a RomanDigit class and I need to make sure that RomanDigits are map friendly, .i.e. they work correctly when added to maps. Following is a test for the same:

    @Test
    public void areMapFriendly() throws Exception {
    	RomanDigit one = new RomanDigit('I', 1);
    	RomanDigit anotherOne = new RomanDigit('I', 1);
    	Map romanDigits = new HashMap();
    	romanDigits.put(one, "One");
    	romanDigits.put(anotherOne, "AnotherOne");
    	assertEquals(1, romanDigits.size());
    	assertEquals("AnotherOne", romanDigits.get(one));
    	assertEquals("AnotherOne", romanDigits.get(anotherOne));
    }

    Is there a better way to write this test so that I only need one assert statement per test?

  • http://blogs.agilefaqs.com Naresh Jain

    Folks, I’m sorry for using a naive code example. I agree the example does not do justice to the point I’m trying to make.

    @Robert, I certainly like your test. Much better.

    Here is another naive example, looking forward to brickbats ;)

    Let’s assume I’m building a RomanDigit class and I need to make sure that RomanDigits are map friendly, .i.e. they work correctly when added to maps. Following is a test for the same:

    @Test
    public void areMapFriendly() throws Exception {
    	RomanDigit one = new RomanDigit('I', 1);
    	RomanDigit anotherOne = new RomanDigit('I', 1);
    	Map romanDigits = new HashMap();
    	romanDigits.put(one, "One");
    	romanDigits.put(anotherOne, "AnotherOne");
    	assertEquals(1, romanDigits.size());
    	assertEquals("AnotherOne", romanDigits.get(one));
    	assertEquals("AnotherOne", romanDigits.get(anotherOne));
    }
    

    Is there a better way to write this test so that I only need one assert statement per test?

  • http://blogs.agilefaqs.com Naresh Jain

    Folks, I’m sorry for using a naive code example. I agree the example does not do justice to the point I’m trying to make.

    @Robert, I certainly like your test. Much better.

    Here is another naive example, looking forward to brickbats ;)

    Let’s assume I’m building a RomanDigit class and I need to make sure that RomanDigits are map friendly, .i.e. they work correctly when added to maps. Following is a test for the same:

    @Test
    public void areMapFriendly() throws Exception {
    	RomanDigit one = new RomanDigit('I', 1);
    	RomanDigit anotherOne = new RomanDigit('I', 1);
    	Map romanDigits = new HashMap();
    	romanDigits.put(one, "One");
    	romanDigits.put(anotherOne, "AnotherOne");
    	assertEquals(1, romanDigits.size());
    	assertEquals("AnotherOne", romanDigits.get(one));
    	assertEquals("AnotherOne", romanDigits.get(anotherOne));
    }

    Is there a better way to write this test so that I only need one assert statement per test?

  • http://blogs.agilefaqs.com Naresh Jain

    Folks, I’m sorry for using a naive code example. I agree the example does not do justice to the point I’m trying to make.

    @Robert, I certainly like your test. Much better.

    Here is another naive example, looking forward to brickbats ;)

    Let’s assume I’m building a RomanDigit class and I need to make sure that RomanDigits are map friendly, .i.e. they work correctly when added to maps. Following is a test for the same:

    @Test
    public void isMapFriendly() throws Exception {
    	RomanDigit one = new RomanDigit('I', 1);
    	RomanDigit anotherOne = new RomanDigit('I', 1);
    	Map romanDigits = new HashMap();
    	romanDigits.put(one, "One");
    	romanDigits.put(anotherOne, "AnotherOne");
    	assertEquals(1, romanDigits.size());
    	assertEquals("AnotherOne", romanDigits.get(one));
    	assertEquals("AnotherOne", romanDigits.get(anotherOne));
    }

    Is there a better way to write this test so that I only need one assert statement per test?

  • http://blogs.agilefaqs.com Naresh Jain

    Folks, I’m sorry for using a naive code example. I agree the example does not do justice to the point I’m trying to make.

    @Robert, I certainly like your test. Much better.

    Here is another naive example, looking forward to brickbats ;)

    Let’s assume I’m building a RomanDigit class and I need to make sure that RomanDigits are map friendly, .i.e. they work correctly when added to maps. Following is a test for the same:

    @Test
    public void isMapFriendly() throws Exception {
    	RomanDigit one = new RomanDigit('I', 1);
    	RomanDigit anotherOne = new RomanDigit('I', 1);
    	Map romanDigits = new HashMap();
    	romanDigits.put(one, "One");
    	romanDigits.put(anotherOne, "AnotherOne");
    	assertEquals(1, romanDigits.size());
    	assertEquals("AnotherOne", romanDigits.get(one));
    	assertEquals("AnotherOne", romanDigits.get(anotherOne));
    }

    Another example: When subtracting RomanDigits, only powers of ten can repeat.

    @Test
    public void onlyPowerOfTenCanRepeate() throws Exception {
    	assertTrue("I is a power of Ten and can repeat", romanDigit('I').canRepeat());
    	assertFalse("V is not a power of Ten and should not repeat", romanDigit('V').canRepeat());
    }

    Is there a better way to write these tests so that I only need one assert statement per test?

  • http://blogs.agilefaqs.com Naresh Jain

    Folks, I’m sorry for using a naive code example. I agree the example does not do justice to the point I’m trying to make.

    @Robert, I certainly like your test. Much better.

    Here is another naive example, looking forward to brickbats ;)

    Let’s assume I’m building a RomanDigit class and I need to make sure that RomanDigits are map friendly, .i.e. they work correctly when added to maps. Following is a test for the same:

    @Test
    public void isMapFriendly() throws Exception {
    	RomanDigit one = new RomanDigit('I', 1);
    	RomanDigit anotherOne = new RomanDigit('I', 1);
    	Map romanDigits = new HashMap();
    	romanDigits.put(one, "One");
    	romanDigits.put(anotherOne, "AnotherOne");
    	assertEquals(1, romanDigits.size());
    	assertEquals("AnotherOne", romanDigits.get(one));
    	assertEquals("AnotherOne", romanDigits.get(anotherOne));
    }

    Another example: When subtracting RomanDigits, only powers of ten can repeat.

    @Test
    public void onlyPowerOfTenCanRepeate() throws Exception {
    	assertTrue(romanDigit('I').canRepeat());
    	assertFalse(romanDigit('V').canRepeat());
    }

    Is there a better way to write these tests so that I only need one assert statement per test?

  • http://thinkaround.blogspot.com/ dexter

    For isMapFriendly, split the size and get asserts into seperate tests as they deal with different aspects of map friendliness and then rewrite the get assert like so…

    assertSame(romanDigits.get(one), romanDigits.get(anotherOne));

    The other one is easier, split the positive and negative test.

    • http://blogs.agilefaqs.com Naresh Jain

      assertSame(romanDigits.get(one), romanDigits.get(anotherOne));is a good suggestion. Agree. Only catch is, we are missing the fact that both return “AnotherOne” and not “One”. If both returned “One”, which is wrong, your test would still pass.

      Also I don’t see how checking the size is different aspect. Can you please explain?Also splitting the positive & negative test what would you call each test method?(funny you call them +ve and -ve, I don’t see one as -ve, they are both expected behavior)

      • http://thinkaround.blogspot.com/ dexter

        Let’s see, there’s 3 things to verify here WRT the map, it should…

        @Test
        public void shouldGetValueAssociatedWithEqualKey() {
        Map romanDigits = new HashMap();
        Object slug = new Object();
        romanDigits.put(new RomanDigit(‘I’, 1), slug);
        assertThat(romanDigits.get(new RomanDigit(‘I’, 1)), is(sameInstance(slug)));
        }

        @Test
        public void shouldMaintainOnlyOneEntryPerKey() {
        Map romanDigits = new HashMap();
        romanDigits.put(new RomanDigit(‘I’, 1), new Object());
        romanDigits.put(new RomanDigit(‘I’, 1), new Object());
        assertThat(map.size(), is(1));
        }

        @Test
        public void shouldGetLastValueAssociatedWithKey() {
        Map romanDigits = new HashMap();
        romanDigits.put(new RomanDigit(‘I’, 1), new Object());
        Object slug = new Object();
        romanDigits.put(new RomanDigit(‘I’, 1), slug);
        assertThat(romanDigits.get(new RomanDigit(‘I’, 1)), is(sameInstance(slug)));
        }

        As for the canRepeat tests, I apologize for the unfortunate choice of words when i referred to the test as +ve/-ve; I merely intended to seperate out the polar case like so…

        @Test
        public void shouldPermitExponentsOfTenToRepeat() {
        assertThat(romanDigit(‘I’).canRepeat(), is(true));
        }

        @Test
        public void shouldNotPermitOtherDigitsToRepeat() {
        assertThat(romanDigit(‘V’).canRepeat(), is(true));
        }

        … although, given the limited set of roman digits, you might just want to build sets (potentially EnumSets) of digits pertinant to each test and iterate over them.

        That said, I agree with @Robert above that there’s no need to be overly dogmatic about this. It’s a good guideline to create fine grained tests that give you better feedback, but if you feel the granularity is approaching unnecessary levels of resolution, coarser tests wouldn’t amount to blasphemy.

  • http://twitter.com/bgswan Brian Swan

    I think it’s difficult to evaluate the one assert per test style when just looking at a single test method. The original Dave Astels approach suggests multiple test classes (fixtures) setting up the object under test in different ways http://www.artima.com/weblogs/viewpost.jsp?thread=35578

    I like to think about tests in terms of the command query separation principle, what we’re testing are the commands and what we assert on are the query methods.

    The extreme example of the one assert per test style wouldn’t call a command outside the setup method and would have a test method per query.

    I prefer a style where each test method is its own little example and potentially does additional setup followed by invoking a command and then asserting on the state of the query methods (Assemble,Act,Assert or Given,When,Then in each test method).

  • Greg Symons

    Certainly I agree that some behaviors have multiple side effects that must be verified, but the problem with multiple asserts in a test method is that the diagnostic no longer tells the wholeheartedly story when the test fails. Some frameworks have weaker asserts that are collected rather than immediately terminating the test (Google Test has EXPECT_*, JUnit has Rules); these can be used to make the diagnostics clearer on multiple assertion tests. But if the test framework doesn’t support such weak assertions, it may be better to use a test case class per behavior rather than a method per behavior and keep to the one assertion per method rule.

    One of the most important things a test can do is fail clearly.

    • http://blogs.agilefaqs.com Naresh Jain

      Thanks Greg. Totally agree, tests should have a single responsibility and they should at the very least fail clearly.

  • http://twitter.com/stoneass Assaf Stone

    I believe that you could easily make the first test run with a single assertion.Moreover, I think it would be clearer, as I’m not testing the value of list.size(), but rather that it increases (as per the name of the test). Here’s my go at it:

    @Test
    public void addingElementIncreasesSizeOfTheList() {
    int previousSize = list.size();
    list.add(“Element”);
    assertEquals(previousSize + 1, list.size());
    }

    Just goes to show that a single assert is a good idea, not a myth :-)

  • Anonymous

    I believe it’s a bad idea to write a test called “isMapFriendly” — it’s something like “RomanDigitClassWorksCorrectly”. It’s much better to write a test named like “AddingTwoIdenticalDigitsToAMapResultsWithOneElement”. However, what you have here is really a single *logical* assert, so it’s still one assert per method.

    I’m not familiar with Java’s HashMap, but do you really need all three asserts here?

    As for “onlyPowerOfTenCanRepeate”, I’d split it into “powersOfTenCanRepeat” and “nonPowersOfTenCannotRepeat”, but that’s a matter of taste. The goal is to make the test pass as simply as you can. Passing both asserts can be more difficult, so you usually want to pass the first one and then work on the second one.

  • http://www.riedquat.de/ Christian Hujer

    The Single Assert Rule does not mean one physical assert, it means one logical assert per test case.
    If you’re testing that the list object behaves properly for adding one element by checking that its emptiness is false and its size is one, you need two assert statements but this is one logical assert, so the Single Assert Rule is perfectly happy with that.
    The Single Assert Rule is just a manifestation of the SRP (Single Responsibility Principle) in the context of Test Cases. We want that if the test case fails, we pretty much already know what’s wrong without having to look at details of the test case. The test case itself and assertion message should speak for themselves. This will not be true if test cases contain multiple logical asserts. If test cases contain multiple logical asserts, the test case failure no longer clearly communicates what it does and while it failed – the test case is no longer able to communicate its intent.

    In other words, what it says is that in each test case we arrange, then act, then assert and then, if required, annihilate. We don’t do arrange, act, assert, act, assert, act, assert… in the same test case.

    All these principles are guidelines for achieving maintainability. Some of these guidelines can be applied exaggerated, and then become counter-productive.


    Licensed under
Creative Commons License