Agile FAQs
  About   Slides   Home  

 
Managed Chaos
Naresh Jain's Random Thoughts on Software Development and Adventure Sports
     
`
 
RSS Feed

Recent Thoughts
Tags
Recent Comments

Single Assert Per Unit Test: Myth

Sunday, November 14th, 2010

You are telling me, if I were unit testing a List class and I wrote tests like the following with 2 assert statements in them, then I’m violating xUnit testing “best practices”.

@Test
public void addingElementIncreasesSizeOfTheList() {
	assertEquals(0, list.size()); 
       //or assertListSizeIs(0); - define your custom assert statement
	list.add("Element");
	assertEquals(1, list.size()); // assertListSizeIs(1);
}

If so, how and why?

I understand that if I wrote a test like the following its not very communicative:

@Test
public void removeElementBasedOnItsIndexAndReduceSizeByOne() {
	list.add("Element");
	assertEquals("Element", list.remove(0));
	assertEquals(0, list.size());
}

in this case, it might be better to write 2 test instead:

@Test
public void removeElementBasedOnItsIndex() {
	list.add("Element");
	assertEquals("Element", list.remove(0));
}
 
@Test
public void reducesSizeByOneOnRemovingAnElement() {
	list.add("Element");
	assertEquals(1, list.size());
	list.remove(0);
	assertEquals(0, list.size());
}

Notice our goal was better communication and one of the tests we ended up (incidentally) with had just one assert statement in it (which is better than the previous test which was asserting 2 different things).

Our goal is not one assert statement per test. Our goal is better communication/documentation. Using the one-assert-per-test heuristic helps to achieve better communication. But splitting tests into more tests or deleting an important assert statement from a test just to avoid multiple assert statements in a test is mindless dogma.

Conclusion: One-Assert-Per-Test really means test one aspect/behavior in each test (and communicate that clearly in your test name). It does not literally mean, one test should have only one assert statement in it. Test one aspect/behavior in one test and related aspect/behavior can be tested in another test. Sometimes we need more than one assert statement to assert one aspect/behavior and its helps to add those assert statements in the test for better communication.

So don’t be afraid, go ahead and add that second or third assert statement in your test. Be careful not to fill your test with many asserts, it becomes difficult to understand and debug.

UPDATE:
Folks, I’m sorry for using a naive code example. I agree the example does not do justice to the point I’m trying to make.

@Robert, I certainly like your test. Much better.

Here is another naive example, looking forward to brickbats ;)

Let’s assume I’m building a RomanDigit class and I need to make sure that RomanDigits are map friendly, .i.e. they work correctly when added to maps. Following is a test for the same:

@Test
public void isMapFriendly() throws Exception {
	RomanDigit one = new RomanDigit('I', 1);
	RomanDigit anotherOne = new RomanDigit('I', 1);
	Map romanDigits = new HashMap();
	romanDigits.put(one, "One");
	romanDigits.put(anotherOne, "AnotherOne");
	assertEquals(1, romanDigits.size());
	assertEquals("AnotherOne", romanDigits.get(one));
	assertEquals("AnotherOne", romanDigits.get(anotherOne));
}

Another example: When subtracting RomanDigits, only powers of ten can repeat.

@Test
public void onlyPowerOfTenCanRepeate() throws Exception {
	assertTrue("I is a power of Ten and can repeat", romanDigit('I').canRepeat());
	assertFalse("V is not a power of Ten and should not repeat", romanDigit('V').canRepeat());
}

Is there a better way to write these tests so that I only need one assert statement per test?

Key Attributes of a Metrics

Tuesday, June 9th, 2009

A good metrics:

  • Guides/Drives me. When I look at it, I know what to do (how to react to the data highlighted by the metrics)
  • Positively influences (motivates) me to take steps towards improvement
  • Has no ambiguity regarding how the data can be interpreted
  • is Visual: Once understood, it should take less than 3 seconds to get the essence of the metrics
  • Tracks real data; not something else which is easy to measure
  • Is Automated: Eliminates waste and reduces the feedback latency
For example, xUnit report is a good metric. When I look at the results, I know what to do. If I have a failing test I know that needs to be fixed. It drives me positively to fix the test. If all tests are working, then it motivates me to add more tests. There are no 2 different ways to interpret the data. A failing test means program did not meet the expectations and a passing test means everything is working as expected. Also we are tracking real data generated automatically instead of guesswork. 
In my experience focusing on 3 metrics at a time, is very effective. Once we start tracking more than 3 metrics, we loose focus and clarity. 
How do you know which 3 metric to watch?
Usually, I use ToC to figure out what is the biggest bottleneck in my system. Based on the bottleneck, I define 2-3 metrics that we want to measure as we try to address the bottleneck. Hopefully soon, something else will be the bottleneck and we’ll start measuring things related to that.
This is a main reason why I think we cannot have a set of metrics across the organization. Every team has different bottlenecks and they are at a different stage of evolution. Metrics should be dynamic and they should keep evolving. 

MbUnit to NUnit and back

Wednesday, March 18th, 2009

Recently I was helping a team @ Directi working on a .NET project. They were using MbUnit for unit testing and acceptance testing. The team was using VS 2008 as their IDE. (Seriously its a joke to call VS as an IDE).

As I was pairing with one of the developers, I was watching him make code changes, hit F6 to build the project and then switch to MbUnit UI to run the tests. What a freaking waste of time! Ideally I would make a code change and hit a keyboard shortcut to execute the tests.

So I suggested that we use ReSharper plugin or at least use Test Driven .Net . But the big problem was that ReSharper does not support MbUnit, it only support NUnit. So we looked at what was different in MbUnit that was not in NUnit. The main thing that stood out was RowTest feature. But then we soon found NUnit extension for RowTest. Great!

So we went ahead and changed all our MbUnit tests to use NUnit. (It was a breeze). But then, when we tried to run the tests inside VS using resharper, it was detecting all our test fixtures but did not detect any tests in them. Only later, we realized that ReShaper does not support RowTest extension.

So we simply reverted to MbUnit and installed the MbUnit Resharper Plugin to run MbUnit tests from ReSharper.

    Licensed under
Creative Commons License