Agile FAQs
  About   Slides   Home  

Managed Chaos
Naresh Jain's Random Thoughts on Software Development and Adventure Sports
RSS Feed
Recent Thoughts
Recent Comments

Unit Testing Dilemma: Should I Invest or Not?

Every single line of code must be unit tested!

This sound advice rather seems quite extreme to me. IMHO a skilled programmer pragmatically decides when to invest in unit testing.

After practicing (automated) unit testing for over a decade, I’m a strong believer and proponent of automated unit testing. My take on why developers should care about Unit Testing and TDD.

However over the years I’ve realized that automated unit tests do have four, very important, costs associated with them:

  • Cost of writing the unit tests in the first place
  • Cost of running the unit tests regularly to get feedback
  • Cost of maintaining and updating the unit tests as and when required
  • Cost of understanding other’s unit tests
One also starts to recognize some other subtle costs associated with unit testing:
  • Illusion of safety: While unit tests gives you a great safety net, at times, it can also create an illusion of safety leading to developers too heavily relying on just unit tests (possibly doing more harm than good.)
  • Opportunity cost: If I did not invest in this test, what else could I have done in that time? Flip side of this argument is the opportunity cost of repetitive manually testing or even worse not testing at all.
  • Getting in the way: While unit tests help you drive your design, at times, they do get in the way of refactoring. Many times, I’ve refrained from refactoring the code because I get intimidated by the sheer effort of refactor/rewrite a large number of my tests as well. (I’ve learned many patterns to reduce this pain over the years, but the pain still exists.)
  • Obscures a simpler design: Many times, I find myself so engrossed in my tests and the design it leads to, that I become ignorant to a better, more simpler design. Also sometimes half-way through, even if I realize that there might be an alternative design, because I’ve already invested in a solution (plus all its tests), its harder to throw away the code. In retrospect this always seems like a bad choice.
If we consider all these factors, would you agree with me that:
Automated unit testing is extremely important, but each developer has to make a conscious, pragmatic decision when to invest in unit testing.
Its easy to say always write unit tests, but it takes years of first-hand experience to judge where to draw the line.
  • “Automated unit testing is extremely important, but each developer has to make a conscious, pragmatic decision when to invest in unit testing.”

    I can see the point you have with this post, particularly with the time cost of maintaining tests and understanding tests, but I’m not sure I agree with the quoted statement.  My trouble with it is that the developers will couch a lazy decision as a “conscious, pragmatic” one.  That is, the ‘pragmatic’ reason for not writing the unit test will be “I don’t feel like it.”  Of course, that’s not what they’ll actually say, but that’s what it will amount to.  

    As I see it, all of these objections that you raise here as valid ones that people might have (I see from your other posts that you’re a TDD advocate, so I’m addressing these hypothetical other developers) are ones that could be addressed through practicing the process and getting better at it.  And, if the practice is held up as sub-optimal at times, people have the excuse they’re looking for not to bother practicing it and getting better.

    Personally, I’d amend the statement to be “Manual* unit testing is extremely important, and it requires that each developer make a conscious, pragmatic decision to invest time in improving his or her skills at it.”

    Interesting post, though.  I read it and some of your other ones about TDD, and they’ve given me food for thought.

    (*I say “manual” because I consider tests that I write by hand to be manual and ones generated by a utility like Pex to be automated)

    • Eric, thanks for sharing your views with us.

      What do you propose we should do, to deal with those “other” developers who will make stupid excuses not to do a good job?
      Also my concern is why are people so worried about those “other” developers? Does this worry really help? 

      If we are so sure about the importance of *Manual* unit testing, does Darwin’s “survival of the fittest” or “natural selection” does not apply in our field?

      • What to do with the others depends, I suppose, on one’s position in relation to them.  I’m not a manager, and even if I were, I don’t believe that I would mandate unit testing.  I think, instead, the thing to do might be to create an environment where the benefits were demonstrated.  For instance, if there are code reviews to be done, rather than pointing out mistakes, commit unit tests that expose those mistakes and charge the developer with making the tests pass.  This is an introduction to tests that’s relatively gentle and easy to understand.  “Green means fixed.”  On the flip side, if reviewed developers contend that their code is correct when the reviewers think it isn’t, challenge them to prove it with unit tests or change it.

        As to survival of the fittest, I don’t think that applies in the sense that non-testers will be filtered out.  It might if you compared a series of freelance contractors who worked as individuals and were tasked with maintaining code and dealing with their users directly, but in the world of organizations, I don’t think it will work out that way.  Often times there are different maintenance programmers and developers, and often times defects and problems crop up in the distant future when the cause has either left the company or else no one really knows exactly what the cause was.  Add to that the fact that many short-sighted managers and shops without a real process create an environment where “bang out code and worry about maintenance tomorrow” carries the day, non-testing developers might prove ‘fittest’ in some situations.  If they’re behind schedule and tell their manager they won’t bother to test because that takes up too much time, they will be rewarded and hence prove fittest.  And, though more macroscopic Darwinism kicks in often times with such shops themselves, the individual developers will move on with their references and accolades and repeat this anti-pattern elsewhere.

  • I had these kind of thoughts before I really understood TDD.

    But then from experience, you learn that when everything goes green – it doesn’t mean anything – there could still be holes because you were “judicious” in deciding what you “felt” was important to test.

    If your business insists you must rush to get a feature out, and they accept that the long term will be exponentially worse than the short term gain – then skip the tests. But TDD is a design process, and often guides you down the route of getting things done in the shortest amount of time. Especially when you are prototyping and revising your design all the time.

    But hey, it’s good to question things and you absolutely should. 

    Licensed under
Creative Commons License