It’s easy to speak of test-driven development as if it were a single method, but there are several ways to approach it. In my experience, different approaches lead to quite different solutions.

In this hands-on workshop, with the help of some concrete examples, I’ll demonstrate the different styles and more importantly what goes into the moment of decision when a test is written? And why TDDers make certain choices. The objective of the session is not to decide which approach is best, rather to highlight various different approaches/styles of practicing test-driven development.

By the end of this session, you will understand how TTDers break down a problem before trying to solve it. Also you’ll be exposed to various strategies or techniques used by TDDers to help them write the first few tests.

Recently at the Agile India 2013 Conference I ran an introductory workshop on Behavior Driven Development. This workshop offered a comprehensive, hands-on introduction to behavior driven development via an interactive-demo.

Over the past decade, eXtreme Programming practices like Test-Driven Development (TDD) and Behaviour Driven Development (BDD) have fundamentally changed software development processes and inherently how engineers work. Practitioners claim that it has helped them significantly improve their collaboration with business, development speed, design & code quality and responsiveness to changing requirements. Software professionals across the board, from Internet startups to medical device companies to space research organizations, today have embraced these practices.

This workshop explores the foundations of TDD & BDD with the help of various patterns, strategies, tools and techniques.

Is writing inline comments always bad? Are comments really evil? I keep getting these questions over and over again.

Often you see code like this:

// If the item is taxable, get the taxed amount using tax calculator
if( objItem.bTaxable )
{
	objItem.fTax = objCalculator.TaxForLocal(objItem.fItemRate);
}
 
// Additional tax is applicable if the item is an imported one
if( objItem.bImported )
{
	objItem.fTax += objCalculator.TaxForImported(objItem.fItemRate);
}
 
// Add tax to item rate
objItem.fTaxedRate = objItem.fItemRate + objItem.fTax;
 
// Return the final amount
double fFinalAmount = objItem.fTaxedRate * objItem.nNumberOfItems;
return fFinalAmount;

What is the real value of these comments?

When I see stuff like this, I usually tell people

When I was learning programming, I was thought that great programmers write great comments. These days I tell people lousy programmer write comments.

Immediately people who write inline-comments get defensive. And that’s completely understandably. I don’t think we’ve really explained our rationale for making such a ridiculous statement. So let me step back and explain the rationale.

Folks in the extreme-programming community will tell you:

Comments are often used as deodorant. Comments represent a failure to express an idea in the code. Try to make your code self-documenting or intention-revealing. When you feel like writing a comment, first try to refactor so that the comment becomes superfluous.

Most people will also tell you, that the biggest problem with comments is that they soon become outdated. The original intent of the person writing the comment was to help a developer who comes later to understand the code better. But unfortunately over a period of time, the comments get outdated and it adds more to the confusion. Speaking to many programmer, they simply delete or ignore the comments because they find them ambiguous. Even though the person who wrote the comments wrote them with a good intension, one needs to ask if it really solved any problem?

And then they question, why not put the same effort and time to write well-crafted code so that comments are never required? Is it impossible to do so?

While this argument is a good one, I find it hard to connivence people just based on this argument.

I’ve found the following approach work really well for me. First let’s understand why programmers write comments. Based on my experience, programmer write in-line comments for 3 different reasons:

1. To explain what the code does
2. To descrive how the code does what is does
3. Why the code is written the way its written

If you think about it, the “what” and “how” of the code should really be expressed by self-documented code. IMHO its simply a failure on part of the programmer if they cannot express the “what” and “how” in the code itself.

However the “why” is little bit more tricky. It’s a reminder, telling us: “Hey, you are doing something complicated and someone else will not understand why. Even if you wrote a comment, they might not necessarily understand it.” At this point I might stop and see if there is a better way to design/model/code this, such that the why becomes obvious via the code. This is certainly more challenging and time consuming than to write a comment and moving on. However this short-term hack might bite me back. Luckily, most often than not, I can find a way to avoid the comment. But there are special cases when I need a comment to explain the why. Let’s see a few examples:

• There is a bug in the underlying framework/library I’m using. Searching on the net, I found the bug report and a workaround. Looking at just the code might not help someone understand the need for the workaround. Generally I would write a small comment saying Workaround with the version number of the framework/library and add the link to the workaround and continue. In future, someone can remove the workaround & delete the comment if the issue is fixed.
• I’m implementing a complex algo and its not common that everyone understands it. I would add a link to the Algo description (rather than duplicating the algo description in the code. DRY principle applies to comments as well.) and continue with my coding.
• And so on…

So think again before you leave a comment

Should you consider BDD for your team? I just wanted to share my experience with BDD, which you could consider as food for thoughts:

• Why? – How often does your development team ask the business team why certain feature is important? More importantly, why now and how much of it is enough? In my experience with BDD, the approach naturally lends the teams to have a constructive discussion about this. I’ve seen this leading to a genuine collaboration between business and development. In short, BDD helps getting the right people to discuss the right amount (depth) of things at the right time.
• Better commitment and buy-in: BDD lays very heavy emphasis on Business value. It forces the Business to justify the priority by showing concrete value. Also it helps the development teams to embrace the prioritizes set by Business, because they understand the thinking process much better. Interestingly, I’ve noticed that teams that practice BDD, I rarely see either sides throwing their weight around to push their pet features through. Also due to the focus on value, rarely I find teams building useless feature for the sake of proving productivity.
• Ubiquitous domain language: By working together, the business team & the development team build a shared understanding by using tools/techniques like product discovery, story maps, user stories, acceptance criteria and scenarios to create an ubiquitous domain language. The ubiquitous language is usable/understandable not only by domain experts, but also by every member of the team. This goes a long way in reducing the complexity in the domain and making the barrier-to-entry much lower for new members.
• Right focus: BDD can help you focus on the user’s needs and the expected behavior instead of jumping ahead and getting caught up in implementation details upfront. Also I’ve seen teams which are new to Test Driven Development (TDD), seem to focus too much on “testing” missing the point. BDD helps the teams focus on system’s behavioral aspects rather than focusing on testing your implementation. Over the years, we’ve learned that up-front analysis, design, planning and test planning, all have a diminishing return. In my experience BDD helps strike the balance.
• Evolutionary Design: Agile (eXtreme Programming in particular) has killed the notion of “fully-specified, completely-detailed-out requirements upfront.” It is a well accepted fact that we won’t be able to figure out all the requirements when we are starting off with a project. BDD, at its very crux, embraces the fact of evolving product understanding and hence helps your system’s design evolve to keep pace with the changing needs.
• Breaking the knowledge silos in distributed team: Having worked with distributed teams for over a decade, due to the distributed nature, one common problem I see is domain knowledge gap between the team members. Esp. teams which are away from the real business, have a hard time understanding and questioning the business requirements. Usually there are gaps or misunderstanding, leading to rework and frustration. Even if your team is not distributed, you might be familiar with the problems of knowledge silos. I don’t think BDD alone solves this problem, but at least it encourage teams to work more closely and reduces the gap. It also encourages a generalizing specialist attitude in team members, helping with other eXtreme programing practices like pairing, collective ownership, etc.
• Greater ROI: Behavior has a much longer shelf life than your implementation and implementation related documentation. Also as the business rules change, BDD captures it better than it being lost in documents and code.
• Predictability & Confidence: Also in my experience working with teams using BDD, I see that they have much better (or at least feel they have better) predictability and confidence in their work.

What’s your experience?

Also to clarify, what I mean by Business team and Development team, my definitions below:

Business Team: Folks who focus on “Are we building the right product.” This usually includes the following roles: Product Owner/Manager, Stakeholders, Product Designer (User Experience), Subject Matter Expert, Business Analysts, and so on.

Development Team: Folks who focus on “Are we building the product right.” This usually includes the following roles: Architects, Tech Leads, Developers, Testers, Graphics designers, Database administrators and so on.

I’ve come to believe that Code is a liability. The less of it we have, the better off we are. Since code is a liability, would you not take extra care to keep it minimal and simple?

However I keep running into code shown below. I still wonder what drives developers to write more code, when they can do away with a fraction of it?

public class Calendar {
	List<Integer> busySlots;
 
	public void addBusySlot(int i) {
		if(busySlots==null){
			busySlots=new ArrayList<Integer>();
		}
		busySlots.add(i);		
	}
 
	public boolean isFree(int time) {
		if(busySlots==null)
			return true;
		return (!(busySlots.contains(time)));
	}
}

vs.

public class Calendar {
	private final List<Integer> busySlots = new ArrayList<Integer>();
 
	public void addBusySlot(int time) {
		busySlots.add(time);		
	}
 
	public boolean isFree(int time) {
		return (!(busySlots.contains(time)));
	}
}

In fact I would argue that this is a Lazy Class and you can simply use the list directly where ever you need the Calendar.

Another example:

if(some_condition == true) {
    return true;
} else {
    return false;
}

vs.

return some_condition;

Suppose we had the following (utterly) simplistic requirement:

@Test
public void politiciansDontNeedToPayTax() {
    whenIncomeIs(10000).verify(POLITICIANS).pay(0);
}
 
@Test
public void residingAliensGetAwayByPayingLittleTax() {
    whenIncomeIs(10000).verify(ALIENS).pay(1000);
}
 
@Test
public void richPeopleArePunishedWithHighestTax() {
    whenIncomeIs(10000).verify(RICH_PEOPLE).pay(3000);
}

where:

TaxPayer RICH_PEOPLE = new RichPeople();
TaxPayer ALIENS = new ResidingAliens();
TaxPayer POLITICIANS = new Politicians();

To fullfil this requirement, we’ve the following code:

Parent Class:

public abstract class TaxPayer {
    protected abstract double getTaxPercentage();
 
    public double calculateTaxAmount(final long income) {
        if (getTaxPercentage() == 0)
            return 0;
        return income * getTaxPercentage() / 100.0;
    }
}

Each child class:

public class Politicians extends TaxPayer {
    private double taxPercentage = 0;
 
    @Override
    protected double getTaxPercentage() {
        return taxPercentage;
    }
}

public class RichPeople extends TaxPayer {
    private final double taxPercentage = 30;
 
    @Override
    protected double getTaxPercentage() {
        return taxPercentage;
    }
}

public class ResidingAliens extends TaxPayer {
    private final double taxPercentage = 10;
 
    @Override
    protected double getTaxPercentage() {
        return taxPercentage;
    }
}

One would wonder what good are these child classes? Feels like a class explosion. Sure enough! This is the pathetic state of Object Oriented programming in many organizations, esp. the Enterprise software side of the world.

One could have easily used an enum to solve this problem:

public enum TaxPayer {
    Politicians(0), Aliens(.1), RichPeople(.3);
 
    private double taxPercentage;
 
    private TaxPayer(double taxPercentage) {
        this.taxPercentage = taxPercentage;
    }
 
    public double calculateTaxAmount(final long income) {
        return income * taxPercentage;
    }
}

Unfortunately many developers think they already know enough values and principles, so its not worth spending anytime learning or practicing them. Instead they want to learn the holy grail of software design – “the design patterns.”

While the developers might think, design patterns is the ultimate nirvana of designing, I would say, they really need to learn how to think in their paradigm plus internalize the values and principles first. I would suggest developers should focus on the basics for at least 6 months before going near patterns. Else they would gain a very superficial knowledge about the patterns, and won’t be able to appreciate its true value or applicability.

Any time I get a request for teaching OO design patterns, I insist that we start the workshop with a recap of the values and principles. Time and again, I find developers in the class agreeing that they are having a hard time thinking in Objects. Most developers are still very much programming in the procedural way. Trying to retrofit objects, but not really thinking in-terms of objects and responsibilities.

Recently a student in my class shared that:

I thought we were doing Object Oriented Programming, but clearly its a paradigm shift that we still need to make.

On the first day, I start the class with simple labs, in the first 4-5 labs, you can see the developers struggle to come up with a simple object-oriented solution for basic design problems. They end up with a very procedural solution:

• main class with a bunch of static methods or
• data holder classes with public accessors and other or/er classes pulling that data to do some logic.
• very heavy use of if-else or switch

Its sad that teams don’t understand nor give importance to the following important OO concepts:

• Data and Logic together – Encapsulation (Everyone knows the definition of encapsulation, but how to put it in proper use, is always a big question mark.) Many developers write public Getters/Setters or Accessors by default on their classes. And are shocked to realize that it breaks encapsulation.
• Inheritance is the last option: It is quite common to see solutions where slight variation in data is modeled as a hierarchy of classes. The child classes have no behavior in them and often leads to a class explosion.
• Composition over Inheritance – In various labs, developers go down the route of using Inheritance to reuse behavior instead of thinking of a composition based design. Sadly, inheritance based solutions have various flaws that the team can’t realize, until highlighted. Coming up with a good inheritance based design, when the parent is mutable, it extremely tricky.
• Focus on smart data-structure: The developers have a tough time coming up with smart data-structure and putting logic around it. Via various labs, I try to demonstrate how designing smart data-structures can make their code extremely simple and minimalistic.

I’ve often noticed that, when you give developers a problem, they start off by drawing a flow chart, data-flow diagram or some other diagram which naturally lends them into a procedural design.

Thinking in terms of Objects requires thinking of objects and responsibilities, not so much in terms of flow. Its extremely important to understand the problem, distill it down to the crux by eliminating noise and then building a solution in an incremental fashion. Many developers have a misconception that a good designs has to be fully flushed out before you start. I’ve usually found that a good design emerges in an incremental fashion.

Even though many developers know the definition of high-cohesion, low-coupling and conceptual integrity, when asked to give a software or non-software example, they have a hard time. It goes to show that they don’t really understand the concept in action.

Developers might have read the bookish definition of the various Design Principles. But when asked to find out what design principles were violated in a sample design, they are not able to articulate. Also often you find a lot of misconception about the various principles. For example, Single Responsibility, few developers say that each method should do only one thing and a class should only have one responsibility. What does that actually mean? It turns out that SRP has to do more with temporal symmetry and change. Grouping behavior together from a change point of view.

Even though most developers raise their hands when asked if they know code smells, they have a tough time identifying them or avoiding them in their design. Developers need a lot of hands-on practice to recognize and avoid various code smells. Once you learn to recognize code smells, the next step is to learn how to effectively refactor away from them.

Often I find developers have the most expensive and jazzy tools & IDEs, but when you watch them code, they use their IDEs just as a text-editor. No automated refactoring. Most developers type “Public class xxx” instead of writing the caller code first and then using the IDE to generate the required skeleton code for them. Use of keyboard shortcuts is as rare as seeing solar eclipse. Pretty much most developers practice what I call mouse driven programming. In my experience, better use of IDE and Refactoring tools can give developers at least 5x productivity boost.

I hardly remember meeting a developer who said they don’t know how to unit test. Yet time and again, most developers in my class struggle to write good unit tests. Due to lack of familiarity or lack of practice or stupid misconceptions, most developers skip writing any automated unit tests. Instead they use Sysout/Console.out or other debugging mechanism to validate their logic. Getting better at their unit testing skills and then gradually TDD can really give them a productivity boost and improve their code quality.

I would be extremely happy if every development shop, invested 1 hour each week to organize a refactoring fest, design fest or a coding dojo for their developers to practice and hone their design skills. One can attend as many trainings as they want, but unless they deliberately practice and apply these techniques on job, it will not help.

Following Object Oriented Design Principles have really helped me designing my code:

• Single Responsibility Principle
• Open Closed Principle
• Liskov Substitution Principle or Design by Contract
• Interface Segregation Principle
• Dependency InversionPrinciple
• Once and Only Once
• DRY – Don’t Repeat yourself
• Tell Don’t Ask
• The Law of Demeter
• Triangulate - When you are not sure what the correct abstraction should be, instead of pulling out an abstraction upfront, you get the second case to work by duplicating and modifying a small piece of code. Once you have both the solutions working, find the “generic” form and create an abstraction

Along with these principles, I’ve also learned a lot from the 17 rules explained in the Art of Unix Programming book:

• Rule of Modularity: Write simple parts connected by clean interfaces
• Rule of Clarity: Clarity is better than cleverness.
• Rule of Composition: Design programs to be connected to other programs.
• Rule of Separation: Separate policy from mechanism; separate interfaces from engines
• Rule of Simplicity: Design for simplicity; add complexity only where you must
• Rule of Parsimony: Write a big program only when it is clear by demonstration that nothing else will do
• Rule of Transparency: Design for visibility to make inspection and debugging easier
• Rule of Robustness: Robustness is the child of transparency and simplicity
• Rule of Representation: Fold knowledge into data so program logic can be stupid and robust
• Rule of Least Surprise: In interface design, always do the least surprising thing
• Rule of Silence: When a program has nothing surprising to say, it should say nothing
• Rule of Repair: When you must fail, fail noisily and as soon as possible
• Rule of Economy: Programmer time is expensive; conserve it in preference to machine time
• Rule of Generation: Avoid hand-hacking; write programs to write programs when you can
• Rule of Optimization: Prototype before polishing. Get it working before you optimize it
• Rule of Diversity: Distrust all claims for “one true way”
• Rule of Extensibility: Design for the future, because it will be here sooner than you think

“Its God’s gift” or “S/he was born talented” or “S/He just lucky” is a common myth that undermines the relentless hard-work experts put to attain mastery in their respect work.

Benjamin Bloom, a pioneer who broke this myth found out that:

“All the superb performers, he investigated, had practiced intensively, had studied with devoted teachers, and had been supported enthusiastically by their families throughout their developing years.”

Later research, building on Bloom’s study revealed that the amount and quality of practice were key factors in the level of expertise people achieved.

Consistently and overwhelmingly, the evidence showed that:

“Experts are always made, not born.”

The journey to truly superior performance is neither for the faint hearted nor for the impatient. The development of genuine expertise requires struggle, sacrifice, and honest, often painful self-assessment. There are no shortcuts. It will take many years if not decades to achieve expertise, and you will need to invest that time wisely, by engaging in “deliberate” practice; practice that focuses on tasks beyond your current level of competence and comfort. You will need a well-informed coach not only to guide you through deliberate practice but also to help you learn how to coach yourself.

One study showed that psychotherapists with advanced degrees and decades of experience aren’t reliably more successful in their treatment of randomly assigned patients than novice therapists with just three months of training are. There are even examples of expertise seeming to decline with experience. The longer physicians have been out of training, for example, the less able they are to identify unusual diseases of the lungs or heart. Because they encounter these illnesses so rarely, doctors quickly forget their characteristic features and have difficulty diagnosing them.

Practice Deliberately: Not all practice makes you perfect. You need a particular kind of practice – “deliberate practice” – to develop expertise. When most people practice, they focus on the things they already know how to do. Deliberate practice is different. It entails considerable, specific, and sustained efforts to do something you can’t do well – or even at all.

Let’s imagine you are learning to play golf for the first time. In the early phases, you try to understand the basic strokes and focus on avoiding gross mistakes (like driving the ball into another player). You practice on the putting green, hit balls at a driving range, and play rounds with others who are most likely novices like you. In a surprisingly short time (perhaps 50 hours), you will develop better control and your game will improve. From then on, you will work on your skills by driving and putting more balls and engaging in more games, until your strokes become automatic: You’ll think less about each shot and play more from intuition. Your golf game now is a social outing, in which you occasionally concentrate on your shot. From this point on, additional time on the course will not substantially improve your performance, which may remain at the same level for decades.

Why does this happen?

You don’t improve because when you are playing a game, you get only a single chance to make a shot from any given location. You don’t get to figure out how you can correct mistakes. If you were allowed to take five to ten shots from the exact same location on the course, you would get more feedback on your technique and start to adjust your playing style to improve your control. In fact, professionals often take multiple shots from the same location when they train and when they check out a course before a tournament.

Computer gaming is an excellent example where I’ve seen people practice deliberately to get better. They focus on what they can do well, but they also focus on what they can’t do well. Most importantly, when practicing, the gamer is not just mindlessly playing. It’s a very thoughtful, deep, dedicated practice session.

War games serve a similar training function at military academies. So do flight simulators for pilots. Unfortunately in software development, very few people practice deliberately.

Genuine experts not only practice deliberately but also think deliberately. The golfer Ben Hogan once explained, “While I am practicing I am also trying to develop my powers of concentration. I never just walk up and hit the ball.”

Deliberate practice involves two kinds of learning:

1. Improving the skills you already have
2. Extending the reach and range of your skills.

“Practice puts brains in your muscles” – Golf champion Sam Snead

The enormous concentration required undertaking these twin tasks limits the amount of time you can spend doing them.

How long should you do deliberate practice each day?

“It really doesn’t matter how long. If you practice with your fingers, no amount is enough. If you practice with your head, two hours is plenty.”

It’s very easy to neglect deliberate practice. Experts who reach a high level of performance often find themselves responding automatically to specific situations and may come to rely exclusively on their intuition. This leads to difficulties when they deal with atypical or rare cases, because they’ve lost the ability to analyze a situation and work through the right response. Experts may not recognize this creeping intuition bias, of course, because there is no penalty until they encounter a situation in which a habitual response fails and maybe even causes damage.

Many research show the importance of a coach/mentor in deliberate practice. Some strongly favor an apprenticeship model. However one needs to be aware of the limitation of just following a coach or working alongside an “expert.”

Statistics show that radiologists correctly diagnose breast cancer from X-rays about 70% of the time. Typically, young radiologists learn the skill of interpreting X-rays by working alongside an “expert.” So it’s hardly surprising that the success rate has stuck at 70% for a long time. Imagine how much better radiology might get if radiologists practiced instead by making diagnostic judgments using X-rays in a library of old verified cases, where they could immediately determine their accuracy.

All an all, “Living in a cave does not make you a geologist” .i.e. without deliberate practice you go no where.

Original Article: The Making of an Expert

Recently we realized that our server logs were showing ‘null’ for all HTTP request parameters values.

SEVERE: Attempting to get user with null userName
parameters=[version:null][sessionId:null][userAgent:null][requestedUrl:null][queryString:null]
[action:null][session:null][userIPAddress:null][year:null][path:null][user:null]

On digging around a bit, we found the following buggy code in our custom Map class, which was used to hold the request parameters:

@Override
public String toString() {
    StringBuilder parameters = new StringBuilder();
    for (Map.Entry<String, Object> entry : entrySet())
        parameters.append("[")
                .append(entry.getKey())
                .append(":")
                .append(get(entry.getValue()))
                .append("]");
    return parameters.toString();
}

When we found this, many team members’ reaction was:

If the author had written unit tests, this bug would have been caught immediately.

Others responded saying:

But we usually don’t write tests for toString(), Getters and Setters. We pragmatically choose when to invest in unit tests.

As all of this was taking place, I was wondering, why in the first place, the author even wrote this code? As you can see from the following snippet, Maps already know how to print themselves.

@Test
public void mapKnowsHowToPrintItself() {
    Map hashMap = new HashMap();
    hashMap.put("Key1", "Value1");
    hashMap.put("Key2", "Value2");
    System.out.println(hashMap);
}

Output: {Key2=Value2, Key1=Value1}

Its easy to fall into the trap of first writing useless code and then defending it by writing more useless tests for it.

I’m a lazy developer and I always strive real hard to write as little code as possible. IMHO real power and simplicity comes from less code, not more.