Design is the art of making constant trade-off decisions; Good Design has balanced Trade-offs!

The problem with design, as with life is that …… You can’t have it all!

Its easy to stick with extremes (complete upfront design v/s  code & fix style design), but its really hard to strike a balance (just enough or just in time design). Just enough is relative and means different things to different people. Also varies from context to context. Just enough design in one context can mean over-engineeing in another context.

Also, there is no ONE way of doing things. There are always options when it comes to designing software and one has to choose a approach that makes most sense. Is that the best design? Do you really need the best design? Its always easy to find a better design in retrospect.

Hence I say, the only Universal truth is that there are no Universal truths. Even law of gravity does not apply universally.

My 2 cents to help with all the confusion out there with the word Design. (when I say Design, I really mean Object Oriented Design).

When we are developing and hence designing using Test Driven Development (TDD), the meaning of the word design is very different from what it means in the context of Big Upfront Design (BUFD). Even if we are not referring to BUFD, the amount of design required can vary significantly. Some people even challenge tge whole notion of design being different from code. There has been (and will continue to be) a disagreement regarding how much to design before writing code?

Of late I find myself using the phrase micro-design to refer to the baby steps in the evolutionary design process, the kind of design that is really driven by the unit tests. While micro-design is very important and an integral part of development, there are times when you need a slightly bigger picture. For Ex. sometimes a feature can span across the software system and scope of this design is slightly more than micro-design, but very less compared to BUFD. I refer to the design step of this scope in the evolutionary design process as macro-design. I find prototypes and acceptance tests are really great at driving macro-design.

I find it really useful to call out to developers that this is micro-design step v/s this is macro-design step v/s this is BUFD.

I’m about to make a rather controversial statement now: “Developers who use purely inside-out avatar of TDD, .i.e. use unit tests to drive their development, are really using tests to validate their design and not so much to drive the design”.

The reason I think this way is because in my experience, developers who drive development using unit tests already have a mental model, whiteboard model, CRC card model, etc. of their design. Unit tests really helps them validate that design. (This is clearly not same as BUFD (Big Upfront Design). In this context its micro-design, design of a small feature not the whole system). Of course there is some amount of driving the design as well. When developers find out that the design they had thought about does not really fit well, they evolve it and use unit tests to get feedback. But as you can see the intent is not really to discover a new design by writing the test first.

Is this wrong? Is this a sign that you are a poor developer? Absolutely no. Its perfectly fine to do so and it also works really well in some cases for some people. But I want to highlight that there is another approach to influence design as well.

In my experience, using an Acceptance tests helps better with driving/discovering micro-design. When your test is not in the implementation land, when its in the defining-the-problem-land, there are better chances of tests helping you to discover the design.

While TDD is great at driving/discovering micro-design, Prototyping is great as driving/discovering macro-design.

At the Agile 2008 conference in Toronto, Bill Wake and I faciliatated a workshop on Styles of TDD (had to change the name from “Avatars of TDD”, based on feedback from people that most people don’t know the word Avatar). You can find a quick summary of the workshop here : http://agile2008toronto.pbwiki.com/Styles-of-TDD

http://nareshjain.com and http://nareshjain.in

This is the end
International Travel, the end
This is the end
Flying International Airlines, the end

Policies and Invitation letters, the end
Long Visa application queues, the end
Customs and immigrations, the end
I’ll never live out of a suitcase…again

Forex and conversion fees, the end
Jet-lags and sleep-walking, the end
Bland food with Tabasco, the end
I’ll never lose days traveling…again

Lousy long flights, the end
Living at your mercy, the end
Delays and surprises, the end
I’ll never live through this…again

Can you picture what will be
So limitless and free
Never in need of some stranger’s hand
In a…strange land

It hurts to set you free
But you’ll never follow me
The end of laughter and soft lies
The end of nights we tried to fly

After a very rough year with international travel, I’m glad to say, no more international travel for me. (not at least for work or conferences).

* Altered lyrics from “The End” by The Doors. Long live Jim Morrison.

I used to think that Design really beginnings once you have the requirement (user story) in place. Over the years, I have learnt that design is not just a phase on the project. Design is quite intertwined with other activities we do during development and is spread through-out the life-cycle of a project. IMHO, there’s no discrete point in time on your project when you are not designing. To put some meat to this topic, let me share a recent experience with you which will reinforce my point.

A good buddy of mine, (who had been a TDD practitioner for a while) called me up and told me to have a look at some code. He was not very comfortable with the way the classes were interact with each other (design smell). I looked over the code and I agreed with him. The classes depended on each other in an awkward manner. Then I looked at the tests and I felt the tests were not really right. Could not tell why exactly, but something did not seem right. I asked him to show me the user story he was working on. I looked at the story and I realized what was happening. The way he had his story broken down, was quite different from how I would break it.

So I asked him, if I could give a shot at breaking down the stories differently. We spent some time and came up with user stories that kind of make sense to me. That was followed by a short TDD sessions. We got rid of the awkward class dependency issue, but ran into another issue. Well that’s not the point. What’s important here is, we realized that breaking down the user story differently lead to a very different design. What does this mean? We concluded saying, the way we create our user stories can impact the software design. So by breaking the user stories differently one can lead to a different design. Hence while we are creating/breaking down user stories, we are making design decisions.

We can take this one step further. I’ve had experience on real projects where depending on the order in which we developed our stories, the design evolved differently. Jeff Patton likes to say that when the stakeholders take a decision of building software to solve a business problem, they have made a design decision.

Conclusion: Design begins way before we think it actually does. We know Prototyping and TDD is a great way to validate those design decision. Don’t forget the best feedback is got when actual user use your software. Hence try to realse as frequently as possible. May be on a daily basis.

Amongst the gazillion myths about Agile, one of them is “Agile means no design, its a cowboy coders’ feast”. Give me a break, you think its cool to pretend that code just flows out of your fingers without any design/thinking? Why is thinking about design before coding production code looked down upon? I completely understand the pitfalls of big upfront design and I’m not proposing that. Whether you use CRC cards, white board, acceptance tests, mental models or whatever have you, you are designing. You might not think about it as traditional design. But think about it, you are actually designing. And there is nothing wrong with it. You are not a poor developer if you do that. In fact as I know, some of the most respected programmers in the industry do some form of design before they code production code.

In fact I claim that when you are breaking down your user stories you are designing your software. I actually have an example of how breaking down your stories differently can lead to a very different design. In this blog post, I explain why.

One of the things that I have found very useful when building software is to just prototype 2 or 3 different approaches, to explore the problem domain, before actually TDDing the whole solution. Recently I added version control support to FitNesse.  The approach I used was I tried first logical solution that came to my mind. It worked, but as I started using it, I need more stuff and the design I had would not really fit well the new requirement I had. I could have incrementally refactored the design to suite the new requirements, but since I treated the code as prototype, I threw it away and started from scratch. It seemed a lot less effort to throw it away and rebuild it from ground zero. Again this time, I just treated the code as prototype. I was really exploring a new idea and I wanted to test the design by writing some quick and dirty code. Usually these sessions might last a couple of days. These are clearly not months worth of effort.

It turned out that my second design worked for single user, but would not work for multiple users. It had a fundamental design problem that I could not have thought before hand. (May be some other smart people could have found it before we started, but I could not. May be having a pair would have helped, but this was open source and I did not have a pair).  Again this was a quick and dirty way to get a high level design feedback.

This style really complements the whole TDD philosophy. Its not that you do a quick and dirty prototype and you don’t need to do TDD any more. Prototypes really help you explore a broader aspect of your system design than TDD can do. TDD helps get feedback about your design and also helps build a safety net, but there are multiple design options and TDD helps you achieve one of those designs. How do you know if there is some other class or library that can actually work, sometimes TDD can help you with that. But if you are completely unaware of its existence, you would need an exploratory session to figure out your options. Once you know what options you have, TDD can really help you solidify it. Hope you get my point.

Sometimes, I write a high-level automated test and build various prototype. The automated test really cuts the manual check at the end.

Prototype is similar to the spiking practice that XP suggests. Somehow spiking has really fallen of the face of the earth. But XP really recommends you spike out various approaches to solve a problem, pick the best and then TDD the solution.

Prototyping is not a new practice its been around for ages. In fact its been around not just in software but also in other disciplines (craft related) like painting. A lot of times, painters create clay models of their subject to understand the depth or other dimensions about the subject. This really helps them give the real feel to their painting.

Now that your are convinced about the importance of prototyping, the next question is should you always prototype? Of course not. If you are not sure about the design, if you want to explore the problem domain, or if you think there is high technical risk with what you are building, sure. Else its just ritual (waste of time).

So when does prototyping stop? At some point you might feel confident enough that you understand the problem domain enough and you can switch to TDDing your production code. But if the requirements changes and something fundamental on which your design decision was based changes, you might be able to just incrementally refactor your design to suit the new requirement. Sometimes, you hit the point where trying a prototype and throwing away your existing design might be the best option. You are the best judge.

In my last post, Open Space (OST) and Open Source(OSS), I’ve highlighted some core fundamentals (values) of OST and OSS. After reading it, you might be thinking, boy, I really want my software project to be built on these fundamentals. Agile Software Development is really a step in this direction. A lot of people including myself have talked/presented about OSS and Agile. There are lots of similarities between them.

When Deb and I were organizing the first Agile Coach Camp conference, we invited John Engle to facilitate the open space. During our discussion about theme and target audience, we were explaining John how Agile works and he was surprised how fundamentally (at the values level), both Agile and OST were similar.

At the Agile 2008 Conference in Toronto, Dennis Byrne and I facilitated a panel on Agile and Open Source. It was a slight twist on this topic. Our panel was proposed under the “Questioning Agile” stage. Our panel was titled “Successful open source projects with little or no Agile”. We were really questioning if Agile is the only way to build software? Agile Software methods like Scrum and XP are getting more and more bloated. Checklists and templates have emerged. God knows what all crazy stuff folks have come up with in the name of Agile. But one thing that no one talks about really, came out of the panel. While there is some amount of volunteerism built into Agile, it has a long way to go compared to OSS or OST. Agile projects can be structured around Volunteerism and my belief is it’s really going to help.

In some organizations I’ve worked, we have had good success with corporate sourcing model. Once various project teams get into the rhythm and start building their products, the next issue they hit is to collaborate with other teams across the organization. I call this, the teams are being good tribesman. But now they need to be good citizens also. In corporate sourcing model, the core team acts as committers on the project, while anyone in the company can suggest features or contribute code patches. Volunteerism comes in because people have freedom to work on other projects that really interests them. In companies like Google, its a little more structures. They have the 80-20 rule. Where in your 20% time you can work on stuff you like. Another way to encourage volunteerism.

A lot of companies have a big problem, because they have some existing projects and they have some new projects. Most developers want to work on new, green field projects. Rarely anyone wants to maintain existing projects. But those projects are very critical to the company and the company cannot move all developers on new projects. What do you do in this situation? Morale of tea members maintaining existing projects is really low. But they do have some spare time and tons of idea which can really help new product development. The way I have seen some companies solve this problem is, they either set up a rotation program where team members from maintenance projects can choose which project they want to work for on a rotation basis. In some other companies, I have see, team members are allowed to contribute code patches or suggest new features to the new projects in their spare time.

There are various ways to encourage volunteerism in your organization. Not just at project level, but in other ares as well. If someone is interested in marketing or sales or training or recruitment, companies can use their valuable contributions. In most cases this creates a win-win situation. This also helps folks to shape their career and become all rounded individuals. More the number of roles a person can play, that much more valuable they become to the organization. So remember, volunteerism in very important not just in your project but also at your organization level.

Both, Open Space Technology (OST)* and Open Source Software (OSS) are structured around volunteerism (.i.e people are the center of the universe). They really encourage people to step up and help. Volunteerism applies at many levels.
In OSS,

• you can author a piece of software and put it out to help others OR
• you can join an existing project and contribute to it.
• you can use the software and give feedback to the authors. Filling in a bug report or requesting an enhancement is a great way to participate in this movement. There are million ways to contribute to OSS, you can write some user document for the project or give a demo to your local user group.

Similarly in OST, you can propose a topic and be a facilitator or participate in a topic proposed by someone else or blog about your thoughts.

With Volunteerism comes responsibilities and rights. You have the right to participate and contribute. If you don’t like the direction in which its going, you have the right to leave or spawn off something that is how you would like it. This very behavior leads to exploration and discovery. Hence leads to innovation. While the participants have all these rights, they are generally good citizens and are very responsible & self-disciplined. You can really see a touch of craftsmanship in their work.

In my experience both these approaches are amazing at how they can involve/engage people. In OST we say,

“Who ever comes it the right people and whatever happens is the best that can happen”.

This holds true for OSS projects as well. You can’t really force somebody to contribute. Based on my experience the core for both is passionate people.

From the outside, both OST and OSS looks very chaotic in nature. One has to act, sense and respond. When someone throws open a piece of software, they have no idea how it would evolve or even if it will interest anyone. They have to act first by putting it out there. After that based on the feedback from the community (sense), the author/s have to respond. Project which are able to do this, see a big user community around it and hence innovation. Same holds good for OST topics.

Even though we try to put some (very limited) structure around them, both OST and OSS by nature are very self-organizing, self-emerging and self-adjusting or self-correcting. They also have a self-filtering nature. Its easy to leave non-passionate and undisciplined people behind. Nor are these hero-centric systems. Also the moment we try to put too much structure or process around them, they tend to break down. They don’t necessarily fail, but evolve into something very different in nature and the kind of people around it. In OST we say,

“It starts when it starts, it ends when it ends”.

Topics in OST and features in OSS evolve over a period of time. One thing influences another and another influences some third thing. This very nature makes it very adaptive. Adaptive systems are here to stay, rigid systems will soon be extinct.

* Don’t get confused with the word Technology in OST. In this context Technology means tool. Its really a meeting/conference format.