Why do Web 2.0 companies over look the importance of a solid suite of automated tests?

From an end user’s perspective it looks like they use their first thousand users as their manual testers.

I’ll give you an example, today LinkedIn launched a new set of Applications like SlideShare, Amazon, WordPress, TripIt, etc. When I try to use any application by installing it, I keep getting random errors.

There was a problem installing My Travel.
Fix this by reinstalling the application.

Sorry, unable to fetch your blog. Please try again later!

The server did not respond. Please try again.

Its the Web 2.0 companies or the Microsoft’s of the world who can get away with this attitude. If this was a high-end competitive market, such broken applications would result in significant loss of reputation and business.  The Web is certainly changing this. Not sure for the good or bad. On one hand, I like the fact that I can quickly release features and improve it over time. But on the other hand, I don’t like the fact that in the urgency to release new features, we compromise on quality and release dysfunctional stuff.

All I can think is, companies still struggle trying to strike the right balance. They are caugh up in tyring to have the cake and eating it too.

I’m a big Eclipse fan. Since 2002 I’ve been promoting Eclipse inside my company and at local conferences. In 2003, I gave a Technical Overview of Eclipse platform @ Linux Bangalore Conference (now known as FOSS.IN). My claim was Eclipse is not just an IDE, its a platform to build IDEs and other interesting applications. People laughed at me saying I was fantasizing. They claimed that Eclipse was nothing more than an IDE. Anyway, time has proved that Eclipse is really a platform that has contributed a great deal to the software world.

While I continue to be a proud Eclipse supported, over the last few days Eclipse has started disappointing me. Last whole week, Eclipse kept crashing on my machine if I upgraded some Eclipse platform plugins using their update-site. Today I stumbled upon a new issue. On my Mac, when I try to launch Eclipse.app, it complains “The Eclipse executable launcher was unable to locate its companion shared library”.

It turns out that in my Eclipse.app/Contents/MacOS/eclipse.ini file launcher.library points to a old plugin folder that does not exist. I had to manually go and update that folder to the correct folder name (version differences).

I had to change :
–launcher.library
../../../plugins/org.eclipse.equinox.launcher.carbon.macosx_1.0.100.v20080509-1800

to

–launcher.library
../../../plugins/org.eclipse.equinox.launcher.carbon.macosx_1.0.101.R34x_v20080731

Hope these issues are resolved as soon as possible, so that end-user experience is not compromised and my faith in Eclipse does not die.

Wow! Today you can download CrossOver for Mac and Linux. You can also download CrossOver Games for Mac and Linux for free. This offer is only valid on 28th Oct 2008. Currently their main site is down, but you can download these softwares from http://down.codeweavers.com/

Enjoy!

Today, I wanted to move a SVN repository from my windows laptop to my Mac. So I was trying to dump the SVN repository from Windows machine and load it on my Mac. I was accessing my SVN repository via an URL (svn://machine_ip) instead of a Path (file://machine_ip)

Unfortunately svnadmin dump seems to accept only file paths:

svnadmin dump svn://localhost/project_name > project_name.svn_dump
svn: ‘svn://localhost/project_name’ is an url when it should be a path

Luckily I remembered the location of the repository on the file system. So then I tried the following and it worked

svnadmin dump /users/naresh/local_svn_repos > project_name.svn_dump

Please note that the local_svn_repo folder must be the actual root of your svn repository. This folder usually contains the following folders:

• format (file)
• conf
• db
• hooks
• locks

Recently I was designing a web application which has a standard user registration module. You need a unique email address to successfully register a new account. Once the user fills in an unique email address along with other basic info, the use can go ahead and register the account. To prevent bots from trying to register bogus users, we had to introduce a captcha image, so that real human users can enter the captcha code for registration. This is the first instance where because of security reasons we have to introduce an extra step in the registration process and hence compromise on the usability. 

Since most website have captcha today, I can live with this compromise. But the story does not end here. To make the registration process smooth, I had provided AJAX calls on the email address field on the registration page, so that if the email address is already registered, then the user can be notified much before entering all the other info and hitting the submit button. Little did I know that this could lead to a form of Dictionary attack, where a bot can brute force all the registered email addresses on our website. Since this was an AJAX call, a bot can exploit this without having to enter any captcha codes. To avoid these security loop holes, I had to reorder fields on the registration page such that, email address was the first field. Immediately after email address, I had to put captcha image. Once the user enters the captcha code, we fire an AJAX call to check if the captcha code is correct. If yes, then we check if the email address is already registered or not. If it is, then we notify the user about the error and load a new captcha image.

Another example of how there is always a tension between making an application usable and making sure its secure. 

Some people still believe that you can put an indispensable process and set of tools that support it in place and then fill the void with dispensable people. It feels like the process and tools are in control and the people happen to be. 

Unfortunately, I’ve never seen this work. What works for me is to have a small set of smart, intelligent people who can be in control of things and decide what process and tools they want to use. If the process and tools are not evolving, every few weeks, then I feel the process and tools are back in control. 

I had heard about Nokia Test several times before and always discarded it without looking at the test. (Tells you, how judgmental I’m). Until, recently when I was shepherding an experience report by Jeff Sutherland and some folks from Xebia on Fully Distributed Scrum: The Secret Sauce for Hyperproductive Outsourced Development Teams .

My immediate reaction (then) after reading the Nokia test was, “Say Hi to 21st Century Software Process Police”. Unfortunately nothing has changed since. More details on “Am I post Agile?”

I’ve also seen many people call this as the “Nokia Test for Agile”. Please note that it is Nokia Test for Scrum and not for Agile. Scrum and Agile cannot be interchangeably used. IMHO, being Agile takes a lot more than doing Scrum. Being agile (as defined by the manifesto) is lot more important than doing Scrum or XP or any other process. My mind says, focus on outcomes or goals not means. (in some ways focusing on being agile itself is like focusing on means rather than outcomes).

I say enough of getting caught in proving whether you are “foo Certified” or “bar compliant”.

Focus on building products that make life easy and enjoyable for your users at the same time raising the bar for your competitors.

Now you can ask, well that’s easy to say, how to do it? Understand one thing, there is No Silver Bullet. If there existed any such process that would make all teams successfully irrespective of the people and market they are in, guess what? People are not dumb. Everyone would be all over it. Since there is no such thing called Silver Bullet, you’ll have to figure out what works for you. In my experience defining process tests not only encourages you to focus on wrong things, but also gets in the way of achieving what you really want to achieve.

Concerned about your team’s performance? Looking for ways to enhance your team’s productivity and getting rid of all those mundane tasks? Years of consulting and building products have thought me that Automation is the key to improving team’s productivity and bringing joy back to software development.

If your team is seriously looking at automating as much as possible on their projects, you must consider some items like:

• Setting up an automated build for your project. You must explore tools like Ant, NAnt, Maven, GAnt, Rake, etc
• Creating automated unit, acceptance and UI tests for your project. Have a look at xUnit frameworks, FitNesse, Selenium, Watir, Abbot, etc
• Setting up a Continuous Integration (CI) process for your team. Configuring a CI server with Version Control System for your project
• Setting up developer work-stations with all the necessary tools and framework to be able to independently build the project from developer desk in a predictable and repeatable way
• Setting up an evolutionary database design and data migration process using tools like Migrations, Liquibase, etc
• Enabling the automated build to switch between Development, CI, Staging and Live environment using simple properties
• Setting up coding standards for the team and using tools like CheckStyle and other Static analysis tools to automatically validate code format and other aspects with each build
• Adding code coverage or test coverage frameworks like Cobertura, Jester, etc to your build
• Setting up an integrated project dashboard on CI server with various project metrics like BurnUp/BurnDown chart, test coverage trends, test counts, etc. This Dashboard is visible to everyone on the team and all the stakeholders
• Setting up a build promotion process
• Setting up a team wiki which is versioned along with the whole project

The general rule of thumb I use when is comes to automation is:

If you need to do something more than once a day, it got to be automated. Period!

After burning my hands several times, I have concluded that

Throwing away code regularly will save a lot of time, effort, frustration, etc in the long run

Also

Throwing away code is an effective and efficient way to achieve Simple Design

Hence I propose, we add “Throwing away code” as a practice under the Simple Design, XP practice.

Today I spent a good 3 hours trouble shooting issues with code coverage on a legacy project. I’ve been using Cobertura for a good 3 years now. Hence I decided to use Cobertura to help a team working on a legacy project. What looked like a simple 10 mins job, ended up taking forever.

To start off, the team did not have an automated build. So I quickly put an Ant script together. Last 2 days I had helped them write some acceptance tests using FitNesse. I set up the build file to run the FitNesse tests as part of the build. Once I had the testing in place, I wanted to see what kind of code coverage we had. So we started adding Cobertura Ant tasks to the build. This was a fairly trivial task. But surprisingly cobertura-report kept displaying N/A for Line and Branch coverage. Clearly there was something wrong.

After spending a good amount of time, I realized that we need to compile the source code with debug=true option, else Cobertura does not generate any coverage numbers. What’s amazing is, it does not complain (in the normal mode) about this nor does cobertura’s documentation talks about this. When I ran the ant build in verbose mode, it showed the following warning

[cobertura-instrument] WARN visitEnd, No line number information found for class

That’s when it occurred to me that I need to compile my source code with debug=true option.

Once I solved this problem I hit the next roadblock. Even thought I fixed the debug=true option issue, cobertura-report was still displaying N/A for Line and Branch coverage. I quickly wrote a dummy unit tests and that seemed to work. Cobertura report started showing some numbers. But I was not able to generate any coverage numbers from the main Java application (server) that I was trying to test.

I knew there was some problem with the Shutdown hook not been executed correctly to flush the coverage numbers to the ser file. It turned out that if you have a server which is started as a daemon process in a forked JVM in the build, it needs to have a way to shutdown the server gracefully. So I ended up writing a shutdown command for the server which would basically do a System.exit(0). One need to explicitly call the shutdown script before calling the cobertura report target.

Finally after 3 hrs, there was some ray of hope in my life when I saw a good 86% code coverage. This meant I could now go and refactor the code till I dropped dead.