What tools do you use for Code Analysis of C/C++ projects?

This is a common questions a lot of teams have when we discuss Continuous Integration in C/C++.

I would recommend the following tools:

UPDATE: I strongly recommend looking at CppDepend (commercial), one stop solution for all kinds of metric. It has some very cool/useful features like Code Query Language, Customer Build Reporting, Comparing Builds, great visualization diagrams for dependency, treemaps, etc.

• Cyclomatic Complexity
  • Cyclo – Open Source
  • CCCC – Open Source
  • McCabe IQ – Commercial
  • CMetrics – Open Source
  • Cyclomatic Complexity Viewer – Visual Studio .NET Addins
• Code Coverage
  • GCov – Open Source
  • Bullseye Coverage – Commercial
• LOC Metric
  • Most of the tools listed above would give you this measure
  • LC2 – Open Source
  • CNCC – Open Source
• Copy Paste Detector – PMD – Open Source

Wikipedia page on Static Code Analysis Tools has a list of many more tools.

This post is moved to Industrial Logic’s Blogic.

For many years now, our Agile eLearning albums have been offered at a flat, USD price, regardless of where you lived in the world.

That is about to change!

Our Agile eLearning albums are now available to individuals around the world at new, lower, regionally adjusted prices.

What Does This Mean To You?

It means that we have introduced a new Individual pricing model to complement our Corporate pricing model. The new model, based on Purchasing Power Parity (PPP), makes our Agile eLearning more affordable for individual purchasers.Individuals will still see USD album prices — however, they will be adjusted to reflect the purchasing power of your country’s currency.

As of April, 2010, if you live in India , you can purchase an Individual License to our Code Smells album in Java for USD $126.00, rather than the Corporate price of USD $210.00.

Even if you live in a country where the PPP discount is modest, Individual Licenses will still receive a 15% discount:

As of April, 2010, if you live in the United State , you can purchase an Individual License to our Code Smells album in Java for USD $179.00, rather than the Corporate price of USD $210.00.

Why Are We Doing This?

We’re devoted to helping people throughout the world acquire Agile skills at a reasonable price. We’ve helped large corporations, like Google, HP and Standard Life, realize this goal and now we’d like to make the same training affordable for individuals as well.To learn more about Agile eLearning, please visit our Greatest Hits Shop.

I was just looking through the Google Analytics on my blog, I was surprised to see Firefox dominate the browser usage race. Also surprised to see how fast Google Chrome has penetrated the market. (Over the years, Firefox has got really fat and slow. Chrome’s launch timing was perfect. Just when many geeks could not take FF’s speed and instability anymore.)

Yesterday, my blog had 3,697 visits from 16 different browsers.

And if you see the break up by Operating Systems

This data matches that from w3schools’ Browser Stats, except that my blog is not very IE friendly hence less IE traffic. Also I hope Windows usage goes down…

For Freeset, I’ve always been in the quest of Simplest Thing that Could Possibly Work. In a previous post, I explained how we’ve embraced an ultra-light process (call it lean, if you like) to build their e-commerce site.

In that post, I’ve talked about our wish to create a Selenium test suite for regression testing. But it never got high enough on our priority list. (esp. coz we mostly have static content served from a CMS as of now).

While that is something I wanted to tackle, last night, when I was moving Industrial Logic and Industrial XP‘s site over to a new server hardware, I wanted some quick way to test if all the pages were correctly displayed after the move. This was important since we switched from Apache to Nginx. Nginx has slightly different way to handle secure pages, etc.

So I asked on Twitter, if anyone knew of a tool that could compare 2 deployments of the same website. Few people responding saying I could use curl/wget with diff recursively. That seemed like the simplest thing that could work for now. So this morning I wrote a script.

rm -Rf * && mkdir live && cd live && wget -rkp -l5 -q -np -nH http://freesetglobal.com && cd .. && mkdir dev && cd dev && wget -rkp -l5 -q -np -nH http://dev.freesetglobal.com && cd .. && for i in `grep -l dev.freesetglobal.com \`find ./dev -name '*'\`` ; do sed -e 's/dev.freesetglobal.com/freesetglobal.com/g' $i > $i.xx && mv $i.xx $i; done && diff -r -y --suppress-common-lines -w -I '^.*' dev live

I’m planning to use this script to do simple regression test of our Freeset site. We have a live and a dev environment. We make changes on dev and frequently sync it up with live. I’m thinking before we sync up, we can check if we’ve made the correct changes to the intended pages. If some other pages show up in this diff that we did not expect, it’s a good way to catch such issue before the sync.

Note: One could also use diff with -q option, if all they are interested to know is which pages changes. Also note that under Mac, the sed command’s -i (inline edit) option is broken. It simply does not work as explained. If you give sed -i -e …., it ends up creating backup files with -e extension. #fail.

If you have your web application running on one tomcat instance and want to add another tomcat instance (ideally on a different machine), following steps will guide you.

Step 1: Independently deploy your web application (WAR file) on each instance and make sure they can work independently.

Step 2: Stop tomcat

Step 3: Update the <Cluster> element under the <Engine> element in the Server.xml file (under the conf dir in tomcat installation dir) on both your servers with:

<Engine name="<meaningful_unique_name>" defaultHost="localhost">      
     <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
              channelSendOptions="8">
          <Manager className="org.apache.catalina.ha.session.DeltaManager"
                   expireSessionsOnShutdown="false"
                   notifyListenersOnReplication="true"/>
          <Channel className="org.apache.catalina.tribes.group.GroupChannel">
               <Membership className="org.apache.catalina.tribes.membership.McastService"
                           address="228.0.0.4"
                           port="45564"
                           frequency="500"
                           dropTime="3000"/>
               <Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
                         address="auto"
                         port="4000"
                         autoBind="100"
                         selectorTimeout="5000"
                         maxThreads="6"/>
               <Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
                   <Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
               </Sender>
               <Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
               <Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>
          </Channel>
          <Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
                 filter=".*\.gif;.*\.js;.*\.jpg;.*\.png;.*\.css;.*\.txt;"/>
          <ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/>
     </Cluster>
     ...
</Engine>

For more details on these parameters, check https://sec1.woopra.com/docs/cluster-howto.html

Step 4: Start tomcat and make sure it starts up correctly. You should be able to access http://locahost:8080. Most default tomcat installations come with an examples web app. Try access http://localhost:8080/examples/jsp/ You should see a list of JSP files.

Step 4.a: Also if you see catalina.out log file, you should see:

INFO: Initializing Coyote HTTP/1.1 on http-8080
Nov 9, 2009 9:29:43 AM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 762 ms
Nov 9, 2009 9:29:43 AM org.apache.catalina.core.StandardService start
INFO: Starting service <server_name>
Nov 9, 2009 9:29:43 AM org.apache.catalina.core.StandardEngine start
INFO: Starting Servlet Engine: Apache Tomcat/6.0.16
Nov 9, 2009 9:29:43 AM org.apache.catalina.ha.tcp.SimpleTcpCluster start
INFO: Cluster is about to start
Nov 9, 2009 9:29:43 AM org.apache.catalina.tribes.transport.ReceiverBase bind
INFO: Receiver Server Socket bound to:/<server_ip>:4000
Nov 9, 2009 9:29:43 AM org.apache.catalina.tribes.membership.McastServiceImpl setupSocket
INFO: Setting cluster mcast soTimeout to 500
Nov 9, 2009 9:29:43 AM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers
INFO: Sleeping for 1000 milliseconds to establish cluster membership, start level:4

Step 5: Stop tomcat.

Step 6: We’ll use the examples web app to test if our session replication is working as expected.

Step 6.a: Open the Web.xml file of the “examples” web app in your webapps. Mark this web app distributable, by adding a <distributable/> element at the end of the Web.xml file (just before the </web-app> element)

Step 6.b: Add the session JSP file. This JSP prints the contents of the session and also adds/increments a counter stored in the session.

Step 6.c: Start tomcat on both machines

Step 6.d: You should see the following log in catalina.out

Nov 9, 2009 9:29:44 AM org.apache.catalina.ha.tcp.SimpleTcpCluster memberAdded
INFO: Replication member added:org.apache.catalina.tribes.membership.MemberImpl[tcp://{-64, -88, 0, 101}:4000,{-64, -88, 0, 101},4000, alive=10035,id={68 106 92 39 -110 -8 73 124 -116 -122 -15 -3 11 117 56 105 }, payload={}, command={}, domain={}, ] 
 
Nov 9, 2009 9:29:49 AM org.apache.catalina.ha.session.DeltaManager start
INFO: Register manager /examples to cluster element Engine with name <server_name>
Nov 9, 2009 9:29:49 AM org.apache.catalina.ha.session.DeltaManager start
INFO: Starting clustering manager at /examples
Nov 9, 2009 9:29:49 AM org.apache.catalina.ha.session.DeltaManager getAllClusterSessions
WARNING: Manager [localhost#/examples], requesting session state from org.apache.catalina.tribes.membership.MemberImpl[tcp://{-64, -88, 0, 101}:4000,{-64, -88, 0, 101},4000, alive=15538,id={68 106 92 39 -110 -8 73 124 -116 -122 -15 -3 11 117 56 105 }, payload={}, command={}, domain={}, ]. This operation will timeout if no session state has been received within 60 seconds.
Nov 9, 2009 9:29:49 AM org.apache.catalina.ha.session.DeltaManager waitForSendAllSessions
INFO: Manager [localhost#/examples]; session state send at 11/9/09 9:29 AM received in 101 ms.
 
Nov 9, 2009 9:29:49 AM org.apache.catalina.core.ApplicationContext log
INFO: ContextListener: contextInitialized()
Nov 9, 2009 9:29:49 AM org.apache.catalina.core.ApplicationContext log
INFO: SessionListener: contextInitialized()
Nov 9, 2009 9:29:50 AM org.apache.coyote.http11.Http11Protocol start
INFO: Starting Coyote HTTP/1.1 on http-8080
Nov 9, 2009 9:29:50 AM org.apache.jk.common.ChannelSocket init
INFO: JK: ajp13 listening on /0.0.0.0:8009
Nov 9, 2009 9:29:50 AM org.apache.jk.server.JkMain start
INFO: Jk running ID=0 time=0/49  config=null
Nov 9, 2009 9:29:50 AM org.apache.catalina.startup.Catalina start
INFO: Server startup in 6331 ms

Step 6.e: Try to access http://localhost:8080/examples/jsp/session.jsp Try refreshing the page a few times, you should see the counter getting updated.

Step 6.f: You should see the same behavior when you try to access the other tomcat server. Open another tab in your browser and hit http://<other_server_ip>:8080/examples/jsp/session.jsp

Step 6.g: At this point we know the app works fine and the session is working correctly. Now we want to check if the tomcat cluster is replicating the session info. To check this, we want to pass the session from server 1 to session 2 and see if it increments the counter from where we left.

Step 6.h: Before accessing the page, make sure you copy the j_session_id from server 1 (displayed on the http://localhost:8080/examples/jsp/session.jsp). Also make sure to clear all cookies from server 2. (All browsers give you a facility to clear cookies from a specific host/ip).

Step 6.i: Now hit http://<server_2_ip>:8080/examples/jsp/session.jsp;jsessionid=<jsession_id_from_server1>

Step 6.j: If you see the counter incrementing from where ever you had left, congrats! You have session replication working.

Step 6.k: Also catalina.out log file should have:

Nov 9, 2009 9:42:03 AM org.apache.catalina.core.ApplicationContext log
INFO: SessionListener: sessionCreated('CDC57B8C5CFDFDDC2C8572E7D14C0D28')
Nov 9, 2009 9:42:03 AM org.apache.catalina.core.ApplicationContext log
INFO: SessionListener: attributeAdded('CDC57B8C5CFDFDDC2C8572E7D14C0D28', 'counter', '1')
Nov 9, 2009 9:42:05 AM org.apache.catalina.core.ApplicationContext log
INFO: SessionListener: attributeReplaced('CDC57B8C5CFDFDDC2C8572E7D14C0D28', 'counter', '2')

While this might like smooth, I ran into lot of issues when getting to this point. Following are some trap routes I ran into:

1) java.sql.SQLException: No suitable driver tomcat cluster
Make sure your DB Driver jar (in our case mysql-connector-java-x.x.xx-bin.jar) is in tomcat/lib folder

2) In catalina.org if you see the following exception:

Nov 7, 2009 3:48:53 PM org.apache.catalina.ha.session.DeltaManager requestCompleted
SEVERE: Unable to serialize delta request for sessionid [1F43C3926FF3CC231574EF248896DCA6]
java.io.NotSerializableException: com.company.product.Class
	at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1156)
	at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:326)
	at java.util.ArrayList.writeObject(ArrayList.java:570)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)

This means that you are storing com.company.product.Class object (or some other object that holds a reference to this Object) in your session. And you’ll need to make com.company.product.Class implement Serializable interface.

3) In your catalina.out log if you see

INFO: Register manager /<your_app_name> to cluster element Engine with name <tomcat_engine_name>
Nov 7, 2009 11:56:20 AM org.apache.catalina.ha.session.DeltaManager start
INFO: Starting clustering manager at /<your_app_name>
Nov 7, 2009 11:56:20 AM org.apache.catalina.ha.session.DeltaManager getAllClusterSessions
INFO: Manager [localhost#/<your_app_name>]: <strong>skipping state transfer. No members active in cluster group</strong>.

If both your tomcat instance are up and running, then check if your tomcat servers can communicate with each other using Multicast with the following commands:

$ ping -t 1 -c 2 228.0.0.4
PING 228.0.0.4 (228.0.0.4): 56 data bytes
64 bytes from <server_1_ip>: icmp_seq=0 ttl=64 time=0.076 ms
64 bytes from <server_2_ip>: icmp_seq=0 ttl=64 time=0.645 ms

— 228.0.0.4 ping statistics —
1 packets transmitted, 1 packets received, +1 duplicates, 0.0% packet loss

or

$ sudo tcpdump -ni en0 host 228.0.0.4
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on en0, link-type EN10MB (Ethernet), capture size 65535 bytes
22:11:50.016147 IP <server_1_ip>.45564 > 228.0.0.4.45564: UDP, length 69
22:11:50.033336 IP <server_2_ip>.45564 > 228.0.0.4.45564: UDP, length 69
22:11:50.516746 IP <server_1_ip>.45564 > 228.0.0.4.45564: UDP, length 69
22:11:50.533613 IP <server_2_ip>.45564 > 228.0.0.4.45564: UDP, length 69
…

If you don’t see the results as described above, you might want to read my blog on Enabling Multicast.

So, when it comes to using a tool for Story Wall (read it as Agile PM tools): My first reaction is, you don’t need them unless you have used physical story wall (task boards) enough that you understand their limitations and you want to evolve.

If you are a small, co-located team, it is just simpler to use a physical story wall backed with a wiki or a spreadsheet. As the team grows (smell something is wrong), you might want to break down the large team into smaller self-contained (cross-functional) teams and continue using the same practice at 2 levels.

• Team Level: Each team manages the user stories they are working on.
• Meta Level: At a meta level, we manage the features (epics) each team is working on.

This scales fairly well.

For distributed teams, I’ll use the following approach:

1. Start using Physical Story wall at each location backed with a wiki or a spread sheet. How the story wall is used really depends on how the work is distributed across teams.
   • If all the development and testing is taking placing in one location (offshore), then I would only use one story wall at that location.
   • If development takes places in both location, I would duplicate the story wall on both sides and during the “one-on-one standing meeting” (not a distributed stand-up meeting, I don’t think they work), each side updates their story wall with updates from others side. This ensures both sides are really collaborating.
   • And sometimes, teams can maintain their independent story walls and then sync up once a week.
   • One needs to figure out what works best for their situations.
2. Once the team understands and matures using this. Next step could be to do away with the physical story wall in each location and just use a Wiki or a Distributed SpreadSheet (something like GoogleDocs) to maintain their backlog and story wall.
3. If you belong to an organization where everything has to be “enter-price” class, then I might consider one of the following tools:
   1. Mingle from ThoughtWorks
   2. Silver Catalyst
   3. Jira and GreenHopper
   4. VersionOne or Rally
   5. And so on…

You might ask, what about tracking, planning, project management dashboard (fancy charts: burn-downs) and so on. Well, IMHO a lot of it is hype. You don’t really need all of that. You need some of them and its simple to generate them without having to use a heavy weight tool that adds more complexity than it takes out.

At one point, I really wanted to try out a tool for distributed teams. There was nothing good available at that point (2004), so I wrote one myself using RoR. The tool was fine, but it was very difficult to get the same feel as real story wall. So I dropped the tool and went back to physical story wall (this was a distributed team).

I’ve become a big fan of displaying metric using Treemaps. Julias Shaw‘s Panpoticode is a great tool to produce useful design metric in the treemap format for your Java project.

In the past, I’ve used these graphs to show Before and After snapshots of various projects after a small refactoring effort. In this blog I want to show you a healthy project’s codebase and highlight somethings that makes me feel comfortable about the codebase. (Actually there is not much to talk, a picture is worth a thousand words.)

Following is the code coverage report from a project:

Couple of quick observations:

• Majority of the code has coverage over 75% (Our goal is not to have every single class with 100% code coverage. Code Coverage does not talk about Quality of your tests.)
• There is a decent distribution of code across packages, classes and methods. (No large boxes standing out.)
• You don’t see large black patches (ones you see are classes that were mocked out for testing).

Lets look at the complexity graph:

• Except for a couple of methods, most of them have Cyclomatic Complexity under 5.
• You don’t see large red or black boxes which are clear indicators of complex code.

Panopticode combined with CheckStyle, FindBugs and JDepends can give you a lot more info to check the real pulse of your codebase.

On one of the projects we are using Cargo Maven Plugin to run an embedded Jetty server for our builds. Out of the blue, today, I started getting the following error when I was running my Selenium Tests after deploying the application.

1
2
3
4
5
6
7

WARN:  Nested in org.apache.jasper.JasperException: org.apache.jasper.JasperException: Unable to initialize TldLocationsCache: null:
org.apache.jasper.JasperException: Unable to initialize TldLocationsCache: null
at org.apache.jasper.compiler.TldLocationsCache.init (TldLocationsCache.java:253)
at org.apache.jasper.compiler.TldLocationsCache.getLocation (TldLocationsCache.java:224)
at org.apache.jasper.JspCompilationContext.getTldLocation (JspCompilationContext.java:526)
at org.apache.jasper.compiler.Parser.parseTaglibDirective (Parser.java:422)
...

No clue why this is happening. Surprising this is, this issue cannot be reproduced on a Windows box. Only on my Mac with JDK 1.6 and Maven 2.0, I’m getting this issue.

On goolging for this issue, I make across this bug report which kind of indicated that this might be an issue with the Cargo Maven Plugin. On upgrading the plugin to version 1.0, the issue was solved. 🙂

Need to find out what caused the problem in the first place.

Recently I was helping a team @ Directi working on a .NET project. They were using MbUnit for unit testing and acceptance testing. The team was using VS 2008 as their IDE. (Seriously its a joke to call VS as an IDE).

As I was pairing with one of the developers, I was watching him make code changes, hit F6 to build the project and then switch to MbUnit UI to run the tests. What a freaking waste of time! Ideally I would make a code change and hit a keyboard shortcut to execute the tests.

So I suggested that we use ReSharper plugin or at least use Test Driven .Net . But the big problem was that ReSharper does not support MbUnit, it only support NUnit. So we looked at what was different in MbUnit that was not in NUnit. The main thing that stood out was RowTest feature. But then we soon found NUnit extension for RowTest. Great!

So we went ahead and changed all our MbUnit tests to use NUnit. (It was a breeze). But then, when we tried to run the tests inside VS using resharper, it was detecting all our test fixtures but did not detect any tests in them. Only later, we realized that ReShaper does not support RowTest extension.

So we simply reverted to MbUnit and installed the MbUnit Resharper Plugin to run MbUnit tests from ReSharper.