Friday, January 07, 2011

Primitive Debugging

At the end of a release at work our product has to be tested mixed in with other products.  In development we tend to be focused on our individual product and we try to keep things simple for reproducibility sake.  Our customers focus on a problem and need multiple products to be used in complex ways.  This helps give us some comfort that things won't break terribly in the customer's environment.

These are no the most fun issues to be had.  It is difficult if at all possible to reproduce these issues outside of the test lab.  The test lab also exists on an isolated network.  These combine to reproduce some interesting challenges when debugging.

By dumb luck I ended up being the one to usually go down to look at these problems.  This time around a different developer got wrangled in.  This is an experienced person who previously only dealt in app software.  His favorite tools like remote desktop and Visual Studio debugging (remote or even local) are thrown out the window.
Normally I end up using WinDBG but some quick looking leads us to believe it is an issue with the startup of a service we have.  Debuggers aren't much use in these situations because you can't start the service from inside it and you can't always attach in time.  So even though my coworker dreads WinDBG, he can't even use this tool.  That leaves us with print messages from a debug build of our service.  I am the one to generally maintain the service in question and I've been sure to put in debug messages in all the most useful spots.  I also showed my fellow developer a few tricks with debugging through DebugView.

This just gave me pause to reflect on how useful it is to use (in my case exclusively) these "primitive" tools.  In regular day-to-day debugging I can use the right tool (user space debugger, kernel space debugger, prints).  In addition I have the experience to deal with these more extreme debugging situations.  This is another great benefit for tools like VIM which you can use in a GUI, local command prompt, and through SSH.