Faster Feedback for Code Analysis tools
One of the main limitations of most code analysis tools is that their feedback cycle is too slow. Generally I have my code analysis tools, such as FxCop, or NCover or Simian run in my build process -- either locally before I check in or on the integration server after I've checked in. The problem is that the feedback from these tools generally comes too late for me to want to do anything about it. If I run it before my check in then I don't really want to mess with the files that I'm about to commit as I've made enough changes already. And if I run it after the check in then I'm generally ready to move onto something else -- I don't really want to go and start working on the same files again.
In my opinion, the right time to get feedback from code analysis tools is when you're working on the files in the IDE. This is where ReSharper's little green box in the right hand corner of the screen is a thing of beauty. If my class contains compilation errors this box is red. If it contains unused using statements or unused code statement then the box is orange. This provides me with all the feedback I need to realise that the code is not yet done. ReSharper's analysis engine runs asynchronously in the background, analysing my code as I change it and providing feedback on the results. It would be lovely to be able to integrate new rules, validations or other analysis tools into this feature, but I have no idea how to do this or if it is even possible (knowing JetBrains, they've probably already thought of this).
So I started thinking about what simple things I could do right now to work towards this goal. External tools provide a simple way to hook applications into the Visual Studio IDE. Pairing with my colleague Alex, we created a simple External Tool reference to run Simian to detect duplicated code in the currently open file. This proved to be very easy; here is the configuration that we used:
- Command: \simian-2.2.2.exe
- Arguments: $(ItemFileName)$(ItemExt)
- Initial directory: $(ItemDir)
- Select "Use Output window"
Once you set up an External Tool, you can then attach a keyboard short cut to it. I've now got Simian running over the current file every time I press Crtl-Alt-1. Pretty sweet.
I also looked at doing the same thing with FxCop:
- Command: \FxCopCmd.exe
- Arguments: /console /f:$(TargetName)$(TargetExt) /t:$(ItemFileName)
- Initial directory: $(TargetDir)
- Select "Use Output window"
This works fairly well, but it is not as nice as the Simian integration. Simian works from source files, whereas FxCop works using binaries; this means that you need to recompile each time before you run the analysis. You can also use the "/p" argument to specify a FxCop project, so that you can customise the rules that you want to run; though this does require setting up the project and maintaining the rules through the FxCop client. FxCopCmd also produces a lot of header output that it would be great to strip out and ignore. Maybe I'll try creating a stylesheet so that I can limit it to just include the results.
The next step would be to build some kind of a Visual Studio add-in, but that's a much larger proposition. External tools provide a simple mechanism that I can leverage right now to integrate code analysis tools into the IDE so as to shorten my feedback cycle.