Test-Driven Development
Test-Driven Development
Jamie Cansdale is continuing his work on version 2.0 of the must-have TestDriven.NET Visual Studio add-in. Jamie mentions that TestDriven.NET 2.0 beta is available via (the now Microsoft-owned) FolderShare. Here's Jamie's list of v2.0 improvements:
- Built and tested using the RTM bits of .NET 2.0 and Visual Studio 2005
- Support for Visual Studio 2005 Team System unit testing and debugging
- Support for Visual Studio 2005 Express Editions
- Compatible with 64bit versions of Windows
- Backward compatible with Visual Studio .NET 2002/2003
- Shiny new extensibility model
In NUnit news, Charlie Poole points out that next iteration release (2.2.3) is now available, with some interesting improvements. Highlights from the release notes:
-
NUnit can now be installed and run under .NET 2.0 and used in conjunction with VS2005. A problem that prevented installation when .NET 2.0 Beta 2 was the only installed framework has been corrected. NUnit 2.2.3 installation requires one of the following .NET framework versions to be present: 2.0.50727, 2.0.50215, 2.0.40607, 1.1.4322 or 1.0.3705.
-
A number of new Assert Methods have been added:
- The StringAssert class has been moved from the extensions assembly into nunit.framework. It supports the following methods:
- StringAssert.Contains(string,string) - Tests whether a string contains an expected string.
- StringAssert.StartsWith(string,string) - Tests whether a string begins with an expected string.
- StringAssert.EndsWith(string,string) - Tests whether a string ends with an expected string.
- StringAssert.AreEqualIgnoringCase(string,string) - Compares two strings for equality without regard to case and gives an error message showing the actual strings, without case adjustment.
And finally, if you happen to read this within the next few hours, Noah Coad posts about the next Team System Chat (for the Developer and Tester Editions) today (11/16) at 1:00 PM EST.
-Chris
As we get closer to the release of Visual Studio Team System, I'd like to begin sharing some of its less-publicized features. Of course, these aren't really "hidden", but they can be easy to miss if you're new to using the product.
In this post, I'll introduce two of the specialized classes available for unit testing in Visual Studio Team System, CollectionAssert and StringAssert.
A major feature that Team System brings to the developer is the ability to create unit tests from within the development environment, without having to rely on external third-party tools. Unit testing involves creating code which verifies typically low-level behavior in other code. If you're not already familiar with VSTS unit testing, there is a good introductory article on MSDN.
At the center of unit testing in VSTS is the Microsoft.VisualStudio.TestTools.UnitTesting namespace. This can be found in the "Microsoft.VisualStudio.QualityTools.UnitTestFramework.dll" assembly. You'll quickly become familiar with the Assert class and the many features it offers for developing unit tests. Those familiar with NUnit will feel right at home. However, VSTS offers some additional features to help abstract several other common unit testing scenarios.
Using CollectionAssert
The CollectionAssert class, also found in the Microsoft.VisualStudio.TestTools.UnitTesting namespace, features methods which will help you write unit tests that verify behavior of collection types. For example, you might be using collections and wish to verify their contents or perhaps you've written a custom collection class and need to verify it works as expected.
The following methods are exposed by CollectionAssert:
- AllItemsAreInstancesOfType - Verifies all elements are a specific type (e.g. System.String)
- AllItemsAreNotNull - Verifies each element is not null
- AllItemsAreUnique - Ensures no elements are repeated
- AreEqual / AreNotEqual - Verifies two collections are (or are not) exactly equal, having the same number of elements and each element is reference-identical
- AreEquivalent / AreNotEquivalent - Similar to AreEqual, but each element needs only to be value-equivalent, not necessarily referencing the same instance
- Contains / DoesNotContain - Searches a collection for a specific object
- IsSubsetOf / IsNotSubsetOf - Ensures all members of a collection are (or are not) also in another collection
Here's a very simplistic example of using some of these methods, using a fictitious collection class as a target. This evening as I write this I'm in a "beer" mood, so you'll have to pardon the example...
[TestMethod]
public void CollectBeer()
{
BeerCollection beers = new BeerCollection();
beers.Add("Porter");
beers.Add("Barleywine");
beers.Add("Wee Heavy");
beers.Add("Stout");
// beers.Add(12345); // Breaks AllItemsAreInstancesOfType below
// Was everything not null and a string type?
CollectionAssert.AllItemsAreNotNull(beers);
CollectionAssert.AllItemsAreInstancesOfType(
beers, typeof(string));
// Labmics need not apply
CollectionAssert.DoesNotContain(beers, "Lambic");
BeerCollection strings2 = new BeerCollection();
beers2.Add("Porter");
beers2.Add("Stout");
// Verify the collections do have the same members
CollectionAssert.AreNotEquivalent(beers, beers2);
// All members of the second collection
// can be found in the first
CollectionAssert.IsSubsetOf(beers2, beers);
}
Using StringAssert
Another class within the TestTools.UnitTesting namespace, StringAssert, contains methods to make common text-based assertions.
StringAssert exposes the following methods:
- Contains - Searches a string for a substring and fails if not found
- StartsWith / EndsWith - Fails if the string does not begin (or end) with another string
- Matches / DoesNotMatch - Matches a string with a regular expression, failing if no matches (or any matches) are found
These are generally straightforward to use, but Matches and DoesNotMatch do require use of a System.Text.RegularExpressions.Regex instance. Continuing the beer theme in another example...
[TestMethod]
public void StringMeAlong()
{
// Various searching options...
StringAssert.Contains(
"Have a favorite beer? What is it?", "beer");
StringAssert.EndsWith(
"Roomba? No, 'Beerba', a robot to fetch beer", "beer");
StringAssert.StartsWith(
"Beer is not a substitute for milk in cereal", "Beer");
// Ensure this sentence has whitespace
StringAssert.Matches("Dogfish Head make some fantastic brews",
new System.Text.RegularExpressions.Regex(@"\s+"));
// ... and that this one does not
StringAssert.DoesNotMatch("NoWhitespaceFoundHere",
new System.Text.RegularExpressions.Regex(@"\s+"));
}
Conclusion
Integrated unit testing is a major feature offered to developers with Team System and it is important to understand what tools are available for creating effective unit tests. The Assert class contains much of what you'll need, but you'll soon find common cases where you're applying the same logic in your tests. Before VSTS, developers had to create libraries of functions to accomplish these common unit testing tasks. Now, with Team System, Microsoft has integrated some of these common scenarios, allowing you to simply use CollectionAssert and StringAssert, relying on the Framework itself instead of custom code.
Now you can spend your newly-found time... sampling beer? Enjoy!
-Chris
James Newkirk has posted notice that an early build of his tool to convert your NUnit-based unit tests to Team System unit tests is now available on GotDotNet. (Note that this is currently only for C#, but that will change.)
I had the chance to speak with him at TechEd about the challenges involved and the technology he used to write the tool - interesting stuff! He is planning to release the source code, so I'm looking forward to seeing more of the implementation.
It's also good to see that he used the Guidance Automation Toolkit (GAT), another interesting product from the Pattterns & Practices team, to plug in to Visual Studio 2005.
-Chris
Let's hope we don't get too much snow for this weekend's (sold-out**) Code Camp III!
Thom has announced that Microsoft will conduct hands-on labs at the Microsoft Technology Center (MTC) on Saturday from 1-5. Seating is limited and you can sign up for these when you arrive at Code Camp on Saturday morning. The MTC is about a mile drive from the Jones Road offices.
He also sends word that the final agenda is ready (and probably won't change.) The session descriptions are online as well.
I wanted to have more time to see other peoples' sessions, so I've cut back from doing four at the last code camp to just two:
- “Moving to Visual Studio 2005 Team System Unit Testing” - The session will concentrate on introducing the unit testing features of Team System, pointing out how they compare and contrast with solutions available today as well as issues to plan for during migration. Experience with unit testing is not required, but is of course helpful.
- “Effective Development Practices in .NET” - This peer discussion (“chalk talk“) went so well at Code Camp II that I wanted to conduct it again. The exact topics are open, but I have a list of potential topics to get things started. Conversation revolves around the practices and tools with which people have had success (or sometimes failure!)
Looking forward to seeing you there!
-Chris
** Although the Camp is “sold-out”, if you're interested, I recommend you sign up for the waitlist and show up anyway. Chances are excellent that the snow will keep a larger percentage of registrants away (at least on Saturday.)
Ask anyone who has heard me speak on Test-Driven or Agile development, and you'll hear that I'm a very big fan of the TestDriven.NET add-in for Visual Studio. Jamie Cansdale has done a great job making a great tool that Just Works. May you never Alt-Tab away from Visual Studio to run or view your unit tests again.
After numerous releases of 0.* versions, Jamie has cut the first 1.* version. You can download it from TestDriven.net.
Jamie has added a slightly lengthy survey which you must complete before downloading, but the add-in has saved me so much hassle over time that I had no problem supplying him with the details he's interested in.
It is best described on the website itself: “TestDriven.NET is a free, zero friction unit testing add-in for Microsoft Visual Studio .NET. The current release of TestDriven.NET supports multiple unit testing frameworks including NUnit, MbUnit, csUnit and MS Team System and is fully compatible with all versions of the .NET Framework.” If you haven't seen it in action, take a look at some screenshots and read the quickstart guide.
The fact that TestDriven.NET works with Visual Studio Team System is most welcome. Although unit testing is now fully integrated with Team System (Developer and Tester editions), having the “Run Tests” command available from additional contexts like the code editor is a great time saver.
If you create unit tests for your .NET systems (and of course you are, right?) you should download it now.
-Chris
P.S. No, I'm not on Jamie's payroll, but for the number of referrals I've given, I should get a percentage of the royalties. What's that you say? It's FREE? ;)
I'm a bit late to the dance with my after-Camp post, but I'd just like to say I had a wonderful time at Code Camp. Everyone was truly motivated to learn and it was great to compare experiences working with .NET.
Saturday was a long, but fun day, content from 9 AM to 9PM followed by a gathering for drinks over at the Westin. I delivered a session in the morning on Agile Development in .NET. On Sunday, I gave three sessions, ASP.NET 2.0, Team System and a “chalk talk“ on effective development practices in .NET. Of course, I completely ran out of time in the Whidbey sessions - there is simply too much to convey in those miniscule 90-minute slots, but I hope I conveyed the core ideas and infected others with excitement around what's coming.
I was impressed by the level of participation and interest from the audiences. I suppose I shouldn't be surprised at all, since it was a weekend, people were clearly not there to escape the office but were there in the sincere interests of learning and community.
It was great spending time with the other speakers, some of whom I only knew from their blogs. Here are some of the summaries from the others: Jason Bock, Sam Gentile, Robert Hurlbut, Carl Franklin, Chris Pels and DonXML Demsak.
Hats off to Thom Robbins for doing a fantastic job coordinating the event and for making everybody very nervous by walking around with a video camera... Thom is working on adding content to MSDNCodeCamp.com, where you'll be able to find slides and code from the camp as well as information on the next Camp (tentatively scheduled for February.)
CORRECTION: If you attended my Team System session, I'd like to correct one misstatement. Somehow in the fog of late night editing before Camp, I put the right idea on the wrong slide. I said that SourceSafe in Whidbey shares no bits with the previous version, but I meant to say that Hatteras (the source control management system in Team System) shares no bits with SourceSafe, being a completely new product. SourceSafe is certainly enhanced in Whidbey, but is not rewritten. I knew something was fishy when I was saying it, but I pushed on, thinking “it must be right, it's on the slide!” Sorry for the confusion.
I hope everyone enjoyed Code Camp and that we'll see even more of you for Code Camp III!
-Chris
This Wednesday night (July 14th), I delivered a session to the Boston .NET User Group entitled “Unit Testing and Test-Driven Development in .NET”. We're working on getting the slides posted to the site, but in the meantime, since a number of people have requested the links that were in the slides, I've added them below.
Thanks to everyone who attended. I hope you're going to give unit testing and TDD a try in your projects!
Please feel free to contact me (use the “Contact” link at top-left, or leave blog feedback below) if you have any questions.
-Chris
NUnit
Addins for NUnit
Unit Testing and TDD Resources
Mock Objects
Visual Studio 2005 Team System
Other Tools
Books
UPDATE: Corrected “Quick Start” document link
Benjamin Mitchell has a must-read report on the Unit Testing “Birds of a Feather” session at TechEd Europe which featured James Newkirk.
http://benjaminm.net/PermaLink.aspx?guid=fdd292d0-a324-4178-b46e-f02a114bde74
Great job, Benjamin!
-Chris
Visual Studio Team System has many great features baked right in. Here's what might be a typical experience:
- Fire up the Application Connector Designer. Plot systems... Generate projects - CHECK!
- Bring up a Class Diagram. Model classes and interfaces... Generate files - CHECK!
- Add some code here and there... CHECK!
- Create my Test Project... CHECK!
- Right-click on some code, choose "Create Tests"... Add unit tests - CHECK!
- Start working on my tests...
Hmmm... Wait a minute. Something is not quite right here.
One of the ideals of Test-Driven Development (TDD) is that you should not write any implementation code without first having a failing test. Also, a tactic of TDD is to address one test at a time from a list of continuously-evolving candidate tests. These ideals are described in the excellent books "Test-Driven Development in Microsoft .NET" by Jim Newkirk and "Test Driven Development: By Example" by Kent Beck.
With these time-saving VSTS tools at my fingertips, I'm enticed to take the automatic route and generate as much as possible - projects, classes, unit tests, etc. However, when I use the Class Diagram tool and then generate my tests from the resulting classes, I'm creating code without first having a failing test. If I opt to follow TDD by manually creating a (failing) unit test first and then emploing the Class Diagram tool, I have to sacrifice the productivity other developers enjoy with the "Create Tests" option. (Almost by definition whenever I run "Create Tests", I've probably written code without a failing test.)
I suppose one argument could be that by using the Class Designer et al., you are not actually writing "implementation", just the scaffolding upon which you will add the real mechanics. You create your unit tests from the shell classes with empty members, then start addressing all of the Assert.Inconclusive() results.
That argument only goes so far - the concern remains that the "Create Tests" feature generally runs contrary to the approach where you maintain a list of candidate tests based on functionality. Instead, you see the "Unit Test Generation" dialog, choose your classes and members, and effortlessly now have a suite of tests. Unfortunately, these are based upon class members and not on specific slices of functionality to be tested.
This boils down to a basic concern that the "easy" or even "normal" way of developing systems with VSTS can run counter to some of the tenets of TDD.
Let's be clear, nobody is forced to use these tools. However, my job as architect and standards-driver could become harder if I need to tell developers not to leverage available automation for sake of TDD purity, especially given looming deadlines.
Are there any features still to be added to VSTS which could help with this? Maybe a feature to help people track candidate test lists? (A new view into the Task List?) It might include a wizard/designer such that I can take a test from the list, select a template, or target method(s) and have a shell test generated (and the test crossed off the list.) Perhaps I could right-click on a method, a block of code or even the state of an object in the debugger and say "Add to Candidate Tests" so I could review it later?
Are my concerns unwarranted? Does anyone have experiences implementing the “create a test first, then implement” approach in a large-scale .NET project? If so, how do you feel VSTS will affect your approach?
-Chris