As a consultant I get to work with a wide variety of clients and projects. I mainly do two different types of projects; end-to-end total enterprises where my team develops an entire system for a client and technical consulting on projects carried out by the client or other service providers. This gives me good insight into how different businesses carry out their projects. Most of these are web applications, and in some of these the three w’s should be an acronym for wild wild west, things have gotten way out of hand!
When a project is kicked off people are usually ambitious to use the correct methodologies. The usual suspects are The Rational Unified Process (RUP), eXtreme Programming (XP), applying design patterns and other en-vogue methodologies.
The motivation for jumping a methodology bandwagon is often rooted in previous projects having huge budget overruns, never being completed or simply because someone on the team or in management favor a methodology.
It’s the same ol’ song
Projects turning out too expensive or making the market too late is not news. This is a chronic illness of our industry. There are about as many books and papers on this subject as the average project overrun is large, so I will not elaborate further on this topic.
I find interest in the latter motives. A rule of thumb is to “do what you know and do it well”. This is a rule and not a law for a reason. You need to experiment and try out new things to have progress. Doing XP on a smaller, low-risk project is safe. Stating that “for this project we’re going to use RUP” with no prior experience with the framework is ludicrous. Many regard the Rational Unified Process as simply doing some use-case diagrams and a couple of reverse engineered class diagrams. These may make nice wall decorations if you’re office is particularly ugly, but will not help you make the deadline or cut your expenses.
Project methodologies make up a toolbox where you carefully chose the tools to use. After all you wouldn’t hire a carpenter who intended to rebuild your house just using just a saw. You probably wouldn’t feel safe if he kept on bragging about his amazing new monster tool which he was using for the first time either. Experience is a must.
Some variety of the easily grasped RUP artifacts such as envisioning the project and defining a domain model to capture the business context are part of most projects. These are non-technical artifacts and all companies should be able to define what they do and where they want to go pretty accurately. Most companies also manage to layout business process workflows to describe the business interactions of the above artifacts. These may not be done exactly by the book, but mostly they’re pretty close.
Unfortunately this is often all the quality artifacts you see. Usually someone has a go at defining some detailed requirements and writing an architecture document, but the urge to get down to the nitty-gritty and start hacking out code takes over and the requirements and architecture documents end up as a good intention gone bad.
I do not believe that managers should micro-manage projects, but often projects could benefit from being pressured to produce proper architecture documents, UI specifications et cetera. Sadly many managers suffer from “the Dilbert management syndrome” and believe that everything they don’t understand is unimportant. This often applies to detailed architectures.
Staring to write code without a proper plan is like walking onto a heavily trafficked road without looking left and right; you need more luck than a leprechaun to survive.
In “The Pragmatic Programmer”, Andrew Hunt and David Thomas describe a bad practice named “programming by coincidence”. This is where you’ll end up without a proper plan of what to do. It is not important whether this plan is sketched out on a napkin or is the tapestry in your newly redecorated office as long as you got a plan.
Ignorance.Equals(bliss)==false
Hacking out code without some form of detailed design will most likely cause problems later. Many people start out doing the business logic for their applications. This code sits in the very center of the application and will be used by tons of other components. Therefore a change to the interface of a business component will ripple through entire application. This causes developers to be very careful about touching these components. On the other hand, an erroneous low-level component will pollute all client code. Therefore being ignorant to the design of such components gives a lose-lose situation.
A while ago I was hired to help out on a project where none of the collections in the business layer where “real” .NET collections. They could not be accessed via indexers; you had to use get methods. The collections were one indexed; not zero indexed. Neither did they support enumeration. The code for these collections had been written to satisfy a business requirement which stated that the users relate to one or more instances of different entities without thinking about how these collections should be used.
Since these collections did not implement the System.Collections interfaces they could not be used as data sources for bindable controls nor could you iterate through the collections using “for-each” constructs.
To workaround these issues all client code was adapted to the awkward collections. Data grids where populated by hand, methods with property names where called and “for”-loops counted from one and up.
When the problems with the design of the business layer surfaced the project team chose to ignore them instead of going back to the drawing board to redesign the collections to conform to the .NET design guidelines.
This is a classic case of software entropy. Just like a change to a core component would ripple through an entire application, rot in such a component will also spread to the as a whole making the code error prone and hard to maintain.
The cost of living with rotten designs is much greater than dealing with the bad apples straight away.
Don’t over do it
Programming is fun. Programmers often focus on doing things right rather than doing the right thing. Mindlessly focusing on the detailed design of an application is just as bad as being ignorant about it. Chossing the right O/R mapper will not help you produce the right thing. After all, a state of the art technical design has little value without sound requirements.
As Edward Yourdon pointed out in his article “When Good Enough Software is Best” you can discipline yourself to write software that serves it’s purpose.
This does not imply cutting corners or producing low quality code. You just have to be able to see when you’re done.
It is important to stress that software only is good enough when it so. The users must be involved in the trade offs that are made. In a project I reviewed, the users had complained about the application having a proprietary login system whereas most other applications leveraged Windows identity and access management to provide a single sign-on solution.
The request to use Windows integrated security had been turned down by the project team since their existing identity and access management system “worked fine” and exchanging this with integrated security would add complexity and that the cost of this was not justifiable.
The application had approximately 800 users each having to logon to the system an average of 7 times a day. The large number of subsequent logons was due to session timeouts. Each logon took about 30 seconds, including navigating to the form the user was working with.
Given the number of workdays per year each user logged on an average 1,575 times each year. With an average salary of NOK 350,000.00 (˜$51,100) per year each login cost about NOK 2. This gives a total cost of over NOK 2.5 million (˜$365,100) pr. year in lost productivity for the logons alone. At that price, you can develop an outstanding sign-on framework.
Of course calculations like these do not reflect reality, but they give some indication of the absurdity of not implementing the single sign-on functionality based on economical factors.
Save the trees
Luckily, in most cases people do get things right. Proper requirement documents are written, a domain model is laid out, business workflows mapped, a formal architecture defined and the application gets detailed designs for its various modules.
In their youth many projects are top-notch when it comes to following good methodologies. However, these documents tend to be printed and then laid on desks to gather dust. Don’t do this, save the trees by not printing the documents in the first place.
As requirements or other factors change these have to be reflected in all relevant documents. If this is not done, the value of the project artifacts is reduced.
When the business domain changes each of the subsequent layers must be revised to reflect the changes. If not a change to one module that appears to be trivial might unexpectedly break other functionality in other modules. If the initial design document has been neglected for a while the project might be experiencing a high degree of entropy making the functionality breach pass unnoticed into production.
Conclusion
It doesn’t matter whether you follow a methodology called RUP, XP or any other combination of letters. What is important is that it works for you and that you think about artifacts that are not written in code through out the entire project, not just at in the few weeks before and after the project kick-off. Don’t be afraid if your projects at times resemble the Wild West, a few cowboys and Indians won’t hurt your project, but keep in mind that too many will. You can play the sheriff, armed with formal methods to keep law and order on the project – just don’t turn the project into a police state.
Remember that methods are nothing more than tools in your toolbox. You are in charge of what to do, not the RUP poster on your office wall.