John Wood

Another Stored Procedures Argument

It's easy to think there are two types of programmers in this world - those who are for using stored procedures and those who are against. New and rehashed arguments come up every so often, and the most recent emergence here isn't any less thought provoking.

However I'd like to offer a different stance. I think both Eric and Jeremy are right, but for reasons they perhaps don't see.

Actually I'd like to offer a couple of different stances. And the first is more simple and pragmatic, and always comes to mind with such deep rooted, religious arguments.

You see, the measurement of an application's success is how well it has served its users and the company that pays the programmers. Period. The variables that go into this equation are plentiful - and range from the skill set and cost of the programmers, to the complexity of the application and potential scope for expansion.

Every situation is different. If your key developer prefers writing T-SQL to C#, then I think you can justify the amount of sprocs in your system. If you have someone with more C# experience and little to no T-SQL, then I think your best bet would be to write the app using simple dynamic SQL statements and do the logic in C#.

In a lot of cases it's purely preference and I don't think there's anything wrong with that - they can both get the job done. If the person allocating the company budget could see the arguments about whether to sproc or no sproc, they probably would have something to add and it wouldn't be an extra zero.

Now for my second perspective I'll get a little deeper.

If you look at Eric's original post you can pretty much replace 'stored procedure' with 'data tier' and the same points he makes hold true. Data tiers offer more security (when distinction between tiers is both physical as well as logical). They protect the UI and business tiers from change. Data tiers can be developed in C# - as they would if Jeremy were architect. Data tiers can also be developed in T-SQL and can live on the SQL Server as stored procedures. They are both data tiers and both offer similar advantages.

However I'd argue that C# and ADO just aren't very good at describing data query and manipulation. Perhaps Linq will make this better in the future (or perhaps not), but right now C# just isn't very data-oriented. You wouldn't describe Beethoven's Symphony No. 9 in algebra, you would describe it using music notation on a score. And for the same reason, using a general purpose imperative language to describe data manipulation and queries just isn't the best choice.  Set and array based languages are far more suitable, can more accurately and consisely articulate the intention and can thus be far more readable and easier to maintain.

Now to address Eric's second point about T-SQL not being portable. That's really not the biggest weakness. The biggest weakness is that T-SQL (the language) sucks. It's dated, syntactically retarded, stagnant and feature deficient - resulting in a spaghetti of select statements and temporary table constructions for an operation that could look far more concise.

Also his statement about dynamic where clauses is just puzzling. In my applications, at least, the user has a lot of flexibility in determining the filter that is applied to what they see, and this has a direct affect on the where clause that ends up being exected by the data tier. It's really not difficult at all, and effects nearly every query.

This is the reason I personally don't use stored procedures. I do always have a rich data tier, and this rich data tier is nearly always written in a declarative language. Effectively it's a domain specific language but the domain is that of data access.
.
The problem occurs, though, when you start putting business logic in your data tier. Any business logic written in T-SQL is in the wrong place. Delineating business logic and data access logic isn't always an easy thing (is any kind of domain-specific decision business logic?), and it can be excused in minor offenses, but often enough stored procedures are plagued with complex business logic that just shouldn't be there.

Putting business logic into stored procedures is inexcusable in all but the simplest of applications with no life expectancy. I'm almost certain that both Eric and Jeremy would never endorse such a crime, but much of the time people look at Pro vs. Anti Sproc arguments as justification for advocating such an abuse.

I honestly hope that T-SQL will one day get replaced with a new, more contemporary query language (perhaps XQuery? well perhaps not), or perhaps get the upgrade it desperately needs. But either way I think we can all agree that data tiers are a good thing regardless of what language they are developed using.

Why Programming is Hard

I'm certainly not the first to rant about how hard programming has become.

Programming is a surprisingly arbitrary art. The countless decisions you have to make in the process of writing code in your programming language are more often than not finger-in-the-air.

Lets look at some of these decisions:

1. Naming. Naming of classes, namespaces, assemblies, enumerators, variable names, parameter names etc.

2. Methods granularity. At what point do I decide to move a portion of procedure A into a separate procedure?

3. Interfaces and abstraction. Should this class be an interface, will it be implemented elsewhere? Or perhaps it should be a base class? Or perhaps it should be an abstract class?

4. Code layout. Do I put my methods in alphabetical order? Do I group interface implementation methods together? Do I just put methods anywhere because it's just not important?

5. Commenting. Does this line justify being commented, or is it clear already? Who's the audience anyway, and what level of expertise should I expect them to have?

6. Re-use. I have to write X. Do I modify and use the very similar Y code, or do I just write X again because modifying it will break other code or be more hassle than it's worth?

7. Testing. It's simple code, does it *really* need a unit test?

I'm sure there are a lot more decisions than that too. And then there's GUI code, in which the decisions just explode in number because of the seemingly infinite possibilities that are available in what we can present visually.

The problem is there are a vast number of choices for each of these decisions. Given the number of programmers and the number of problems to solve, you can imagine that nearly every choice has been tested by someone.

This is what makes software complex.

Imagine if when a town was built, the construction workers for each street could decide on the type of material to use, the shape, color, width, lighting etc. to use when building that street.  You would quite simply end up with a collage of streets that wouldn't properly connect with each other. Now consider how that would complicate trying to drive around the town and you'll get an idea of how decisions complicate software.

These decisions also take time and time is money. Decisions make software expensive. What's more, most of these decisions are really a bad use of time. They're time spent on problems that are mostly irrelevant to the problem domain at hand. They're small change. Every minute spent thinking about a name for a method is probably a wasted minute.

Programming languages are too general, that's a big part of the problem. They're flexible in order to meet the needs of the diversity in software that needs writing, and the diversity in opinions on many of these decisions.

But sometimes you just have to say enough is enough - the street will be X feet wide. Stop the debate. Coding standards help here, but we need more than just standards. People rarely obey standards unless they have a gun pointed to their head. We just need less flexible languages. IDEs that keep our methods in a specific order. That better understand the code to help us name the method. That know when a method can be re-used, or is getting too large, and should be split up. And when it doesn't understand something it should mandate a comment. GUI frameworks that don't give us the opportunity to decide on what color to make this button.

(side thought - WPF is really a step backwards there, in WPF you're also left with the decision on whether you should make your button dance in 3D).

The IDE understands too little about what we're writing and until it wises up a little, and starts making decisions for us, programming won't get any easier and software won't get any simpler.

End of rant.

Is Software as a Service a Threat to Microsoft?

You hear the term banded about quite a bit. Recently a Credit Suisse analyst suggested that Microsoft's revenue decline was a direct result of the pressure they're feeling from 'software as a service' against their foundation of desktop software.

But what does that mean exactly?

Regardless of formal definitions, the term Software as a Service (SAAS) appears to be used to refer to anything that is run over the web. So I can only imagine it has the following qualities:

1. SAAS is downloaded every time it's invoked. This means there is no need to install software locally, as there is in desktop software. It also means the software is always up to date. To use a software application you simply need the url. Contrast this to desktop software where you will need the url to download and install the software, and will then have to know how to launch it on subsequent uses. Desktop software also stays on your computer when you're no longer using it.

2. SAAS focuses on service more than user interface. It often utilizes a minimal subset of the desktop's capabilities, but no matter what the capabilities of the desktop the service will still be rendered. For example, JavaScript and Ajax allows for client-side functionality to be used if available. Yet another site that does not utilize Ajax is often available for browsers that don't have that capability.

3. SAAS is optimized for a client/server topology, where the majority of the processing occurs on the server and the client is used purely for display and interaction purposes. The benefit being an increase in reliability, because there are fewer compatibility issues and less network traffic. You can contrast this to the Application Service Provider model where typically fat-client applications are metamorphasized into a client/server architecture through terminal services or some other remote desktop technology. The result is usually a less than steller user experience.

4. SAAS applications are inherently safe. This is important in a world where the pool of available services is so broad that you really have no idea how trusted the source of the service actually is. Desktop software offers little in the way of security.

But lets say that you are a WinForms developer. You specialize in writing client, UI-rich applications. Perhaps you don't like developing in HTML because you think the platform is inferior and limits the user experience. Is there any life for you in this new 'software as a service' world?

We can certainly learn a lot from what has happened over the past 10 years. The Internet has redefined software. Instead of the emphasis being on the quality of a small set of applications, the emphasis is now on software accessibility. It's on a broad pool of diverse functionality and being able to quickly and easily access software from potentially millions of providers.

HTML has proven to be a good test bed for this new way of accessing and delivering software. But HTML is very limited. Perhaps the time has come to put the user experience back into the equation. Desktop applications provide a rich user interface experience. Nothing in the benefits of software as a service excludes the possibility of a rich user interface.

The answer that Microsoft have provided us is called ClickOnce.

ClickOnce technology (as found in .Net 2.0) provides a good bridge between the old-school desktop application and browser based web applications. Applications are installed on-demand from a url. They are updated automatically. They get removed from the user's machine if they're no longer used. They're safe.

By utilizing ClickOnce you really can reap the benefits of 'software as a service' while writing client, rich user interface applications - referred to as smart clients.

So, yes Software as a Service has been a threat to Microsoft's software foundation. But they're reacting by providing great solutions. While the competition are focusing on dumb-terminal platforms such as HTML, Microsoft are busy bringing the richness and maturity of their desktop technology foundation to the Internet and the new world order of 'software as a service'.

Hidden Complexity in OOP Code

Cyclomatic Complexity measures the number of linear paths through a program. It claims that the more paths there are, the more complex your code is. In essence the more paths there are, the higher the possibility of bugs.

The most common mechanism to compute the cyclomatic complexity is by counting branches in the code, given that a branch defines a point in your code where program flow will potentially bifurcate depending on a condition. (nb. CCMetrics uses this technique).

Cyclomatic Complexity was really developed for procedural languages. Some attempts have been made to apply CC to object orientation, but for the most part this involves averaging out the complexity of the methods in the class to provide a class complexity metric.

But I think there are many more ways to encode complexity into object oriented code. Where as a simple if statement can generate two different paths of execution, what if the code made use of an object instance that was injected into the procedure or class? Polymorphism allows us to see any object in terms of  a specific interface regardless of its implementation. Yet changes in the implementation can drastically impact the program flow.

We may assume that the object being called back will be audited as part of our complexity analysis, but there is no guarantee of that. By definition an interface can be implemented by *any* object. What's more we may never know the exact combinations of objects being passed in, and what their dependencies are and so on. It's quite clear that using polymorphism dramatically increases the possible states in your application, and this has a direct impact on the program's complexity.

It's easy to think that seemingly elegant OOP solutions can actually reduce complexity. I recently read of a design pattern (the State Pattern) that, among other things, promotes the use of polymorphism in order to reduce the number of branches in the code. However given the choice between a simple if statement versus calling into a potentially unknown object, the less complex solution may well be the simple if statement.

The bottom line is that OOP complexity auditing techniques need more study. Cyclomatic complexity is fairly useful when limited to the scope of a single method, but has less use when looking at the complexity of an entire class hierarchy.

I think we need more metrics for determining the complexity of object oriented applications, especially one that takes aspects such as polymorphism, coupling and cohesion into account. If you know of any good papers on the topic, please leave a comment.

CreateInstall - Free Setup Generator

I have to say I do like CreateInstall for creating quick setup programs.

If ClickOnce or the Visual Studio setup wizard doesn't work for you, then you should really give CreateInstall a go. They have a free version and it's pretty cool. It's very simple to use. The projects are saved in a semi-xml format so you can easily even create them yourself. The gui for creating the setup projects is super simple. Creating the setup program can be done by loading that file into the GUI or by using it from the command line. The setup programs have minimal overhead and come with an uninstall. All in all, a very simple solution to something that honestly should be a simple process.

CCMetrics Update for .Net 2

CCMetrics is a complexity analyzer I wrote a couple of years back that works on .Net binaries. It provides some information on the cyclomatic complexity, and some other statistics on code and complexity re-use. It's a command line tool and can also generate XML output that can be used as part of an overnight build.

I've recently posted an update to CCMetrics, to fix some problems people have been experiencing and also to enable it to work with .Net 2.0 binaries.

It's still beta quality software. If you want to try it out, go ahead and let me know if you find any more bugs. At some point I'm planning on releasing the source code if anybody is interested.

The Two Faces of SOA

To OOP fanatics Service Oriented Architecture is a language that different applications speak to each other in order to integrate. The applications are usually developed using OO technologies, but SOA provides the glue between them because OO falls down in this regard.

And then to others SOA is procedural programming in disguise. More precisely it's seen as an emergence of concepts that include procedural programming, data centric programming, and ubiquitous protocol standards for communicating this data between the procedures. 

Don't get me wrong, SOA can quite happily exist in both worlds, for better or for worse. Clearly the latter group is utilizing SOA at a much deeper level, and the former's adoption could be seen as merely cosmetic and convenient. They both have that SOA signature though. But which is better off?

Some people have said that it is SOA's current implementation that limits it from integrating better with object oriented designs. I feel this is wrong because object orientation is fundamentally founded on the concepts of exposing data piecemeal and only through agents that encapsulate the data. This approach is diametric to SOA where the focus is on data sets and schema and discovering services that can add value to that data.

Talking of data sets, Microsoft sneaked in a very SOA like model with their DataSet implementation, which is clearly not object oriented (there was no way until recently, for example, to add your own 'business logic methods' to a typed DataSet). This model clearly puts the emphasis on data and writing services that 'do things' with this data. Poor man's SOA perhaps, but definitely smells like SOA to me. A programmer's use of DataSets is usually a good indication of which of the two camps they fit into.

I think the biggest benefit people have drawn from using SOA concepts at the application level, rather than inter-application level, is that of loose coupling. Objects just don't loose couple very well. Sure you can have interfaces, abstract base classes and polymorphism to produce something that seems quite 'loose' - but it's not really. It's far more difficult to put OO applications into a remoting situation to reap the benefits of client/server computing, because objects are so tightly bound to an application instance. It's also far more difficult to persist and version objects, simply because the data representation of an object is often an after-thought. And of course everyone knows the integration benefits of SOA.

To be honest there are benefits and drawbacks to both approaches. SOA is certainly the trend at the moment, mostly because people are looking for less complicated and more enterprise-sociable solutions than OO designs offer. As to which approach will prevail, only time will tell.

Why Getting Users Involved is a Good Thing

So it was my turn to do the house chores. Vacuum cleaning this time.

I get the vacuum cleaner out, switch it on and a few seconds later I notice it's making a funny noise. I switch it off and tell my wife. I say something ridiculous like it needs oiling or we need a new vacuum cleaner.

"Oh don't be silly there's just hair caught in it" she exclaims. Over she comes. She gets the scissors and starts cleaning the brushes on the cleaner. A few minutes later she switches on the vacuum cleaner and started vacuuming the floor.

Now why didn't she complain and want me to do it, seeing as it was my job?

Because she wanted to see how well the cleaner worked after she 'fixed it'. I was happy watching her vacuum. She was happy to experience the fruit of her labor. It was a win-win situation.

The moral to this story? Getting people involved is never a bad thing.

You can apply this to every day development. Get users to contribute. When people contribute they have a vested interest in the software and want to experience the rewards of their input. Not only do they feel good seeing their work in action, they'll be more likely to spread good word about the software. And also when things go wrong they will happily take some of that responsibility because now the work is shared.

Recently I've been harking on about the good news of Domain Specific Languages. DSLs are a great example of how to get power users involved. Even simple application configuration is a good avenue to get user contribution.

The bottom line is that flexibility and accessibility are key to building this two way interface with the user.