Scott Stewart

message mapping, from service contract to component interface

The guys over at Thinktecture, namely Ingo and Ralf, have recently shared an interesting email discussion entitled "Interface design in distributed solutions". In the email thread, they discuss some of the similarities and differences between designing interfaces and contracts for services and components. In the end they develop a decision flowchart that can be used for rough generalizations on deciding which type of interface and contract designs to follow (component like patterns versus service like patterns). This is real interesting stuff and I enjoyed seeing the discussion develope the way it did. As Ralf states on his blog: "because hearing contrasting thoughts next to each other sparks your own thoughts in a different way than reading a just an article explaining some subject"

Towards the end of the thread, they discuss the topic of mapping data between the service data contract and the component interface. Basically, what we are getting at here is message to domain modal mapping, or in a lot of cases message to object impedance.

Ralf and Ingo discuss 3 solutions for dealing with this, which correlate roughly to:

1) The service's data contract class implements component contract data interface

2) Use some sort of reflection based framework to perform the mapping (i.e. the service's contract class looks a lot like the component contract interface and so we let reflection handle the mapping)

3) Explicitly define the mapping

Seldom in the real world is the ratio of this mapping relationship forever 1/1. It may start out as 1. And if you are the same person, team, company building components on one side of a service, the service, and the components on the other side of the service, these "component interface to service contract" and "service contract to component interface" mapping ratios may be forced to 1. In which case it's very tempting to go down one of the two easier streets that are numbered choice 1 and 2. However, I generally look at these two options as falling into that camp of slippery slope stuff; similar to the messy ORM discussions that always rant back and forth in the object to relational mapping world.

I generally lean towards explicit mapping. Do it, get it over with and move on. I believe this is a similar mindset to what Udi is getting at when he says "In the end, the only real thing left is to map messages to service layer calls…" Additionally, after doing some good size mappings between service data contract and component interface myself, I can appreciate Buddhike's follow-up on desiring for this type of mapping to be a toolkit code generator thing; and, as long as that automation is "code generator" automation in an attempt to accomplish choice 3 then I'm good with that too. You can utilize reflection in tools to help generate this code at development time if you like; especially at the beginning of the process. But, I do think the explicit mapping is important (regardless if by hand or with code generator help) and I don't think the automation of the mapping is a runtime thing as I think is expressed in choice 2.

At some point in the lifetime relationship between the service and the component, stuff will need to happen. The component will need to change in ways that a service exposing it can't or shouldn't expose, or the service contract will need to change (extend). When this happens, there is impedance, and the ratio is no longer 1. Perhaps, the necessary extension of the service contract will require the creation of a separate component and thus one service data contract will now use data from two separate components. Perhaps, within the domain that the component belongs, one of the data members within it's contract interface will need to change dimension. Perhaps an integer needs to be a double (I know it sounds crazy, but we all know this stuff happens.) The explicit mapping gives you a point to do the necessary rounding and / or type coercions so the service data contract can remain consistent.

I do understand that Indigo uses an explicit "opt-in" model, and this is cool and has tremendous value and I'm really looking forward to it. However, despite this, I still think that any auto-generated (or attribute based serialization, etc) stuff from the service contract message into service contract class realm, still needs to be considered part of the "service domain" and not the domain in which component belongs. The service process may actually host and give life to some group of components, but this doesn't mean that the service domain encompasses the component domain; it uses the component domain and therefore should be linked to it in an adapter like way. This adapter process is the explicit mappings.

I also realize that in my example above where the service data contract is extended and requires the support of two components we could have the service data contract class implement two separate interfaces. Once again, I just think it is generally dangerous to skip the explicit mapping step from service domain to component domain; as tedious as it may be and as 1:1 trivial as it may seem early on. If time, or resource constraints do force you to skip the explicit mapping step, then keep it in mind down the road before deciding to bastardize the service or the component for the sole sake of the other.

Where’s SOA?

You are probably familiar with the Where’s Waldo picture books, especially if there are children around the house. I’m starting to think it might be fun for us Geeks to start a new game; let’s call our version Where’s SOA? Instead of flipping through pages of a book, we’ll click through entries in our RSS aggregators, constantly looking for SOA. Just like the picture books, there will be all sorts of people dressed up like SOA, wearing his shirts, his glasses, maybe carrying his walking stick, but how will we know if its really him? It’s a tougher game then it sounds. At first glance you think you see him everywhere, but upon closer inspection you find it’s just someone imitating him. 

Perhaps the reason we can’t really find him is a) we really don’t know what he looks like, or b) maybe he’s not even really there?

Here are two very well respected people that are beginning to question whether he even exists; Clemens, Rich. If you know anything about these people (which I hope you do), you might fall into one of two camps; “He is the last person I would expect to ask that question” or “I always knew this guy really knew what he was talking about” I'm on the second campsite.

Here is a link to a whole group of people that are going to get together and play Where’s SOA. They promise to let us know what he looks like if they ever find him. It might make things easier for us when we are looking for him. This sounds helpful.

I have to admit, I love playing Where’s SOA, it becomes addicting. It seems every time I open my aggregator these days, I loose focus on the content because that rascally SOA guys keeps peeking at me challenging me to come look for him. This morning I was catching up on this week’s Tech news, looking to see if anything exciting happened while I had my head buried down in code and meetings all week. Besides for all the blogs I read, there are also the articles written by people getting paid to tell us what’s happening. One of the worst culprits for getting me into a good game of Where’s Waldo, oh, I mean Where’s SOA, are these guys. Man, they always suck me right into a good game. I think I’m going to have to unsubscribe from their mailing list, just so I can have more time on the weekends to get important stuff done, like picking up the yard from my dog’s week long mess and cutting the grass. Boy, if my wife ever found out that I postponed my chores just because I was playing Where’s SOA, I think I’d be spending a few days with the dogs in their house. She thinks’ I actually spend a few hours on Saturday morning working.

Here is today’s article that I spent most of this morning’s coffee pot with, playing a pretty fun round of Where’s SOA. This was a pretty tricky one that I really had a tough time with. On first glance, it looked like he was everywhere. But as I got close, he got sneaky and kept hiding behind the word “an”, as in “an SOA”. I hate when he does that because it makes me stumble as I look for him. For some reason, I always want to look for him behind an “a” as in “a SOA”, as in “a Service Oriented Architecture”. I might be wrong grammatically, but the point is that I myself just stumble getting there. I suppose the “an” is really referring to “an architecture” and the “SO” parts are just some silly adjectives that get in the way.

This article claims that these people currently have 180 Waldo’s hiding somewhere. But, then again they claim that they’ve been playing for about 3 years, so I suppose they are pretty good at finding him. They are hopeful that they can get this count up to around 1500 if all goes well. Wow, 1500 instances of Waldo, he should be really easy for us to find here.

They certainly did a good job explaining one of the reason I keep looking for him by quoting someone saying: “It sounds like a light beer commercial, but [an SOA is] faster, it’s cheaper, it’s better, at providing a more coherent IT strategy,” Wow, faster, cheaper, and just plain better! I really can’t wait until I find him too.

I thought I was on to something today (as it always feels when I’m playing), and perhaps that rascal SOA would be captured at last. But unfortunately I lost him again; he managed to finally escape for good somewhere around here “Challenges in SOA deployments include platform heterogeneity, message brokering, data silos, security, and lifecycle management”…”Security and the issue of data silos can be addressed through security and data services layers respectively” and here “The true measure of an SOA is its ability to enable service reuse”...“At some point, someone has to stop writing code”

I thought the SOA I was looking for solved the platform heterogeneity issue, not exasperated it? Right? Isn’t that one of the benefits? “Service A” could be hosted on a Win platform, “Service B” on a Linux flavor, and “Service C” could be exposing some mainframe batch processing results. Isn’t SOA supposed to be what makes this seamlessly all act like one big homogenous “A”. If the platform heterogeneity issue is a challenge of SOA, then I think I’m looking for the wrong guy.

I also thought the SOA I was looking for would not be challenged with “data silos”; but embrace them. If each Service in my SOA follows the tenets of good SO, then there is no such thing as a data silo problem; it is a data silo design. Each service is supposed to be autonomous and explicit. Therefore, it would be expected that any data concerns would be internal to that Service and should be invisible to the other services. You know, “what happens in Vegas, stays in Vegas” It comes with territory. You can leave messages for the people in Vegas, and they might even call you back when they get a chance. But, you will never really know what’s going on in there. That’s just the way it is when people go to Vegas. Additionally, does “solving the data silo concerns” by introducing “data services layers” sound like explicit boundaries and automity? I’m not sure. “Data service layers” is an interesting topic, with a lot of value. But a “Data service layer” shared between the S’s in a SOA is like telling people about your Vegas stories, which I don’t believe you are supposed to do.

I got worried that I might really be missing something here when I read that they are solving the Data Silo challenge of SOA by introducing Data Layer services. So I ran a quick Google search on [“Data Silo” SOA]. And much to my surprise, the number 1 hit was this Enterprise Architect article, entitle “Architect Data Services”, which included this comment in reference to SOA: “…As long as these services each have their own, nonoverlapping data, a messaging layer such as an enterprise service bus (ESB) is sufficient. However, when services share interdependent underlying data—such as inventory or customer data—a data services architecture (DSA) is required in addition to the ESB to ensure that each service is working with consistent information.” That’s just scary. Multiple services operating on the same data? Now, I’m dizzy. These can’t be SO services, can they? If we look at the now famous 4 tenets of SO, could we ever, ever say something like this: “services share interdependent underlying data” And, if they are not SO services, then what does the “SO” in the “SOA” of this article speak to? I’m afraid that I’ve lost Waldo for another day.

more on Database as a service...

Maybe Mr. Lhotka wasn't all that far off the mark when he wrote this over here last month, and perhaps there was less “tongue in cheek” then most assumed.

I came across this article on InfoWorld today titled “Desperately seeking SOA”. The following paragraph from the InfoWorld article seemed to just conjure up spirits of the fallacy article from last month:

 “The database community is also heading toward SOA. Plans are afoot to enable IBM DB2, Microsoft SQL Server 2005, Oracle 10g, Sybase (Profile, Products, Articles) ASE, and other platforms to participate actively in Web services-based SOA activities as first-class citizens -- even without the use of application servers. This will have profound implications for the design and management of widely distributed n-tiered applications because, in effect, hierarchical tiers will become horizontal peers.”

data layers, soa, and tootb

There is a very interesting article over at TheServerSide today by Rocky Lhotka entitled “The Fallacy of the Data Layer”. IMO, there aren't too many people that really “get it” more then Rocky.

Now, before you link over there, read the article, and start getting all worked up...I've followed enough of Rocky's blogs, articles and comments to go out on a limb and assume that he is not suggesting we go into work tomorrow morning and start replacing our Ado.Net data access code with web services. Instead, I think the article challenges us to continue questioning the notion of exactly what role web services play (if any) inside an application.* By thinking out of the box here a little about data sources, and their similarity to other external interfaces, I think Rocky helps focus attention back to the fact that the application itself should in deed be the center of the architecture.

This is the very basic point that seems to get lost in most other articles that mention SOA. And when it gets lost in articles its one thing, but when the point is missed in real world architecture, it can be painfully expensive. I am speaking first hand about this experience.

By thinking of the Data Layer as we would any other external interface, we better align it with other similar responsibilities; things that give data, receive data, and otherwise interact with our application (i.e. event actions). But we also do one more thing that I think is really the kicker here, we eliminate one more reason why people feel overly compelled to use web services inside an application.

Because accessing a data source has additional security concerns, it is often the case that not all physical tiers of an application can physically connect to a given database. When this happens, and we are not looking at our architecture as Rocky suggest in the article, then we often end up with an application architecture that gets pretty messy. We end up with an application that eventually gets split in two (if not more) pieces by a web service. We have part of the application (that can't see the database) on one side of the web service, and the other part of the application on the inside of the web service (the part that can see the database), and we somehow think that these two applications are really one. Business logic begins to get duplicated in both halves of the application. It becomes very tricky to determine which half of the application is responsible for what. We often find that we need some behaviors from one side of the application on the other side. We duplicate security, we duplicate validation, up to the point that someday we realize that what we have now is two applications instead of one. Not only do we have two applications, but they are two very similar applications, often with copies of the same business code in both. The only real difference being that at the edge of one we interact with users and at the edge of the other we interact with a database. Both interact with one another. And often this split of the application with a web service is initially done because of data layer concerns. This is seen and heard in many architectures discussions today and is often described as using a web service between the front end and back end of an application.

You might think, whats the big deal in describing the split in the application as above or saying that data access is a service; “don't I still have this problem of two applications divided by a service“. The answer is yes and no. In the previous description, we attempted to rationalize a single application split in two, but now we have two clearly defined application responsibilities, and therefore two applications. The application that is the Database service, has clearly defined duties. As an application, its externally exposed service is simply sending and receiving data. Internally, this data is simply persisted to a Database. Any business logic performed, would be clearly defined to the responsibility of retrieving and persisting this data; ie. data type, length etc. But, truly only from the perspective of data persistence. The other application (which is our actual business application) is the single domain that all business logic takes place. This includes business rules on data validations, code to perform special calculations, applying user settings, and all role authorizations, etc. The stuff that makes up the core of what the application is about. We no longer have all of this duplicated between both (halves) of the application. Why?- Because we now have two applications, not one application with two halves.

*By inside an application, I am speaking of between layers of an application 

buildings, factories and trains

A recent blog by Ted Neward has created a little give and take on the hot SOA topic. I couldn’t help but chime in. The topic, and the term's missue are just something that I have gotten pretty passionate about lately.

This is actually a follow up to a comment of a comment of a comment (well you get the picture)…

I have to admit that I am biased towards the “SO(A) = = buzz” (or maybe that is choo choo) side of the camp. Now maybe it’s because I am a small time developer, in a small time city, unaware that the SOA train is going to run me over on the way to building a bigger better Metropolis. However, most of the things I work on building on a daily basis are “factories and buildings”. Some client’s and employers like to market them as Enterprise factories and buildings, but they are still for the most part factories and buildings. The problem is that the SOA buzz leads many developers and architects that I speak with, who are in deed designing factories and buildings, to think they need to drive a train between the offices (or dare I say departments) located in the building. This doesn’t make sense. If my factory or building needs to get some goods from this new railroad turning our cities into Metropolises, then by all means take advantage of that. Similarly, if other factories, buildings, cities, or Metropolises can take advantage of goods you offer, then by all means put them on the train. But, be very careful about trying to drive that train inside your building; because the closest it should come to the factory or building is to the building's “edge”. Maybe someday I will become elected to a City council, and my goal will be to create a bigger better City (hopefully this pays better the building “factories and buildings”). When I’m important enough to be entrusted with a task like that, then creating programs that bring the best of all my buildings and factories together will be very important to my ability to grow my city to a Metropolis

I doubt that Pat will ever see this blog entry. However, if he does I hope it is not taken the wrong way. I love the Metropolis speech. I love reading and learning from all things Pat has to offer. I love thinking about the benefits we can realize by taking advantage of functionality already built by separate autonomous systems, and using that functionality between various platforms. I love thinking about the awesome power and concepts that will be realized in Service Oriented Entripises. I love the things that Web Services (and WSE) are doing for us now and promise to do even more of tomorrow. However, I hate the enormous amount of misguided direction and suggestion that when I am building a factory or building I should be running train track between offices. I will always try to place my buildings and factories, as close to as much train track as possible, so that I don’t get left out of the quickly growing Metropolis. I will get whatever goods I can from the train track (as long as they are cheaper then I can get elsewhere), and I will be more then happy to place goods onto the train when others have needs for my goods. I really don’t think this is too far from the true spirit of SOA. But, for whatever reasons (probably too much luggage) the SOA train is currently getting off track and finding its way deep inside our buildings and factories. choo choo