Do I need an ESB to do SOA?

Posted 24 July 2009 By Nigel Bakker and has 4 comments - click to add.

The short answer is no. While an ESB (Enterprise Service Bus) adds significant value, there exists no direct technical reason that you need an ESB.

Perhaps a more pertinent question should be: Should I use an ESB in my Service Oriented Architecture?

ESBs have come a long way over the years driven by many factors including the increasing adoption of SOA and the resulting maturity of relevant standards. Most ESBs started as simple message based integration solutions, providing the means to integrate heterogeneous applications in a consistent manner. The bulk of the major ESB products have their roots in message oriented integration solutions.

Searching the Blogosphere there is an incredible amount of vitriol around ESBs. Much of it seems to me to be an emotional reaction to poor implementations. Personally, I blame the vendors who promise the world and deliver very little, often hijacking your architecture by attempting to fit it to their view of the world.

The fact is that you can implement a “pure SOA” without an ESB. But somewhere in the mix you actually do need real technology. The SOA space is new(ish) and the technologies are still maturing and converging on standards, which themselves are maturing.

There does not even seem to be standard definition of what constitutes an ESB. What does seem to be common amongst the products offered by the industry leaders includes:

  • Sophisticated messaging; Most ESBs have their roots in messaging so it’s no surprise that this is both common and a strength. You typically find both point-to-point and publish-subscribe supported. What varies a lot are the capabilities of the management tools, especially in the management of subscriptions and routing.
  • Support for Web Services; This has become universal – most ESBs provide “out of the box” web service technologies for wrapping existing functionality as web services.
  • Business Process Orchestration; SOA only really begins to add value when services are course grained and business oriented. Business processes cut across business silos, so the use of BPM tools is a natural approach to driving out business services.
  • Mediation; Normally this is provided through so-called adapters. Adapters provide a means of accessing data and functionality in a variety of legacy applications. These range from basic file access, to sophisticated application specific API level access (think SAP etc).
  • Service Orchestration; Often referred to as service composition, this is a means to combine multiple services into a single logical service without writing any code.
  • Routing; Often part of messaging, it can also play at the service level in terms of service virtualization. Routing can be static or dynamic based on content and other criteria.
  • Management; Monitoring, auditing, control, admin etc. This adds significant value from an operational perspective.

What this list represents, is mostly plumbing. Depending on what technologies your organisation uses you should already have some plumbing for creating services. What an ESB gives you is a coherent plumbing toolbox that frees you from needing to concentrate on too much technical stuff.

An ESB is not a silver bullet. In fact I would go so far as to say that silver bullets (of the software kind) do not exist. An ESB will add a number of useful tools to your toolbox, but you will still have to do a lot of work to get your SOA in place. Simply buying and deploying an ESB does not mean that you suddenly have a SOA. In fact, this approach is likely to lead you toward what’s become known as an ESB Oriented Architecture. This has common elements with a SOA, but is not a SOA and will not give you the same benefits. What you will have is an integration platform and not much else.

SOA is a mindset, not a technology stack. Be wary of ESB vendors who come armed with a so-called reference architecture. Architectures must be designed to fit your specific needs. There are many common SOA patterns that will apply in many different organizations, but the way in which they are applied and combined tends to be unique.

When embarking on your SOA implementation, first work out what your architecture should look like (your to be view of the world). Then sit down and evaluate which technologies you need to implement your vision. From this you can decide which ESB (if any) provides the best out of the box fit. There will probably be gaps. Look for products that plug these gaps, or be prepared to plug them yourself (as a last resort). Do not let an ESB vendor highjack your SOA. Make them fit their product to your vision, not the other way around.

So let’s restate the original question as: ‘Can you implement your SOA without an ESB?’ The answer of course is: ‘Yes’. Should you implement your SOA without an ESB? ‘Probably not’.

Categories: Technical SOA

SO(A) what’s all the fuss about?

Posted 13 July 2009 By Nigel Bakker and has no comments yet - click to add.

As is usual in the software industry, there is an incredible amount of hype and noise around this thing called SOA – otherwise known as Service Oriented Architecture.

But business is weary and sceptical of hype, and rightly so. Over the years the IT industry (software in particular) has made many promises but very rarely do these promises get met. Why should SOA be different?

Probably the biggest thing in favour of SOA being different is the fact that SOA is not in fact a product, or even a technology! Rather it’s a mindset, or an approach to actually leveraging some of the value locked into the rat’s nest of software applications foisted on an enterprise by vendors and even their own IT department.

The first rule of succeeding with SOA when you start down the SOA road (when not if!) is do not believe the vendors. They lie. All of them! (So what’s new?). I have had many dealings with various vendors of so called SOA software products, ranging from governance tools, to ESBs (Enterprise Service Bus), and have been lied to by all of them. Oh, they believe what they are saying – which in their minds means they are not lying! But, they all over-promise and under deliver.

Don’t let my vendor rant dissuade you though, there is real value to be had in the tools and products dispensed by the various vendors. It is up to you to find the value and leverage it however.

Back to the real question though, what is the value proposition behind SOA? Haven’t we heard it all before?

Yes we have, but until now, we had not reached a point of maturity that would allow us to really exploit and realize the potential of SOA. Several things have happened, most notably the standardizations that have been driven by the rush to the web. These standardizations have not only happened, but are actually in widespread use. Finally it is genuinely possible to expose functionality from legacy systems as services. Not necessarily easy, but definitely doable for a reasonable cost.

In fact, the current recessionary climate is likely to be a boost to SOA adoption. In simple terms, businesses are looking to sweat their digital assets longer and harder, and SOA is just the approach to enable this. Rather than automatically upgrading to the next version of this or that, businesses are asking themselves the question: “Do I really need the new version? Can I use what I have differently?” Perhaps somewhat surprisingly, the answers are coming up: “No” and “Yes” respectively. The question that follows is then: “How do I use what I have better?”

Part of the answer is SOA. Not the whole answer, but certainly a significant share of the answer lies in enabling the business to leverage what they have, better. SOA provides us with a mental framework for leveraging existing assets. But, there is a caveat. SOA is NOT simple. It is NOT easy. And, most definitely, it is not a silver bullet! (Done without care it may well be a lead bullet to the CIO’s head.)

Adopting SOA is a journey, first you need to understand that it will ultimately change the way you do business. Everything you do in the business needs to be reviewed and weighed and measured to see if it can be leveraged as a service. To do this, you need to revisit all your business processes, and look for automation opportunities within each process. Start with the processes that are fundamental to your day-to-day business and expose the common elements as services. Immediately, these will become available to be used in different contexts. If done correctly, it will not only be possible, but relatively simple to combine existing services in new ways and in so doing, drive innovation in your business.

As you progress down this road, delivering new functionality increasingly becomes a matter of leveraging what you already have whilst only adding new bits to fill the gaps. At this point your business has become truelly agile, and SOA has become a way of life.

As I’ve already said though, this is not easy. Beware the salesman who says “we’ll install so-and-so ESB and SOA suite and you’ll be done!”

The fact is, you probably do need some new technology to make this magic happen, but be sure that you have a clear idea of what you need before you embark on a spending spree. SOA is still a relatively immature area in the software vendor stack. A significant amount of consolidation has happened with big vendors buying smaller vendors to get their hands on bits they don’t have. Most of these acquisitions are still being digested.

If you already have an ESB in your environment, keep it until it reaches the point of becoming a hindrance. At that point you will have a better idea of what your real needs are. Most importantly, employ the services of a vendor neutral SOA specialist to help you get started and to lay the foundation of your own approach. Only then, bring the vendors in and make them fit their tools to your vision, if they can!

Categories: The Business Case for SOA