David Black: SOA

Vista opens the door to SaaS

Fri, 16 Feb 2007 13:35:13 GMT

Vista opens the door to SaaS. Vista certainly makes a good (or bad depending on who you work for) contrast with what is happening to the software market.

James Pasley's Just Enough XML to Survive

Thu, 08 Feb 2007 12:56:35 GMT

Excellent video presentation from James Pasley, Cape Clear's CTO, on XML, XML Schema, SOAP and WSDL. 15 minutes that could just change your (professional) life for the better!

Cape Clear one of the Top 20 Products of 2006

Mon, 22 Jan 2007 16:03:06 GMT

Network Technology Magazine has voted Cape Clear one of the Top 20 Products of 2006. This adds to our already impressive bag of awards for 2006 and demonstrates once again that the Cape Clear ESB is the product to put top of the list when evaluating SOA platforms.

Top 10 SOA Worst Practices webinar

Mon, 22 Jan 2007 09:48:46 GMT

It seems every software company is pitching some form of SOA Best Practices. But even ignoring the built-in bias, some of the best advice can still go very wrong. Instead, by understanding the pitfalls and landmines of SOA implementations, you will be in a better position to create a strategy that makes doing the right thing easy.

Learn from Cape Clear’s expertise across hundreds of SOA deployments and avoid critical mistakes that could spell failure for your SOA initiative.

Join Cape Clear’s CEO Annrai O’Toole for a down-and-dirty discussion on the Top 10 SOA Worst Practices. General topics include:

Short-cuts you can’t afford to take
Using services the wrong way
What not to do with BPEL
Performance testing no-no’s
Executing projects without the proper tools

This webinar will take place on January 31st, 2007 (08:00 PST, 11:00 EST, 16:00 GMT, 17:00 CET) and you can register here.

OSGi-everywhere

Mon, 15 Jan 2007 18:10:23 GMT

There was a lot of buzz in 2006 around OSGi and it looks like 2007 will be the OSGi-everywhere year. One of the more interesting posts I've come across is this one, a reference to a talk about the Newton open source project and dynamic service grids in the enterprise and OSGi as a basis for SOA. I'm hoping to try some of this out for large-scale (and large scaling) distributed systems shortly. More on that in due course.

Neat things to do with your ESB

Thu, 11 Jan 2007 15:26:22 GMT

I just love this post from James.

Cape Clear 7.0 BETA is now available

Wed, 03 Jan 2007 08:23:57 GMT

Happy New Year and Happy New BETA! The Cape Clear 7.0 BETA is now available for download. Check out the brief summary of some new features, participate in the BETA on developer.capeclear.com and have a look at the detailed What's New. I reckon its going to be a great 2007 :-)

Workday Selects Cape Clear for SOA Integration

Tue, 07 Nov 2006 08:20:52 GMT

Workday launched their company and first product offering this morning, with Cape Clear as a strategic partner enabling Workday customers to better integrate with third-party applications. Workday, founded by Dave Duffield, the founder of Peoplesoft, is a highly anticipated, on-demand solution for Enterprise Business Services, and has selected Cape Clear for SOA integration. As a native web services SOA company, Workday needed an agile ESB platform – as opposed to a complex legacy EAI solution – to aid customers in integrating with third-party applications. They chose the Cape Clear ESB Platform to handle their SOA integration because 1) we are the leading ESB platform, and 2) we are the visionaries in ESB technology. We’re really excited here at the Cape because Workday is a first-mover in delivering a software-as-a-service offering based on a SOA, enabling easier and richer service-level integration with customers and their native systems.

Taking advantage of BAM derived business intelligence

Fri, 29 Sep 2006 15:39:31 GMT

For many years I developed real-time data distribution, display and analysis software for the financial markets front office. Traders are a great example of a human BAM. Traders watch real-time data feeds for business activity that interests them. They filter it using both their mental skills and their workstation software. They bring historical context to their analysis, doing complex event processing and running rules in their heads and in their spreadsheets. They make decisions and execute trades that affect the state of the market, altering the business activity they are monitoring. The trader is looking for opportunities with their "business", as opposed to looking for problems. The goal of BAM is not just to know that everything is running smoothly, its to look for opportunities to improve things. It is not surprising that Business Activity Monitoring and Business Intelligence software has its origin in the financial markets.

Having a real-time view of our business operations is largely pointless if we cannot rapidly effect change to put problems right and take advantage of new opportunities. Building business processes on the agile, SOA-enabling, orchestration and services technology provided by an Enterprise Service Bus is the key to taking advantage of new opportunities and fixing problems before they cost us money. The ESB is a natural way to collect, correlate and feed the raw data required by the BAM layer, it then allows us to rapidly improve our processes in response to changing conditions.

Feeding aggregated business intelligence back into processes in real-time creates interesting possibilities. Suppose we’re orchestrating a supply chain. If we have a number of Courier services in the supply chain, that the Shipping service can request pickups from (i.e. pickup packages at the various Warehouses for delivery to customers), we can monitor the timeliness of the pickups and check for Couriers falling behind, failing to pickup on time and failing to deliver at agreed SLAs. We can feed this business intelligence back into the Shipping service, so that it favours couriers meeting their SLAs.

One runtime, many servers

Sun, 27 Aug 2006 20:08:45 GMT

One of the improvements we have in store for the next release of Cape Clear is ability to target a web service at a single runtime, but deploy it on different servers from Cape Clear Studio. This means a web service project (module) can be deployed on e.g. a local server, a remote server or a clustered server without having to adjust its target runtime in Eclipse. We’re looking to push this improvement back into the Eclipse WTP project. To follow the progress of this, see this Eclipse bug.

Cape Clear - one to watch

Tue, 22 Aug 2006 14:41:28 GMT

Cape Clear Software has been named a ‘Company to Watch’ in the 8th Annual Intelligent Enterprise Editor's Choice Awards

Refactoring SOA Artifacts - Part 1

Fri, 18 Aug 2006 08:29:26 GMT

One of the key features that first hooked me on Eclipse was its (Java) refactoring capabilities. Refactoring is an indispensable tool in the Java code jockeys toolbox, and Eclipse makes it easy. The Eclipse Language Toolkit (LTK) provides a framework within which to build refactorings.

As we move into the realm of service-oriented programming, developing services that can be deployed on Enterprise Service Buses, I'd like to see the same level of refactoring available for SOA artifacts. By SOA artifacts I mean things like WSDL, BPEL, deployment descriptors, as well as Java classes and other language files. At the detail level, the relationships between these artifacts can be complex.

With WSDL described services, when the WSDL changes some refactoring often happens automatically i.e. tooling can regenerate the proxy code for a service when the WSDL describing the service changes. But beyond this kind of thing, more sophisticated refactoring is required. In fact, whether this refactoring is done explicitly by the user, or implicitly by the tooling reacting to changes in WSDL and automatically altering other artifacts, much of what needs to be done is the same.

The use case I'm starting with is a simple one. A BPEL script implements a service defined using WSDL. The BPEL script invokes another service also defined in WSDL. It should be possible to rename a portType operation by highlighting the operation in the WTP WSDL Editor in Eclipse, right clicking and choosing Refactor | Rename. I'm considering both the cases of the service implemented and the service invoked.

It is worth noting that the WSDL editor in Eclipse 3.2 provides a basic rename refactoring feature, but to date I think its pretty limited.

The LTK provides a high level framework for refactoring, within which I'm going to put the framework for SOA artifact refactoring. The basic steps are something like this:

Identify the refactoring the user wishes to perform e.g. renaming a WSDL portType operation.
Find all the projects and artifacts in the workspace that should be changed.
Perform the initial sanity check.
Ask the user for more information if necessary.
Calculate the changes required to each artifact.
Check that each artifact will still be valid after the changes.
Present the changes to the user for acceptance or alteration.
Commit the changes.

Lets have a look at a snippet from a really simple WSDL file, HelloWorld.wsdl, that shows an operation called sayHello:

<wsdl:portType name="HelloWorld">
<wsdl:operation name="sayHello">
    <wsdl:input message="tns:sayHelloRequest"/>
    <wsdl:output message="tns:sayHelloResponse"/>
</wsdl:operation>
</wsdl:portType>

Lets suppose this WSDL operation has a SOAP binding. The operation name also appears in the binding and in the soapAction attribute in the binding:

<wsdl:binding name="HelloWorldBinding" type="tns:HelloWorld">
<soap:binding style="rpc" transport="http://schemas.xmlsoap.org/soap/http"/>
<wsdl:operation name="sayHello">
    <soap:operation soapAction="cc:HelloWorldService:HelloWorld#sayHello"/>
    ...
</wsdl:operation>
</wsdl:binding>

As in the example, by convention message names often follow the form: Request, Response. It would be great if the user could choose to also rename these as an optional part of renaming the operation name.

One of the Eclipse plugins I'm going to develop will house the framework core. The implementations of the various parts of the framework can be contributed via extension points.

Lets consider the problem of finding all the relevant artifacts and the projects that house them across the Eclipse workspace. While this first use case considers just WSDL and BPEL, the search-for-artifacts step, at a high level, is general enough.

The job of the Artifact Finders is to find projects and their artifacts that require changing in response to the operation rename.

Consider the case where the HelloWorld.wsdl file lives in one project, some projects have a link to it and others have a copy of it. The projects with a link to it are easy enough to find. Any projects that contain a file called HelloWorld.wsdl are also candidates, but just because one WSDL file has the same name as another does not mean they represent the same service, or the same version of the service. A trickier case again is where the WSDL file is indirectly referenced e.g. it is imported by another WSDL file, or referenced in a descriptor file. For these latter cases we may require some deep searching, which could be time consuming, so we need to bear that in mind.

Artifact Finders are one element of the framework. The Artifact Finder interface will look something like this:

public interface IArtifactFinder {
public RefactoringParticipant[] findArtifacts(SOAArtifactRefactoringArguments args);
}

And the extension point is defined like this:

<extension point="org.codecurl.soa.refactor.finders">

<artifact-finder
    id="foo.id"
    name="Foo Refactor BPEL Artifact Finder"
    class="org.codecurl.soa.refactor.finders.FooFinderImpl"/>

</extension>

Lets look next at the changes required to the BPEL script, considering first the case where the BPEL service implements the interface defined by the HelloWorld.wsdl file. The variable definitions for the input and output messages may need be to changed (if the user has chosen to also rename the input and output message types):

<bpws:variable name="sayHelloRequest" messageType="ns:sayHelloRequest"/>
<bpws:variable name="sayHelloResponse" messageType="ns:sayHelloResponse"/>

For consistency, we should allow the user to rename the variables too if the message type names are changing. The receive starts the process:

<bpws:receive partnerLink="HelloWorld" portType="ns:HelloWorld"
   operation="sayHello" variable="sayHelloRequest"
   createInstance="yes">
</bpws:receive>

So the operation name is changing and maybe also the variable name. And the process ends with a reply:

<bpws:reply partnerLink="HelloWorld" portType="ns:HelloWorld"
   operation="sayHello" variable="sayHelloResponse">
</bpws:reply>

Again, the operation name and possibly the variable name will change.

In the case where the BPEL process is invoking the HelloWorld service, we need to consider the invoke element. The operation name attribute value will change, and the inputVariable and outputVariable attribute values may change:

<bpws:invoke partnerLink="HelloWorld" portType="ns:HelloWorld"
   operation="sayHello" inputVariable="sayHelloRequest"
   outputVariable="sayHelloResponse">
</bpws:invoke>

We need to set up the invoke call by populating the sayHelloRequest message, so we should look for assign elements that use the variables:

<bpws:assign>
   <bpws:copy>
     <bpws:from><xs:string>Bob</xs:string></bpws:from>
     <bpws:to variable="sayHelloRequest"/>
   </bpws:copy>
</bpws:assign>

Callbacks (onMessage) also need to be considered.

One of things I want is APIs for refactoring the various models (WSDL, BPEL, whatever). I like to start with a "perfect" API for something like this, and then implement that with off the shelf components (like WSDL4J) and whatever additional code is required.

Once I have the framework plugin working for this simple example, I will look at contributing to the appropriate Eclipse project/area. Maybe the SOA Tools Platform and/or the LTK.

Clickable SCA diagram at Tuscany wiki

Thu, 17 Aug 2006 08:43:48 GMT

There is a nice clickable SCA diagram available that describes the basic SCA concepts.

Cape Clear ESB keeps on winning with IT Week Magazine Editors Choice Award for 2006

Wed, 09 Aug 2006 09:50:07 GMT

"One of the best ESB products available and it wins our Editor's Choice Award for functionality and performance." - IT Week, July 31, 2006

"Clear's Cape Clear ESB 6.6 provides an intuitive and capable platform for creating, managing and deploying services, processes and data for an SOA infrastructure."

And Cape Clear will just keep getting stronger - we have a host of goodies in store for later this year. I can't wait! :)

SCA v0.95 - some much need simplification

Fri, 04 Aug 2006 16:16:13 GMT

I am so glad to see that in the latest SCA draft specification the assembly composition has been greatly simplified using a recursive model. I thought the whole module, fragment, sub-system and system thing was hopelessly confusing (how many times did I read the 0.9 spec?!). Now we have Composities (which are just the same as Components) that include other Components and so on. So its basically composite services and services.

New Cape Clear ESB University courses starting shortly

Thu, 03 Aug 2006 08:31:09 GMT

The Cape Clear ESB University is an educational and training initiative aimed at providing software developers, systems integrators and IT architects with the information and tools they need to ensure the success of their Service-Oriented Architecture (SOA) projects. These live educational webcasts, which will be held during the month of August, will support organizations in the design, development, deployment and management of SOA and ESB projects, using relevant business case examples.

You can register for these courses here.

The first course covers the Principles of SOA Design Using Enterprise Service Bus. This session focuses on the SOA creation process, principles of good SOA design, and the use of an ESB to enable SOA projects.

During this event you will learn:

- Service-Oriented Architecture: The accepted architecture for reducing development costs, increasing IT agility and improving IT/business alignment
- Enterprise Service Bus: The new technology platform for realizing SOA
- Standards: What are the standards that relate to SOA?
- True ESB: The standards-based ESB platform for SOA

On these courses you can learn how Enterprise Service Bus can Leverage your SOA Initiative.

The Cape Clear ESB University has been designed to accommodate the diverse needs of those who are responsible for creating a SOA strategy, design and implementation, helping to ensure projects result in reduced development costs, and increased business agility.

The ESB University webcasts will include sessions on the following topics:
- Course 1: The Principles of SOA Design Using Enterprise Service Bus
- Course 2: SOA--Introduction to Process Orchestration and Top Down Service Design
- Course 3: Mapping and Transforming Diverse Data Formats Using SOA
- Course 4: Service Enablement and Quality of Service with SOA

Sign up now!

---------------------------------------

Course 1, August 9, 2006

8 AM PDT/11 AM EDT/4 PM GMT

The Principles of SOA Design Using Enterprise Service Bus

---------------------------------------

Course 2, August 16, 2006

8 AM PDT/11 AM EDT/4 PM GMT

SOA - Introduction to Process Orchestration and Top Down Service Design

---------------------------------------

Course 3, August 23, 2006

8 AM PDT/11 AM EDT/4 PM GMT

Mapping and Transforming Diverse Data Formats Using SOA

---------------------------------------

Course 4, August 30, 2006

8 AM PDT/11 AM EDT/4 PM GMT

Service Enablement and Quality of Service with SOA

---------------------------------------

SOA tooling

Mon, 24 Jul 2006 09:10:57 GMT

Imagine that your job is to develop service-oriented programming (SOA) Eclipse-based tooling. I don't have to imagine :-)

Cape Clear continues to lead the Enterprise Service Bus market

Thu, 06 Jul 2006 08:21:37 GMT

Its great to see that Cape Clear continues to lead the Enterprise Service Bus market as "one of the pioneers in the enterprise service bus (ESB) category and continues to be among the leaders in technology innovation and strategic vision in this space". Read the Forrester report here!

OSGi tutorial at ApacheCon Dublin

Wed, 28 Jun 2006 21:37:47 GMT

ApacheCon Europe 2006 is being held in Dublin this week. I couldn't attend the whole week, but I did get to go to a half-day OSGi tutorial given by longtime OSGi evangelist Peter Kriens. Peter, who is the current editor of the OSGi specification, says:

"OSGi is kind of SOA"

"OSGi is kind of a Java Operating System" or at least a "mini-kernel"

OSGi is "the Dynamic Modularity Layer for Java"

OSGi is a Java framework for developing remotely deployed service applications. OSGi started life in 1998 with the original use case of home automation. OSGi applications can run on different software and hardware architectures. There is a lot of buzz around it right now, particularily in the enterprise space, partially because of Eclipse Equinox, which is OSGi R4 based and the technology underlying Eclipse bundles (actually Equinox goes beyond pure OSGi). But there are a bunch of other implementations, including the Apache Felix project. The specification is freely available and royalty free.

OSGi applications adhere to a basic SOA, the contract between client and service is based on a Java interface and discovering and binding available implementations is done through the framework. Client and service are softly coupled. So it provides an in-VM service model. Declarative services, introduced in R4, are akin to having dependency injected components. Applications declare their service dependencies in an XML file, and are notified by the framework at runtime as services becomes available and go away. The last is an important point: services can go away at runtime, just as they can be dynamically discovered. The next major release of Spring is adding support for OSGi. One of the tutorial attendees made the point that OSGi is a generic event notification framework.

OSGi applications are packaged as bundles, handled by the OSGi Module Layer. The Module Layer addresses packaging, class loading modularization, protection and versioning (multiple versions of the same package) - things Java does not do well, or at all, by itself. In addition, OSGi provides basic Life Cycle Management. An API for managing bundles allows them to be installed, resolve their dependencies, started, stopped, refreshed, updated and uninstalled. So basically, bundles register one or more services with the framework service directory, and other bundles can use those services. Service contracts are just Java interfaces, so service implementations are just POJOs and no special classes need be extended nor interfaces implemented.

Its a pity OSGi wasn't somehow used as the basis of JBI, there are so many similarities.

Its also a pity that there are currently two rival JSRs attempting to establish a dynamic modularity layer for Java SE. JSR-291 is the OSGi Alliance backed one that aims to "define a dynamic component framework including component lifecycle for existing Java SE platforms". Peter is on the expert group. JSR-277 aims to do more or less the same thing, expect not as well according to Peter - who is is not on the expert group despite his best efforts! Richard Hall was at the tutorial, he is on both groups. According to Richard he is trying to steer the 277 folks in the right direction. It will be interesting to see what happens. Richard was giving a couple of OSGi related talks in Dublin, I wish I had been able to attend them.

And I think the Eclipse Rich Server Platform proposal deserves some serious attention. There is so much happening in the module/component/service packaging, assembling and deployment space (the McSPAD space) (ok, I guess thats several kind of overlapping spaces). We've got JBI, SCA (though don't hold your breath) and now (yes it has been around for quite some time) OSGi. What we really need to do is get everyone in an aircraft hanger (big enough?), seal the doors and tell them they are not getting out until there is only one. One McSPAD to rule them all.

AMQP: an open specification for queue based messaging

Sun, 25 Jun 2006 20:13:40 GMT

The Advanced Message Queuing Protocol (AMQP) 0.8 spec has been released. AMQ is an open specification for queued messaging that was born at JPMorgan Chase under the direction of John O'Hara. JPMorgan have formed the AMQP Working Group, together with a bunch of companies including Red Hat, Cisco and Iona. They aim to create an alternative to expensive proprietary messaging middleware like IBMs MQSeries. IBM aren't the only vendor selling an overpriced queuing system, I reckon TIBCO fall into that category too. I understand this has been brewing behind the scenes for a while, but a lot of folks weren't sure it would ever see the light of day. As Brian McCallister puts it this is a space badly in need of an open standard, but has concerns about whether it will be properly opened up. According to InfoQ JPMorgan have an implementation in production now. I've worked on a number of messaging systems for investment bank trading floors (and beyond) over the years, going back to NetBIOS days, reliable broadcast with UDP/IP, point-to-point based systems with TCP/IP, including HTTP tunneling based Internet trading infrastructure. Back in those days the messaging systems were the product. These days they should just be part of the woodwork, and hopefully AMQP, OpenWire and implementations like ActiveMQ can bring scalable, performant and reliable asychronous messaging to all enterprises.

Enabling effective BAM with the ESB

Sat, 10 Jun 2006 12:44:24 GMT

By now most folks understand the advantages of running and orchestrating key business processes using SOA principles and Enterprise Service Buses, like the web service standards based Cape Clear ESB. There is a lot of business activity taking place on the ESB, and it'd be great to be able to monitor that, gather the intelligence and analyze what our businesses are telling us. In other words putting the ESB and Business Activity Monitoring (BAM) together.

One of the key issues when considering BAM is being non-invasive. A BAM should be able to collect the data it needs to present a real-time dashboard of business state and Key Performance Indicators (KPIs), without you having to engineer that into your applications. Another key issue is correlation. Lots of bits of data and process state go into creating a holistic picture of a business process. A BAM needs to be able to tie all those bits together. Different systems often use different keys to identify a particular instance of a process. Sometimes the interaction between systems is terse e.g. in a synchronous request/response scenario the response message might not even have a process identifier. This can make it difficult for the BAM to create that holistic view and show you everything you want to see.

Fortunately, both of these issues can be addressed very effectively by the ESB. When business services are exposed on an ESB, the bus is handling all the messages as they flow between services. These messages and their orchestrated flow between services define a large part of the process. The ESB itself can event out these messages to the BAM, there is no need for invasive engineering - so collection is free. The ESB is already doing correlation, internally it knows what process instances it is dealing with, so it can add that information to the messages it sends to the BAM, together with a whole bunch of other useful stuff like timestamps and source and destination services.

Sometimes of course, things happen inside of applications that are invisible to the bus, and may be difficult to collect, but that are important to know. Again the ESB can help by exposing its BAM collection function as just another service that applications can invoke. With the Cape Clear ESB we're enabling effective BAM. We're doing a whole bunch of other great stuff to make BAM easier too, I'll post more on that soon.

In the meantime, how about a new TLA: SAM - SOA Activity Monitoring :-)

Landed at the Cape

Sat, 06 May 2006 20:35:11 GMT

I've wanted to get into the SOA and Enterprise Service Bus space for some time now. When an opportunity arose to join Cape Clear Software, the leading ESB vendor, I was excited by the prospect and impressed by the people I met. From a technical perspective, it fit nicely with my keen interest in Eclipse tooling on the one hand, server technology, and enterprise architecture in general. So pretty much the whole kit and kaboodle then! This is clearly a red hot space right now, and rightly so, SOA is changing the way we do so many things. Cape Clear have a stellar SOA platform that is ready to go out of the box, giving you tons of loosely coupled integration options that work first time. So I charted a course, and I've just landed at the Cape. One of the things I'm going to be helping Cape Clear do is to define extension points for BAM (Business Activity Monitoring). More on that later.