BizTalk Orchestration Design – Choose Your Shapes Wisely

During my past BizTalk engagements, I have had the opportunity to work closely with my clients in developing flexible and maintainable applications. One of the most common issues that I come across is the misuse of some of the shapes available within the Orchestration Designer. By misuse, I simply mean to say that many BizTalk developers will drag and drop shapes into an orchestration to implement the business process, but do not take into account the implications of doing so. More often than not, the result of selecting the wrong shape for the job is not seen until the application is tested or even worse, the production environment. One of the primary reasons that BizTalk is used is the opportunity to build loosely coupled, flexible and scalable applications. By choosing the wrong shape, many developers will wind up doing exactly the opposite, thus setting the application down the course of tight coupling and brittle implementation. In this post, I am going to highlight two of the most often misused shapes, Call Orchestration and Start Orchestration, explain when they should be used and provide an alternate technique to overcome their shortcomings.

Call Orchestration

The Call Orchestration shape is used to invoke another orchestration synchronously. At first inspection, this shape enables the business process to be broken into small components and re-used across multiple orchestrations. The shape also allows users to pass multiple types of parameters into an orchestration much like a typical C# method call and even allows parameters to return to the calling orchestration. Depending on the requirements, all of these features may be exactly what you are looking for. However, there are a few points that need to be considered to ensure the right choice has been made. Since the calling orchestration invokes the other orchestration synchronously, it is important to understand that the calling orchestration will halt processing as it waits for the synchronous response from the called orchestration. The other point that may not be so obvious is that the called orchestration will execute on the same thread as the calling orchestration – even if the two orchestrations are configured to execute in separate hosts on separate threads. The BizTalk Runtime will ignore the configuration because the invocation is happening in-memory. Another side-effect that may not be as obvious is the fact that in order for one orchestration to call another orchestration directly, the parent orchestration has to have explicit knowledge of the child orchestration. Looking forward a few months, if the child orchestration logic needs to be modified, the parent orchestration would also need to be recompiled and redeployed. As a result of this tight coupling, the orchestrations will also need to be enlisted, un-enlisted, started and stopped in a specific order since the orchestrations now have a specific dependency on one another. If the two orchestrations will live closely in projects and will always be deployed and versioned together, this may not be an issue. However, if the intent is to have two or more separate business processes evolving on different cycles and operating on separate threads, the Call Orchestration is not the shape to use.

Start Orchestration

The Start Orchestration shape is used to invoke another orchestration asynchronously. The primary difference between this shape and the Call Orchestration shape is that the calling orchestration will not wait for a response from the called orchestration. Just like the Call Orchestration, it allows the user to pass multiple types of parameters into the called orchestration, but the Start Orchestration shape cannot be configured to accept return parameters. If this functionality is needed, the child orchestration must be explicitly designed to return a message and the parent orchestration must have the appropriate mechanisms in place to receive the message. Unlike the Call Orchestration shape, the parent orchestration invokes the child orchestration by way of the BizTalk MessageBox which allows us to configure the parent and child in separate hosts. In terms of performance, this invocation through the MessageBox does introduce some performance overhead, but if scalability is more important than speed, the overhead is an acceptable trade-off. Unfortunately, just like the Call Orchestration shape, the parent orchestration must have explicit knowledge of the child orchestration, thus presenting the same tight coupling issue. While Start Orchestration provides the benefit of improving scalability, it still presents the issue of tight coupling.

MessageBox Direct Bound Ports

In order to achieve a truly decoupled set of orchestrations, each orchestration must be designed without having knowledge of any other orchestration. In order to use either the Start or Call Orchestration shapes, we must know exactly which orchestration to invoke when the shape is executed. At some point after the initial design, if the parent orchestration no longer needed to call the child orchestration, but needed to call some other orchestration, the parent orchestration would need to be re-written, reconfigured and redeployed. If I have learned one thing from 10 years of consulting, it is that business processes will change. What seems concrete today may be irrelevant 6 months from now. Prior to designing any orchestration, it is imperative that the business process to be implemented is documented completely and examined for discreet units of work. During the design phase, our goal should not be to lay out a monolithic orchestration that encompasses the entire business process, but a set of small, discrete units of work that we can compose or orchestrate into higher level workflows. Even though it is difficult to plan for all business process changes, we can use a different design technique to minimize or possibly eliminate the ripple effect a business process change would cause to the application.

Rather than configuring an orchestration to directly call another orchestration, I highly recommend harnessing the power of the publish-subscribe behavior that is the foundation of BizTalk Server. By publishing messages directly to the MessageBox (direct bound), we have the opportunity to let the BizTalk runtime evaluate the message and dynamically determine the next process or processes that need to be invoked. Orchestrations and Send Ports have the ability to subscribe to the MessageBox which means they can listen for messages that match a predefined set of data. Within the context of BizTalk, this set of data is known as a filter. An example of a filter is the namespace of an XML message, a particular value that exists in one of the nodes of that message or one of the many properties that exist for BizTalk adapters. If a message is published to the MessageBox and a send port filter matches the properties of the message, the message will be removed from the MessageBox and the send port will receive the message for processing. As you can probably guess, the complexity of using this technique is using a consistent and unique pattern for filters to ensure that the message published to the MessageBox does in fact reach the intended recipient. Although this application design takes more forethought and planning, the flexibility and scalability that is gained by decoupling the orchestrations will pay dividends during testing phase and will save you from re-designing the application when it comes time to scale the BizTalk application onto multiple BizTalk hosts on multiple machines.

Another aspect to consider with direct binding is how we might chain several orchestrations together now that we have successfully broken them apart. For example, let’s assume that we have performed our due diligence and we know that the order of execution needs to be Orchestration A followed by Orchestration B and each orchestration is expecting the same canonical message. We clearly cannot filter on Xml namespace since any of the orchestrations could receive a copy of the message in an unknown order and create an infinite loop. To get around this limitation, we could add a header section to the canonical message or define a context property on the message called “NextStep” and set filters on each of the orchestrations to listen for the value of “NextStep” that matches their corresponding orchestration name. Within each orchestration, we could then invoke a Business Rule Policy that simply evaluates the current orchestration name and sets the “NextStep” property. In our case, during the execution of Orchestration A, the Business Rule Policy would set the value of “NextStep” to “Orchestration B” before the message is published to the MessageBox. At this point the subscription for Orchestration B would be matched and Orchestration B would receive the message. Taking the example one step further, what would happen if we needed to modify the business process so that Orchestration A called a new orchestration, Orchestration D, prior to calling Orchestration B? Rather than having to recompile any of the existing orchestrations, we can simply modify the Business Rule Policy to have the rule for Orchestration A set “NextStep” to Orchestration D and have the rule for Orchestration D set “NextStep” to Orchestration B. Since the Business Rule Policy is an artifact outside of the orchestration, we can make modifications to the high-level workflow without affecting the individual business processes in the orchestrations. Additionally, since our overall process is now separated into smaller units, if it is required, we could institute a resubmit-retry pattern in the middle of the business process (Orchestration D) without re-executing orchestrations that have already successfully completed (Orchestration A).

Conclusion

Overall, I am not suggesting that the Start or Call Orchestration shapes be avoided altogether, but I do want to make clear that their behavior and side-effects should be evaluated for the scenario in which they are used. As companies begin to embrace concepts such as Service Oriented Architecture and Enterprise Service Bus, it is imperative that middleware services are designed with flexibility in mind, regardless of the current-state business process. BizTalk Server was designed around the concept of flexible messaging and when we couple thorough business process analysis and sound architectural principles, we can create a change resistant set of business processes that are reusable across the enterprise.

Additional Resources

How to Configure the Call Orchestration Shape
http://msdn.microsoft.com/en-us/library/aa560780(BTS.20).aspx

How to Configure the Start Orchestration Shape
http://msdn.microsoft.com/en-us/library/aa577574(BTS.20).aspx

How to Use MessageBox Direct Bound Ports
http://msdn.microsoft.com/en-us/library/aa949234(BTS.20).aspx

This entry was posted in System Integration and tagged , , , , . Bookmark the permalink.