Welcome!

Open Source Cloud Authors: Vaibhaw Pandey, Stackify Blog, John Walsh, Liz McMillan, Pat Romanski

Related Topics: Government Cloud, Java IoT, Microservices Expo, Open Source Cloud

Government Cloud: Article

Showcasing the Key Design Principles of SOA

A Case Study: Orchestrating Services Using Open Source BPEL

SOA & WOA Magazine on Ulitzer

WS-BPEL 2.0 is the dominant specification to standardize orchestration logic and process automation between Web services. The BPEL model is used to assemble a set of discrete, essentially disparate, services into an end-to-end process flow to transform the existing stateless and uncorrelated Web service infrastructure into cohesive, process-centric applications based on a service-oriented architecture.

Under this model, new applications are developed "on-the-fly" by wiring together external "partner" services that leverage existing enterprise assets. JBoss's jBPM product is an open source platform for BPEL as well as other graph-based execution languages and provides the runtime infrastructure: interpretion, extraction/WSDL document publishing, flow execution, analytics, transactional and persistence support for the deployed service composition model. A use case example demonstrates a simplified process domain comprising of a composite service that consists of two partner relationships: JAX-RPC synchronous and asynchronous service endpoints, respectively.

Web services represent the building blocks of a service-oriented architecture and use a loosely coupled, interoperable, reusable and discoverable model to allow for flexible integration of heterogeneous systems in a variety of domains: business-to-consumer, business-to-business and enterprise application integration.

With the maturity of the JCP specifications (popularly JAX-RPC, JAX-WS and JAX-RS) and various reference implementations and tools supporting the individual technology stacks (WS-* or REST), it is relatively easy to create single service endpoints [1] (or as in the case of REST, Web resources [2]), each encapsulating a unique functionality and have it invoked by one or more clients. However, for more complex systems that require coordinating between multiple services, for example, system integration efforts connecting home-grown legacy systems to ERP and other enterprise systems for fulfillment of a unique business goal require more than just the ability to conduct simple interactions by using standard protocols and the stateless WSDL-supported model of request-response or uncorrelated one-way interactions. What is required is a way for applications and business processes to integrate their complex interactions by using a standard process integration model. The real promise of SOA is not realized by building and consuming disparate services alone but by defining clear cohesive services that wholly describe their interactions that are then capable of being reused, through composition, in a variety of process-based services. Therefore, what you really need in such cases is a way to conveniently express the flow in business semantics, something that sufficiently abstracts away the individual services within a process context, executed on the server; and ultimately exposes a WSDL that can then be called on by a Web service client. In the world of SOA this is called an orchestration and often executed using BPEL, or Business Process Execution Language [3].

About BPEL
WS-BPEL 2.0, the latest underlying OASIS standard around which most current BPEL implementations are based, leverages Web services as the model for process decomposition and defines an XML-based "model and a grammar for describing the behavior of a business process based on interactions between the process and its partners. The interaction with each partner occurs through Web service interfaces" [4]. In essence, WS-BPEL defines how interactions with these partners are orchestrated (coordinated) to achieve a business goal.

A deployed BPEL application represents a new and complex composite service that is composed of basic services and the various interactions/message exchanges between them. BPEL defines an executable language for specifying interactions with these basic Web services that support some of the commonly used features of a WS-* process language construct: XML and WSDL typed variables, property-based message correlations, expressions and queries using XPath, transformations using XSLT, fault/event handlers along with structured logic constructs and flow directives and support for robust workflow features such as parallel execution, flow sequencing and chaining

Common representations are in an XML-formatted document with a .bpel extension that conforms to the WS-BPEL standard and executes within the BPEL engine environment, which provides the interpretion, execution, analytics, transactional support, and persistence store for the deployed model service composition. The interactions with the external entities are abstract in the sense that the dependence is on port type operations and message definitions represented via XML schemas, thereby abstracting business logic into flow directives, which encourages high cohesion and loose coupling between the service implementations.

About jBPM-BPEL
JBoss/RedHat's jBPM is an open source platform for graph-based execution languages [5]. Its pluggable architecture makes it possible to support different workflow languages (mainly jPDL and BPEL) that can be modeled as some kind of abstract flow directive. jPDL (Java Process Definition Language) is jBPM's native workflow language and is similar to BPEL in as much as it was created to model and assemble a discrete set of processes into an end-to-end process flow. This is where the similarity ends, with the main difference being that in jPDL the variable context for the process data is made up of Java POJOs whereas BPEL processes use XML constructs. BPEL is the focus of this article so I'll not delve further on the differences between the two process technology stacks. However, it's worthwhile to note that BPEL is more widely adopted and supported. The main reason being, most business processes involve frequent interactions with heterogeneous systems and partners with interoperability/platform independence being essential. Factor in XML as the populist (efficient, manipulable, interoperable and developer-friendly) message exchange mechanism along with QoS demands for an increasingly wired world and now you know why.

jBPM is built upon a persistence model with Hibernate as its default object-relational mapping solution [6]. This means that installation of the jBPM-BPEL software involves deploying an enterprise archive (.EAR) component (containing required libraries and configuration files) along with configuring a Data Source (pointing to a jBPM database) within the application server. Once successfully deployed, the jBPM-BPEL runtime is ready to extract new process definitions, create process service endpoints, publish WSDL contracts and orchestrate the composite service all of whose state is persisted within the jBPM database.

An Example Process Domain
Figure 1 shows a highly simplified Process domain comprising of a composite service that consists of two partner services: a POJO synchronous service endpoint represented by Forms and a POJO asynchronous endpoint represented by Imaging. In actuality, the partners represent our internal Forms Management and Imaging systems, respectively, whose interactions can be captured in a BPEL workflow and deployed to a jBPM-BPEL runtime engine to collectively work in rendering PDF documents that are then archived away as image files.

The Forms system consists of a JAX-RPC document-style POJO endpoint implemented by JBoss's very own JAX-RPC SOAP stack, the JBossWS framework. The Service Endpoint Interface (SEI) and implementation bean for this service are shown in Listing 1. (Listings 1 - 10 can be downloaded here.) JAX-RPC endpoints leverage the slightly older J2EE 1.4 programming model, which requires packaging of a number of deployable descriptors for the Web service [7] (newer specifications like JAX-WS does away with most descriptors by providing all endpoint associated metadata via annotations [8] that perform compile/runtime dependency injections, thereby greatly simplying the development model). However, WS-BPEL is not supported by JAX-WS, therefore we are limited to using JAX-RPC endpoints for our external service implementations for use by the BPEL composite process.

A number of deployment descriptors need to be created prior to packaging this endpoint and bean implementation (see Listing 1); the webservices.xml descriptor that identifies the deployment of a Web service endpoint, the abstract service WSDL contract and the jaxrpc-mapping.xml descriptor that maps the WSDL document schema elements to Java classes. JBossWS comes with a handy wstools utility that creates all the aforementioned artifacts either from the command line or at build time. The POJO endpoint is packaged as a Web application in a WAR file and is deployed as a servlet accessible via a URL pattern. Once the services are packaged this way and deployed to a JBossWS compliant application server, they are then visible via the Web service console at http://<machine>:8080/jbossws/services.

The Imaging system upon receiving the PDF rendered form (represented as binary stream data) will initiate the image archival process (internal details of which are best left out as they are beyond the scope of this article), an asynchronous component, which finally returns a response consisting of an image index and a system message on a JMS message queue. Other than this, development and packaging of the Imaging service endpoint (ImagingPT) is very similar to what was done earlier for the FormsPT SEI and Service implementation.

Once the partner or external system endpoints have been developed and deployed, we need to let jBPM-BPEL know about its existence. This is typically done through the Partner Sevices Catalog available under the jBPM-BPEL console (http://<machine>:8080/jbpm-bpel/partners.jsp) (refer to Table 1).

A process scenario would have the central orchestrating service call the Forms system with a set of parameters (form design number, form field data etc.) to render a pre-populated PDF form. Next, it would send the PDF form (represented as a byte array data) to the Imaging system to initiate the image archival process. The imaging system responds asynchronously on a JMS queue with the image index and a system message that is then picked up by a MessageDrivenBean (deployed along with the Imaging endpoint) listening on the same queue, which calls back into the orchestrating service, triggered upon receipt of the javax.jms.Message. The service then bundles all the return values into an invoice which is sent back to the calling client of the process.

BPEL Process Development Methodology
There are two main options for designing a business process: Top-Down or Bottom-Up [9]. Both methods lend themselves to a design approach using a visual BPEL process designer of which there are quite a few open source options.

The Top-Down approach is as its name suggests. Essentially, you model all the intermediary (partner) components that make up the composite service. This may involve interviewing business stakeholders and analysts to come up with the ideal service parameter definitions and return values. You basically would want to view the services and compositions from the invoker's point of view, which would necessitate keeping the service granularity at the appropriate level of abstraction. Once done, you can start adding technical details to the process such as port types, schemas, partner links, variables, scopes, sequences, conditional logic, error handling and other task activities to build out an orchestration of the composite service. This design method suits those enterprises that need to develop new partner services and processes from scratch and is what we used.

Alternately, using a Bottom-Up method involves using pre-existing services and the WSDL documents available within your environment to create out the orchestrated business processes wherin you would add the technical details to the process exactly the same way as in the Top-Down approach.

Creating the BPEL Process Definition Document
The BPEL process definition document models the entire workflow in XML, which upon deployment is interpreted and translated by the jBPM-BPEL runtime engine to provide the runtime infrastructure as a centralized orchestrating composite service with its own endpoint and service WSDL contract. This section goes over the basic structure and content of the BPEL document for the example process domain as illustrated in Figure 1.

a. Creating Partner Links and Defining Related Port Types
The <process> element is the root element of all BPEL process definition documents. It declares all the namespaces used within the process (including that of itself) and imports the namespaces for all external or partner services. The namespace declaration is http://docs.oasis-open.org/wsbpel/2.0/process/executable, which is the standard root namespace for all BPEL 2.0 processes. The <import> element is used to declare a dependancy on either an XML schema or the WSDL contract of an external or partner service that must exist in the same relative path (attribute "location") as the central process document and qualify all imported definitions with the namespace attribute. In our example described earlier, the process imports a WSDL file that defines the partner links that it uses and is available in the same directory as the BPEL file along with two other separate WSDL files, for the previously deployed Forms and Imaging endpoints (refer to Listing 2 - a section of the BPEL process document).

Partner links are made available within a BPEL document by adding the namespace of "xmlns:plnk=http://docs.oasis-open.org/wsbpel/2.0/plnktype" and can be thought of as analogous to business partners; they define the different external entities that are consulted during the execution of the business process and define the relationship between the Web services involved in the process fulfillment. The partner link represents an interaction between the central orchestrating service and the external service by indicating the port types used in the relationship (see Listing 3) and is characterized by a partner link type that defines the roles played by each of the services in the context of the conversation. Each partner link type defines essentially two roles: myRole, which indicates the role of the process itself in the relationship, and partnerRole, which indicates the role of the partner. The process document snippet in Listing 2 defines three partner link types (one for the orchestrating service workflowLT and one each for the external services that are invoked formsLT and imagingLT) described by their respective port types in the process WSDL document as shown in Listing 3. Note that the port types, which are an abstraction of the service endpoint interface (SEI) (frm:FormsPT and img:ImagingPT, respectively), are defined in the Forms and Imaging service WSDL definition files that were imported previously into the process context. Therefore, the formsLT partner link type is a conversational relationship, initiated by the process, with the Forms SEI that is defined by port type frm:FormsPT. The workflowLT is initiated by an external client that calls the process whose conversation involves the SEI for the process (defined as pos:workflowPT within the context of the process WSDL definition and shown in Listing 4). The imagingLT has two roles: one that is initiated by the process calling into the Imaging endpoint SEI that is defined by the port type img:ImagingPT and the other is a callback into the process initiated by the asynchronous Imaging service. How that callback exactly occurs is detailed in a later section. It's important to point out that upon deployment of this BPEL process document, each myRole definition on the partner link results in publication of as many port types all served by the same central composite service. Please refer to Listing 5 for a snippet of the composite service's WSDL document created by jBPM-BPEL upon deployment and available under http://<machine>:8080/jbossws/services.

More Stories By Rizwan Ahmed

Rizwan Ahmed is an IT Systems Architect and author. He has about 10 years of experience in the public and private sector architecting technology, systems and security solutions. He holds Bachelor's and Master's degrees from the Indian Institute of Technology and the Florida State University, respectively. Passionate about open source as contributor and evangelist, he is also a frequent speaker at Java user groups, conferences and sponsored workshops. He holds the Sun Certified Java Programmer, Certified Information Systems Security Professional (CISSP), Certified Secure Software Lifecycle Professional (CSSLP) and the Project Management Professional (PMP) credentials. He lives with his wife and two kids in Columbia, SC.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


@ThingsExpo Stories
"Cloud Academy is an enterprise training platform for the cloud, specifically public clouds. We offer guided learning experiences on AWS, Azure, Google Cloud and all the surrounding methodologies and technologies that you need to know and your teams need to know in order to leverage the full benefits of the cloud," explained Alex Brower, VP of Marketing at Cloud Academy, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clar...
Large industrial manufacturing organizations are adopting the agile principles of cloud software companies. The industrial manufacturing development process has not scaled over time. Now that design CAD teams are geographically distributed, centralizing their work is key. With large multi-gigabyte projects, outdated tools have stifled industrial team agility, time-to-market milestones, and impacted P&L stakeholders.
Gemini is Yahoo’s native and search advertising platform. To ensure the quality of a complex distributed system that spans multiple products and components and across various desktop websites and mobile app and web experiences – both Yahoo owned and operated and third-party syndication (supply), with complex interaction with more than a billion users and numerous advertisers globally (demand) – it becomes imperative to automate a set of end-to-end tests 24x7 to detect bugs and regression. In th...
"Akvelon is a software development company and we also provide consultancy services to folks who are looking to scale or accelerate their engineering roadmaps," explained Jeremiah Mothersell, Marketing Manager at Akvelon, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
"MobiDev is a software development company and we do complex, custom software development for everybody from entrepreneurs to large enterprises," explained Alan Winters, U.S. Head of Business Development at MobiDev, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
SYS-CON Events announced today that CrowdReviews.com has been named “Media Sponsor” of SYS-CON's 22nd International Cloud Expo, which will take place on June 5–7, 2018, at the Javits Center in New York City, NY. CrowdReviews.com is a transparent online platform for determining which products and services are the best based on the opinion of the crowd. The crowd consists of Internet users that have experienced products and services first-hand and have an interest in letting other potential buye...
"IBM is really all in on blockchain. We take a look at sort of the history of blockchain ledger technologies. It started out with bitcoin, Ethereum, and IBM evaluated these particular blockchain technologies and found they were anonymous and permissionless and that many companies were looking for permissioned blockchain," stated René Bostic, Technical VP of the IBM Cloud Unit in North America, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Conventi...
SYS-CON Events announced today that Telecom Reseller has been named “Media Sponsor” of SYS-CON's 22nd International Cloud Expo, which will take place on June 5-7, 2018, at the Javits Center in New York, NY. Telecom Reseller reports on Unified Communications, UCaaS, BPaaS for enterprise and SMBs. They report extensively on both customer premises based solutions such as IP-PBX as well as cloud based and hosted platforms.
"Space Monkey by Vivent Smart Home is a product that is a distributed cloud-based edge storage network. Vivent Smart Home, our parent company, is a smart home provider that places a lot of hard drives across homes in North America," explained JT Olds, Director of Engineering, and Brandon Crowfeather, Product Manager, at Vivint Smart Home, in this SYS-CON.tv interview at @ThingsExpo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
Coca-Cola’s Google powered digital signage system lays the groundwork for a more valuable connection between Coke and its customers. Digital signs pair software with high-resolution displays so that a message can be changed instantly based on what the operator wants to communicate or sell. In their Day 3 Keynote at 21st Cloud Expo, Greg Chambers, Global Group Director, Digital Innovation, Coca-Cola, and Vidya Nagarajan, a Senior Product Manager at Google, discussed how from store operations and ...
In his session at 21st Cloud Expo, Carl J. Levine, Senior Technical Evangelist for NS1, will objectively discuss how DNS is used to solve Digital Transformation challenges in large SaaS applications, CDNs, AdTech platforms, and other demanding use cases. Carl J. Levine is the Senior Technical Evangelist for NS1. A veteran of the Internet Infrastructure space, he has over a decade of experience with startups, networking protocols and Internet infrastructure, combined with the unique ability to it...
It is of utmost importance for the future success of WebRTC to ensure that interoperability is operational between web browsers and any WebRTC-compliant client. To be guaranteed as operational and effective, interoperability must be tested extensively by establishing WebRTC data and media connections between different web browsers running on different devices and operating systems. In his session at WebRTC Summit at @ThingsExpo, Dr. Alex Gouaillard, CEO and Founder of CoSMo Software, presented ...
"There's plenty of bandwidth out there but it's never in the right place. So what Cedexis does is uses data to work out the best pathways to get data from the origin to the person who wants to get it," explained Simon Jones, Evangelist and Head of Marketing at Cedexis, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
WebRTC is great technology to build your own communication tools. It will be even more exciting experience it with advanced devices, such as a 360 Camera, 360 microphone, and a depth sensor camera. In his session at @ThingsExpo, Masashi Ganeko, a manager at INFOCOM Corporation, introduced two experimental projects from his team and what they learned from them. "Shotoku Tamago" uses the robot audition software HARK to track speakers in 360 video of a remote party. "Virtual Teleport" uses a multip...
A strange thing is happening along the way to the Internet of Things, namely far too many devices to work with and manage. It has become clear that we'll need much higher efficiency user experiences that can allow us to more easily and scalably work with the thousands of devices that will soon be in each of our lives. Enter the conversational interface revolution, combining bots we can literally talk with, gesture to, and even direct with our thoughts, with embedded artificial intelligence, whic...
SYS-CON Events announced today that Evatronix will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Evatronix SA offers comprehensive solutions in the design and implementation of electronic systems, in CAD / CAM deployment, and also is a designer and manufacturer of advanced 3D scanners for professional applications.
Leading companies, from the Global Fortune 500 to the smallest companies, are adopting hybrid cloud as the path to business advantage. Hybrid cloud depends on cloud services and on-premises infrastructure working in unison. Successful implementations require new levels of data mobility, enabled by an automated and seamless flow across on-premises and cloud resources. In his general session at 21st Cloud Expo, Greg Tevis, an IBM Storage Software Technical Strategist and Customer Solution Architec...
To get the most out of their data, successful companies are not focusing on queries and data lakes, they are actively integrating analytics into their operations with a data-first application development approach. Real-time adjustments to improve revenues, reduce costs, or mitigate risk rely on applications that minimize latency on a variety of data sources. In his session at @BigDataExpo, Jack Norris, Senior Vice President, Data and Applications at MapR Technologies, reviewed best practices to ...
An increasing number of companies are creating products that combine data with analytical capabilities. Running interactive queries on Big Data requires complex architectures to store and query data effectively, typically involving data streams, an choosing efficient file format/database and multiple independent systems that are tied together through custom-engineered pipelines. In his session at @BigDataExpo at @ThingsExpo, Tomer Levi, a senior software engineer at Intel’s Advanced Analytics gr...
When talking IoT we often focus on the devices, the sensors, the hardware itself. The new smart appliances, the new smart or self-driving cars (which are amalgamations of many ‘things’). When we are looking at the world of IoT, we should take a step back, look at the big picture. What value are these devices providing? IoT is not about the devices, it’s about the data consumed and generated. The devices are tools, mechanisms, conduits. In his session at Internet of Things at Cloud Expo | DXWor...