Architect Data Services
for ESB
If your services share interdependent information, you need both a data services architecture and an ESB to ensure consistent data.
by Christopher Keene
March 14, 2005
Service-oriented architecture (SOA) provides a way for companies to assemble applications out of free-standing units of functional code. As long as these services each have their own, nonoverlapping data, a messaging layer such as an enterprise service bus (ESB) is sufficient. However, when services share interdependent underlying data—such as inventory or customer data—a data services architecture (DSA) is required in addition to the ESB to ensure that each service is working with consistent information.
For example, a company might have a Web portal that integrates services to perform order entry, check order status, and update order status. The implementation of these individual services might be provided by a packaged application such as SAP or by custom applications written in Java or C#. To function properly, however, each of these three services must agree on inventory availability and customer account information.
The role of a DSA is to guarantee that data used by custom-written application services is always up-to-date with the information managed by packaged applications such as SAP. This capability is complementary to the ESB—an ESB is concerned with application integration and workflows between services, whereas a DSA is concerned with data integration and transactions occurring within a service.
This article describes best-practice data architectures that enable companies to extend the ESB architecture beyond application integration and provide a framework for developing new custom applications. A DSA abstracts data from both traditional databases and existing packaged or legacy applications and uses data synchronization to ensure that data used by custom applications is always up-to-date (see Figure 1).
Operational Data Consistency
Gartner defines an ESB as a messaging infrastructure that connects and unifies interactions between services in an SOA. In other words, the ESB enables "plug and play" communications between SOA services. Yet an ESB by itself does nothing to ensure that data interdependencies between services are handled properly.
A traditional application is usually deployed as a "silo," meaning the application has its own database that contains a copy of operational business data needed by that application such as customers, products, and inventory levels. Typically, each data silo gets synchronized only periodically, so between synchronizations each silo has slightly different data. These data inconsistencies can cause business errors when silo applications are integrated using an ESB.
When silo applications are turned into services, each with different inventory data, you encounter problems (see Figure 2). In this example, the "show status" service thinks that the inventory level is 27, while the "check_avail" service thinks that the inventory level is 0.
Given the business risks associated with inconsistent data across multiple data silos, the first step for an IT department implementing an SOA is to ensure data consistency. Developers can’t build custom, service-oriented applications without first implementing a DSA that guarantees data consistency across all services.
Ad Hoc Data Consistency
The classic approach to distributed data management in the world of silo architectures is to extract reference data into a dedicated operational data store for each business application. This approach depends on little interaction between silos, so out-of-date data within a particular silo is unlikely to cause problems. You need a DSA to ensure that each of the services being integrated into a common workflow by the ESB is working with up-to-date information (see Tables A, 1, and 2).
You must specifically design the DSA to provide consistent, high-performing, and highly available data across an SOA. The ESB provides a functional abstraction layer for applications and data, while the DSA provides a data abstraction for both applications and data (see Figure 3). So, an application such as SAP looks like a set of function calls to the ESB, and like a set of data definitions (transformed according to a metamodel) to the DSA.
Back to top
|