by Judith Hurwitz, President
There is a striking similarity between almost all of the companies that are focused on any sort of distributed computing platform – they all have some sort of metadata environment that underlies their platform. It is really not a startling revelation. If it is necessary to bring pieces from many different sources together and have them act as though they have a common context, there has to be a metadata layer. Every SOA development and deployment environment includes a metadata layer.
Metadata is the information about what the data within a framework means. There are two types of metadata that are important for SOA: technical and business metadata. Technical metadata includes information about the structure of the data itself and what it means in the context of its use. Business level metadata includes rules, governance, and processes related to the information.
You can think of metadata as though it were the human brain that includes a combination of functional areas that are linked together. While each part of the brain controls a specific function, the parts are linked together and need to communicate messages to each other so a person can function. The brain doesn’t know ahead of time what function it will be asked to undertake. In essence, for a person to function in the world, the brain needs to understand the context in order to interpret the rules of conduct – human metadata.
The same issue applies to metadata in a SOA environment. The interesting issue is if an organization has more than one SOA environment, how do you reconcile different metadata stores? Without this level of understanding, it will be very difficult for organizations to link various services and applications together. This is especially important if the organization does not want to resort to hand coding the pieces of a SOA environment. After all, if one is required to hand code to get the components to work together correctly; it is no longer a loosely-coupled SOA environment.
What makes the whole area of metadata complicated is that there are many types of metadata that must be taken into consideration. For example, database management systems have their own metadata as do information portals and content management systems. Each development environment and SOA framework has its own metadata layer. Each existing applications that might be used in a SOA environment has metadata as well. The bottom line is that you cannot use a specific service or component unless you understand the semantics of the various services and what tasks they are designed to perform.
This is especially important if the organization wants to have the component pieces create an accurate version of the truth. It is a simple fact that when organizations need to integrate various components into composite applications, there needs to be the ability to achieve a common view. Who is going to decide which of the contending data is correct if you have several different sources. Today, this is a manual process requiring the key players to look at this as a governance issue and make decisions about which version of the truth is accurate. If you have multiple touch points that have different types of customer information, there may not be one right answer. The way this is determined has to be with explicit rules that are defined by management to avoid problems with incorrect interpretation of what this new composite actually represents. It depends on context, timing, and a set of criterion. If, for example, you link three different components or services together they must have the right context. For example, each of these services will use various types of data to represent its structure. There are both technical metadata (about the structure of the information) and business metadata.
At the heart of metadata management is the need to exchange information across these highly distributed, loosely-coupled environments. This is necessary to achieve the holy grail of service-oriented integration: getting to a single view of the customer, the process, the product components, or any other collection of business components.
Clearly there is work to be done before we get to the point where it is easy to achieve this level of synergy. Today, organizations are successful in achieving metadata consistency in one environment or one set of application components that have been carefully linked together. To get to the level where it works like the metadata of the human brain will require some work. What are the key issues that have to be addressed? A lot of these issues will be addressed at the vendor level. Rather than trying to convince customers to focus on one implementation of a metadata framework, there needs to be comprehensive work to create XML-based metadata standards that can reconcile differences between approaches. This will require work on standard approaches to registry/repositories as well as common ways to have agreements on data semantics.