Mining the Event Processing Market?
by Arnold Reinhold
On March 14-16, 2006, IBM Research in Hawthorne, NY hosted an Event Processing Symposium, bringing together vendors and researchers in the business event processing field. As many presenters pointed out, event processing has been around a long time. The first event processing system I ever worked on was the Apollo Guidance Computer. Event driven systems are widely used throughout the solar system on NASA probes. Other traditional uses of event-based systems include military command and control, robotics, process control and SCADA systems.
So what’s new? Plenty. Just when IT thought their business stack of database, middleware and SOA tools covered all needs, along comes a group of vendors offering products for event processing that offer significant new capabilities. As one participant pointed out, there must be something to this or customers wouldn’t be spending large sums on these products. But is there a single market? There may be gold in them thar hills, however it’s far from clear that everyone is mining the same vein. Based on the discussion at the symposium, the event processing market can be segmented in the following way:
Event-driven business architecture. In this segment, business events have predictable, though possibly complex, consequences. A customer clicks a “place order” button in an on-line shopping cart screen, an item is scanned at a supermarket checkout line, or a passenger dips his credit card at a boarding pass kiosk. In the programming-in-the-large, service-oriented architecture, such events trigger a programmed cascade of business processes. Credit and inventory levels are checked; fulfillment options are evaluated; additional purchase opportunities are presented. This is the meat and potatoes of modern IT.
Temporal database queries. Several symposium participants commented that traditional relational database systems are poorly suited for dealing with temporal queries. At least one database vendor, Oracle, indicated there was much more real time capability under the hood. In the short term vendors may succeed with database add-ons but long term we expect database vendors to incorporate the necessary extensions. The market is split on programming. There seem to be three modes: using traditional scripting languages, such as PERL and Python, developing special languages for event processing, and extending SQL. The last is aimed at making the introduction of one more software category more palatable by making the programming methods seem familiar to heavy users of SQL.
Complex event processing (CEP). This segment looks for unusual combinations of events that signal business opportunities or problems. Think of this segment as real time data mining. Ease of programming is a priority. Large numbers of rules will have to be specified to find the situation patterns that matter. Ease of integration with the IT stack is also a requirement. The eventual goal is conscious information technology, information systems that are aware of their surroundings and can recognize the unexpected as well as the routine. Prompt awareness of competitive pricing moves, foreign exchange instability, or denial-of-service attacks can make a big difference in the bottom line.
High performance event processing (HPEP). In this segment milliseconds, even microseconds, count. If your computerized securities trading system detects a market pattern that suggests an arbitrage opportunity, gleaning a profit depends on being the first to place that order. If a competitor detects and acts on the same opportunity 10 milliseconds sooner, they win. In such situations the overhead associated with a traditional database two-phase commit may be unacceptable. The event processing system will likely require direct access to the event stream. Besides financial trading, classic high performance event processing applications include computer security, fraud detection and anti-terrorism. IT systems have only seconds to decide if a user may login, enter a restricted area, get cash from an ATM or board an aircraft.