CEP Moving Into the Mainstream
March 3, 2008
Complex event processing (CEP) technology is powering a new generation of applications that enable companies to make faster, better decisions and execute in a more timely and effective manner. CEP software analyzes rapidly changing data about markets, customers, transactions and operations in real time to drive "in the moment" actions.
Event processing technology incorporates a high-performance, event-driven architecture that continuously analyzes "data in motion"--market data, trade data, application data and more--with historical and reference information. The continuous queries of a CEP engine can send analytics results from risk and strategy models, "action signals" to business users, or can tell other applications to take an automated action, as in algorithmic trading.
To date, companies have had two choices for real-time business analysis: attempt to speed up traditional business intelligence (BI) or custom-code their own event-driven architectures. Speeding up BI still leaves too large a latency gap, while custom-coding is costly and creates a brittle architecture that is often difficult to enhance and maintain.
The investment services market has its own unique requirements that exasperate the problems of real-time analysis:
* Ever-growing volumes and speed of market data. Peak data rates from equities and options markets surged to over 400,000 messages per second in 2007, and the Options Price Reporting Authority has told firms to prepare for nearly 1 million messages per second by July 2008. Trading applications need to keep up with these high data and processing rates.
* Submillisecond latency. Gaining an edge in trading now requires applications that can identify opportunities or risk in less than a millisecond and drive automated actions.
* Analyses and applications continuously in flux. The constant addition of new strategies, risk models and markets requires an application architecture than can be continually adjusted to support the needs of the business.
CEP emerged as a new option for firms in 2006. After "kicking the tires," a number of firms began deploying their first set of applications last year, and in the process learned their first lessons. With this experience behind us, 2008 is poised to become the year of mainstream CEP adoption.
Much of the early attention was centered on algorithmic trading--an obvious place where CEP technology could add value. However, three interesting, under-the-radar developments have occurred:
* CEP expanded well beyond algorithms with adoption in a much broader array of financial trading applications.
* Smaller, nimble firms recognized the competitive edge and financial benefit CEP could offer, and jumped on the bandwagon.
* Other sectors such as energy and commodities trading translated the benefits of CEP to their markets and began adoption.
One reason for the broader array of applications is the rapid development and deployment environments that CEP offers. Tight, agile development teams of two to four developers are creating powerful new applications. These teams take on an agile development approach, continuously adding more functionality to the applications and delivering new value to the business.
Many smaller firms are using event processing to build new trading backbones in record time and enabling them to reach new markets faster than their competitors. Examples include:
* Private trading networks using CEP to monitor illicit behavior by traders and capture best-execution analytics;
* Buy-side firms and hedge funds dynamically pricing the value of their instruments; and
