WSO2 Business Activity Monitor

WSO2 Business Activity Monitor (WSO2 BAM) is a tool designed to exercise Business Activity Monitoring (BAM). WSO2 BAM is intended to serve the needs of both business and IT domain experts to monitor and understand business activities within a SOA deployment. It is specifically designed for monitoring SOA deployments, and can be extended to cater for other general monitoring requirements as well.

Introduction

Business Activity Monitoring (BAM) is a term coined by Gartner. The term reflects the interests in the strategic concepts of the zero-latency enterprise and straight through processing. The gap between when the data are collected and when they are available for decision making is defined as latency in this context. Hence, zero latency means that data is immediately available. This enables organizations to be proactive rather than be reactive. Straight through processing is where inefficient steps like manual entry have been eliminated. WSO2 BAM supports both zero latency as well as straight through processing. Data is collected through push or pull models, automatically, and processed real time, to be made available for business and IT users.

To make more informed and better decisions, decision makers need the ability to analyze historical and current data and situations. Both business as well as IT domain experts want right information at the right time and in the right place to monitor and understand business activities. WSO2 BAM enhances the decision making process by collecting current data, summarizing those data and making information readily available to help with decision making. While business experts can use those information for fine tuning the business process, IT experts can use those information for optimizing IT operations, capacity planning/provisioning and fine-tuning deployment architectures.

Monitoring SOA Deployments

Users can detect unexpected patterns, and act in a proactive manner to deal with emerging situations and be agile with the enterprise. In a SOA setup, WSO2 BAM can be used to monitor the overall health of the SOA deployment. Given the way SOA bridges the business and IT gap, BAM used on a SOA deployment can result in an efficient organization, where business and IT objectives are aligned with each other. Services deployed with service hosting environments within a SOA, as well as enterprise service bus (ESB) mediation operations could be monitored with WSO2 BAM.

Activity modeling is the first step in creating a successful BAM system. This involves finding activities to be monitored, as well as the steps associated with activities and metrics tied to those steps. When BAM is done on a SOA based system, we can base our activity modeling on the SOA design itself. Given that the SOA must have been designed with the associated business model in mind, we can take the services and operations break down and the respective service invocations models, and map those directly to the business activities to be monitored. The service invocation model would be part and parcel of the ESB deployment design, in cases where ESB is deployed in a SOA. Hence, just by monitoring the services and the service invocation model, we can do a comprehensive business activity monitoring on a SOA deployment.

Tools for Monitoring

To help analyze data, both real time and historical, WSO2 BAM provides dashboards and associated reports, based on various predefined queries on BAM database. To meet the user demands for information, BAM provides:

For more information on monitoring tools, see visualization

Architecture Overview

WSO2 Business Activity Monitor encapsulates four main aspects to achieve business activity monitoring capabilities.

  1. Data collection
  2. Data persistence
  3. Data summarization and correlation
  4. Business intelligence with visualization tools, including dashboards and reporting

The following figure highlights the high level WSO2 BAM architecture.

WSO2 BAM Architecture Overview

Data Collection

To be able to monitor business activities, we need to collect data from various elements in a SOA. There are two models in collecting data.

  1. Pull model

    In the pull model, we periodically poll the servers for data. In the current implementation of WSO2 BAM, we support polling service data from WSO2 Web Services Application Server (WSAS). The advantage of the pull model is that, there is nothing specific to be done on the sever hosting services (in other words, the server being monitored). We can point to the statistics publishing service of WSO2 WSAS and pull data from that into WSO2 BAM server.

    The disadvantage of this model is that, the server which is the source of data must have populated the service statistics module, and if there are any changes to the service interface of the statistics publishing service interface, then WSO2 BAM would have to update its polling logic.

  2. Push model

    In the push model, data is collected through an eventing API. WSO2 BAM server populates an eventing API, though which both WSO2 products as well as non WSO2 products can populate service data into WSO2 BAM. The key advantage of this model is the loose coupling between the data publisher and the subscriber. Apart form the WS-Eventing based API, there need not be any other pre-requsites for publishing data.

    The disadvantage of this model is that, there will be some performance implications on the server being monitored, becuase, in the message flow, there has to be some handler logic to deal with data collecion, which adds to regular message processing.

Data Persistence

WSO2 BAM supports two main categories of data formats.

  1. Service data

    WSO2 BAM supports collecting service data at server level, service level and operation level. For each level, the following basic data elements are captured:
    • Request counts
    • Response counts
    • Fault counts
    • Average response time
    • Maximum response time
    • Minimum response time
  2. User defined data

    User defined data is persisted as key/value pairs. The user can defined the key set and store the values against those keys, along with a time stamp. Hence, over time, multiple values can be stored against the same key. WSO2 BAM supports persisting mediation data using the user defined data model. Mediation data is collected using the push (eventing) model. Data for the three levels, proxy services, endpoints and sequences are collected and stored as key/value pairs. For each level, the following data elements are captured.
    • Request counts
    • Fault counts
    • Average response time
    • Maximum response time
    • Minimum response time

    The collection of mediation data using the user defined data model serves as a powerful proof of concept for the ability of WSO2 BAM to be extended to capture any custom data model for business activity monitoring. For example, we do not have provision for capturing fault reasoning in the current WSO2 BAM implementation for mediation data. However, combining eventing data collection model with user defined data persistance model, we can pave the way to capture fault reasoning when the fault sequence is invoked.

Data Summarization and Correlation

Data collected from servers and persisted with the data models discussed in previous section are raw data that comes from the servers along the real time line. Over time, WSO2 BAM users would benefit from the ability to summarize data based on time dimensions. To provide analytics capabilities, both service data and mediation data are summarized into the following time dimensions:

In the current implementation of WSO2 BAM, we have provision for capturing and correlating message level service data. To achieve this, we use an Activity ID, that is inserted into each message that flows between a client and a server. Activity IDs correlate data at SOAP message level. Multiple messages, having the same Activity ID are deemed to be related and their data is tracked, and the invocations correlated. This feature is still experimental, and will be improved in future releases.

Business Intelligence

Once both raw data and the summarized data are available, the business users or IT users need to be able to visualize and analyze data to help with their decision making. WSO2 BAM comes with dashboards and reporting capabilities. The main dashboard has the capability to add/configure/remove gadgets. The product is shipped with some pre-designed gadgets. And the user has the option of adding new gadgets, based on user preferences. There are also four pre-defined dashboards that ships with the product, for real time service data, service data analytics based on summary data, real time mediation data and mediation data analytics based on summary data. WSO2 BAM also comes with the ability to generate reports. In the current release, there will be support to generate some static reports based on real time data and summary data. There are plans to support custom reports in the future.

Example Deployment

The following figure shows a sample deployment using WSO2 BAM to monitor WSO2 WSAS, WSO2 ESB as well as generic/custom data sources.

WSO2 BAM Example Deployment