Apache Axis2/C Performance Round #2
|
|
Introduction
In this benchmark we do not take into account complete feature set available with Axis2/C and Axis2/Java. Security, large attachments, reliability etc. are outside the scope of this test, although, we plan to include them in future benchmark tests. In real world scenarios it is not uncommon to have SOAP messages with transport-level security, SOAP message-level security and large attachements. However, this benchmark provides information in terms of comparing infrastructure as well as performance for SOAP-XML level serialization/deserialization. Additionally, the tests were driven using a single client machine using Apache Bench client driver. Apache Bench was run with concurrency set at 25. Using the same client driver instead of using custom test scripts makes the test environment more accurate. This won't simulate real world loads, as in reality, load on the server is caused by distributed client machines and payloads could vary from request to request. In future we plan to do tests using distributed clusters of clients using client driver software having distributed load capabilities.
Our aim of performing these benchmark test is to show how Axis2/C that is written with performance and embedding capabilities in mind, could be compared with the most prominent Java Web services engine today.
Applies To
| Axis2/C | 1.5.0 |
| Environment | Red Hat Enterprise Linux Server Release 5 |
Table of Contents
Test Configuration
In all configurations tracing, application/access logging, authentication is turned off. Axis2/C tests were performed with the keep alive both on and off.
For details on Axis2/Java configuration see Axis2/Java performance testing Round #2.
The server was running on Red Hat Enterprise Linux Server Release 5 on a 4-way Xeon 3.2Ghz server with 2GB RAM. The client was connected to the server via a dedicated 1Gb Ethernet switch. As the client driver, we used the Apache Bench tool shipped with the Apache HTTPD server.
Axis2/C services were run using the Axis2/C Apache2 module on Apache Http Server version 2.2.9 instance built with prefork MPM(Multi-Processing Modules), which is a non-threaded preforking MPM. When configuring Apahce2 we did not load unnecessary shared modules in order to minimize any over heads. Axis2/C is built with Guththila as the XML parser which is the default XML parser for Axiom starting from version 1.4.
The client driver was running on Red Hat Enterprise Linux Server Release 5 on a 4-way Xeon 3.2Ghz server with 2GB RAM.
Test
With this performance test, we intended to carry out the same performance testings as with Apache Axis2/Java performance testing Round #2. Hence, we decided to get the same benchmark WSDL and generated the service using Apache Axis2/C code generation tool with ADB (Axis Data Binding) and implemented the services similar to Java services. We also used the same hardware configurations Axis2/Java used with their performance test Round #2. We used Top to observe that CPU usage under peak throughput is about 80%.
These are the operations that were tested in the benchmark service.
|
echoVoid |
This simply reads a void and returns it |
|
echoInts |
Reads an incoming integer and returns the Integer |
|
echoDoubles |
Reads an incoming Double and returns another Double |
|
echoStrings |
Reads an incoming String and returns another String |
|
echoMeshInterfaceObjects |
Reads an incoming Object and returns another |
|
echoSimpleEvents |
Reads Complex type Object and returns another |
Some words about data used for testing is that we can basically define the data sets used into two groups, namely large and small. Large data sets consists of arrays of 100 elements from each data type in the table above. Small data sets consists of arrays of 1 element. To make data close to real world data, the WSDL introduces two complex data types named MeshInterfaceObjects and SimpleEvents. In order to minimize the overhead due to Web service processing the WSDL defines echo methods on all data types.
Results
When small data sets were used, Axis2/C performed much better than with Axis2/Java. For echoVoid it actually reached 25K requests per second limit. Axis2/C performed consistently more than 5 times greater than Axis2/Java. When considering the fact that Axis2/Java performs well ahead of competing Java Web services engines, it could be said that Axis2/C has achieved its performance goals in its 1.5 release.
As the following graph illustrates, in terms of large data sets, there is not much difference whether you make keep alive on or off. Axis2/C still performs well ahead of Axis2/Java (consistently more than two times).
Finally, the following graph illustrates that Axis2/C scales evenly from small data sets to large data sets. The message sizes vary from .3k to 8k. However, this shows that for large messages, Axis2/C does not scale well.
Conclusion
Axis2/C consistently performs much better than Axis2/Java and Java Web services engines in general. When considering the rich feature set of Axis2/C, tests elaborates and confirms, that Axis2/C is the ideal solutions when it comes to performance hungry web service applications that also rely on rich features that could be expected from a modern web services engine.
Resources
- Use these artifacts to reproduce the test environment and verify results yourself.
- Axis2/C is an Apache Web Services Platform implemented in C language around which the WSO2 WSF/C is built on.
Author
Damitha Kumarage, Senior Software Engineer at wso2, committer Apache Software Foundation, damitha at wso2 dot com
