Introduction

The WSO2 WSF-C QA Test Plan describes the strategy and plan for testing the upcoming WSO2 WSF/C 1.0 release.  This document will be the base for all the QA testing activities during this release process.

Scope and Test Coverage

WSO2 WSF/C is an Open Source framework released under Apache License v2.0, for providing and consuming Web services. It is one of the core implementations of WSO2 Web Services Framework (WSF).

WSO2 WSF/C is based on Apache Axis2/C, Apache Rampart/Cand Apache Sandesha2/C. The advantage of using WSO2 WSF/C is that it comes with all WS-* implementations of the Apache Axis2/C family of projects integrated together, tested and ready to be used. It also comes with an additional XMPP transport and a command line interface for consuming web services, named wsclient.

QA team is planning to do the following test types during the QA testing cycle of WSO2 WSF/C 1.0 release.

• Installation/Smoke testing
• Functional testing
• Regression testing
• Performance testing
• Interop Testing with Java/.Net and C services

Features list of the product for release 1.0

The features which will be available in the 1.0 release will be as follows.

• Installers - MSI, Deb, RPM, bin (zip and tar), src (zip and tar)

• axis2c/samples

• Tools

• Rampart

• Sandesha
• Savan

These will be discussed more under the section "Functional Testing"

Out of Scope

QA test process

 Please refer the WSO2 QA Process for the summarized QA process which we follow at WSO2. The following are the types of testing which we hope to follow in the upcoming release.

Smoke/Installation Testing

Once the build is given for testing, a smoke test is carried out to determine that each relevant component in the build is in a state where further testing can be carried out without any interruption. Further testing will be carried out on the build only if the smoke test results are passed. A smoke test comprises of the functional test cases which includes the critical functionalities of the product and will be executed in each QA build.

Installation testing is done by deploying and configuring the WSF-C on a particular environment as specified in the Installation Guide. The release artifacts such as  NOTICE.txt, LICENCE.txt, NEWS.txt, library files and dll files should be verified as well. In addition, a latest copy of the source is checked out from the relevant SVN location and is built according to the installation guide.

Installation testing will be suspended if one or more of the following is encountered;

• If the installation guide does not have the necessary steps to configure and set up the product
• Files needed for installation are not included in the release (eg:- header files are missing etc..)

Functional Testing

Functional tests are carried out to verify the functional requirements of the WSF-C project. The check lists/ test cases hosted at the SVN location https://www-lk.wso2.com/wso2/svn/proj/quality_assurance/c/wsf4c will be used to verify the identified functionalities of the project. The issues found while going through functional testing, are raised as JIRAs.

The following functionalities will be tested in the release.

•  Installer testing of the following

                - MSI

                - Deb

                - RPM

                - bin (zip and tar) [security, RM, eventing]

                - src (zip and tar) [enable Guththila, SSL, libcurl, security, RM, eventing]

• axis2c/samples

                - execute samples deployed on Apache

                - execute samples deployed on the Simple Axis Server

                - execute samples deployed on IIS

• Tools

                - TCPMon

                - WSClient

• Rampart [User Name Token, Time Stamp, Encryption, UN TS Enc, Interop testing with Java services]
• Sandesha [Oneway, Two way, Single channel, Sequence offer, Last request/ terminate request, In-memory storage, Permanent storage]
• Savan [Savan Client subscription, Publication of a test event and a listener service receiving this event, saven samples agains a java service]
• Check whether all the required folders, files are available in the RPM, Deb and the bin. [Refer Check list to verify the components of different packages which has all the relevant folders and files listed]
• Check if all the necessary components are available in the WSF-C  build. [Refer the check list Check list to verify the components of WSF-C builds]
• Deploy and test on the following application servers,

          - Apache HTTP Server Version 2.x

          - Microsoft IIS Server
 

Regression Testing

Regression testing and retesting cycles are performed to ensure that;

• Changes and defect fixes do not adversely impact previously released versions
• There is an increased stability of the product
• Bugs in the previous releases are addressed correctly

Regression testing will be suspended if showstoppers or critical issues which prevent the testing of major functionalities are  encountered.

Performance Testing

• A simple performance test is done to check if there are any memory leaks when increasing the request count.

          E.g.:- Start the httpd server and send requests to see if memory is increasing each time you send a request to the server or when you increase the request count. (You can use the script Perfomance_test.bat to send several requests at once)

•  Test different message sizes and message formats to see if it affects the performance

           - Use messages with large headers

           - Use messages with large message bodies

           - Use messages with different content types

•   Send multiple messages by using single instance of axis2_http_client_t and axis2_env_t in the transport module (Send nearly 100 messages on windows and around 150 messages on LINUX and see if it hangs at the CLIENT_RECIEVE_HEADER call)

Interop Testing

Interop Testing is done between Java/.Net/C services and Clients as below

• Invoke Java services by .Net and C clients
• Invoke .Net services by Java and C clients
• Invoke C services by Java and .Net clients

These tests are done to verify security, reliable messaging and eventing using Rampart, Sandesha and Savan

Test Environment

WSF-C will be tested on the following software platforms

• Debian - 4.0
• Fedora - Core 5, Core 6 and 7
• Wndows XP
• Ubuntu - 7.04 (Client Version), 7.04 (Server Version)
• Solaris
• Mac OS

          Refer Packaging Vs Testing Platform which lists all the installer types and the OS versions they should be tested on.

Defects Management

WSO2 JIRA will be used as the issue tracking system for release 1.0. The detailed defect tracking information is available at Defects Management

QA Testing Schedule

Appendix