Performance Testing..

Performance Testing can be categorized as follows: -

• Benchmark Testing.
• Contention Testing.
• Performance profiling.
• Load Testing.
• Stress Testing.
• Volume Testing
• Configuration Testing
• Recovery Testing

BENCHMARK TESTING



It is a kind of performance testing that compares the performance of new target-of-test to existing software. It provides performance comparison for software, hardware, and systems. It is used to understand that how a new product will withstand in the competitive environment

CONTENTION TESTING

This test, a test that does a system can handle multiple demands of clients on the same resource. A resource could be data records, memory or something else which could be accessed by clients. It cause the application to fail by accessing same resource so that the underlying defects can be identified, analyzed, fixed, and prevented in the future

PERFORMANCE PROFILING



This is the kind of performance testing that verifies that a system must perform correctly under different configurations while the operational conditions such as number of users, number of transaction remain constant.

LOAD TESTING




Testing of an application under heavy loads, such as testing of a web site on a specific configuration under a range of loads such as number of users, number of transactions to determine at what point the system's response time degrades or fails

Various elements need to be considered before engaging in Load Testing.
Planning Load Testing is like any other activity. Good quality is the result of planning and foresight, enabling the test to be ran over and over again.

Objectives What do we hope to achieve? This question has to be answered in relation to the specified requirements of the Software under Test (SUT). The tester needs to know how the SUT a) should be behave under, b) a given load.

For example for a large site it may be accepted as a business risk that the SUT will run slower at high loads. However, this needs confirming, otherwise the business runs the risk of the SUT due to underestimating the effect of higher loads.

On what? Which aspect of the system is to bear the specified load? The answer ranges from a single component to a Web Services system made up of many third party applications. As the SUT becomes larger, it becomes more complex. Consequently the load testing suite becomes more complex.

For example it is decided a website checkout facility needs testing. This can be tested in isolation as a component or as part of the system. If it can only be tested once the integration has taken place and a fault occurs, was the problem with the cart or somewhere else in the system.
Iterative development styles suit load testing, as they successively build up a full system. The RUP with its inception etc phases gives many opportunities. During the Inception and Elaboration phases the core architecture can be assessed. In construction smaller components can be tested as they added incrementally and then as an integrated whole system.
The Sequential or Waterfall method is less suited, due to its single big bang approach to integration. Thus after months or even years of development, can the system as a whole be load tested. The problem is that it is difficult to establish what you are testing and the causes of failure.

Which Load? How is the load on the SUT to be generated? Multiple users or a small number of users generating large numbers of transactions? The important factor is to replicate the situation the SUT will meet in the real world. For example a bank system, processing payments will have one user processing a lot of data. Whereas a consumer website may have millions of users conducting one or two transactions a day.

Should the data be increased as the test progresses or at a steady sustained level for a long period?

Should the load be of a single type or multiple? For example in our checkout example, a certain percentage of the transactions should be of the user dropping out of the transaction. Again this mirrors the real world.

Testers looking at load testing may want to consider simulations. Here data is randomly generated according to a set of rules. I.e. we do not know which transaction is coming next, but it has a 1 in 5 chance of being a drop out from the checkout.

An important point, often overlooked is that the load may spike. Advertising may make the SUT suddenly very attractive to users. The classic example of failure to realize this was the Victoria's Secrets affair. Here a fashion show of lingerie, during the half time break in the Super Bowl was heavily promoted. Consequently millions of Americans logged on and behold the servers crashed under the strain.

Expected Outcomes The tester has to be aware of what to expect. Should this not be the case, then raise it with the analyst. Just because load testing is non-functional, we still need to know what to expect.

Logging As the test progresses we need to know what exactly a) the load being applied and b) the behavior exhibited. This has two benefits; firstly we can keep track of any deterioration of performance as the load increases.

Tools Due to the large amount of transactions, many Load Testing can only realistically be undertaken with Computer Aided Software Tools (CAST). LoadRunner and Rational Robot are the front runners for this type of testing.

STRESS TESTING

This type of performance tests verifies the target-of-test's performance behavior when abnormal or extreme conditions are encountered. It identifies the boundary values of performance related characteristics of a system. Stress testing is subjecting a system to an unreasonable load while denying it the resources (e.g., RAM, disc, MIPS, interrupts, etc.) needed to process that load. The idea is to stress a system to the breaking point in order to find bugs that will make that break potentially harmful. The system is not expected to process the overload without adequate resources, but to behave (e.g., fail) in a decent manner (e.g., not corrupting or losing data). Bugs and failure modes discovered under stress testing may or may not be repaired depending on the application, the failure mode, consequences, etc.
The load (incoming transaction stream) in stress testing is often deliberately distorted so as to force the system into resource depletion.


VOLUME TESTING
Volume testing will seek to verify the physical and logical limits to a system’s capacity and ascertain whether such limits are acceptable to meet the projected capacity of the organization’s business processing.

It is the testing where the system is subjected to large volumes of data. Volume testing is performed to find faults & it gives credible information about the state of the component, on which business decisions can be taken. As regards faults that should be found through volume testing are those, where the behavior of the software deviates from that expected for a specified volume of data. Thus a bank system will be tested for faults at much larger volumes of data, than that of small retailer software. A fault which is only manifested on a table with a million records will be of no concern to the retail software, but will be picked up by the bank testers.Credible information about how the software will behave is essential. During the dot com boom many websites went live without knowing, what the effect would be if the back end database grew exponentially. Of course many suffered crashes as a result.




CONFIGURATION TESTING

This tests the various software and hardware configurations. Configuration testing is the system testing of different variations of an integrated, black box application against its configurability requirments

The typical objectives of configuration testing are to:
· Partially validate the application (i.e., to determine if it fulfills its configurability requirements).
· Cause failures concerning the configurability requirements that help identify defects that are not efficiently found during unit and integration testing:
Functional Variants.
Internationalization (e.g., multiple languages, currencies, taxes and tariffs, time zones, etc.).
Personalization
· Report these failures to the development teams so that the associated defects can be fixed.
· Determine the effect of adding or modifying hardware resources such as:
Memory
Disk and tape resources
Processors
Load balancers
· Determine an optimal system configuration
RECOVERY TESTING


It tests system’s response to presence of errors or loss of data. This testing aimed at verifying the system's ability to recover from varying degrees of failure. It verifies that the processing results (outputs) of an application are functionally equivalent after one or more changes have been made within the application, and that none of the modifications have adversely impacted the unchanged functionality of the application.


STEPS IN PERFORMANCE TESTING:

The performance requires certain set of steps to be followed which are as follows: -

Determine what factors matter most to you: the number of transactions per second? The speed at which the product it can execute a list of tasks? The number of users that uses your system?
Determine the best way to measure those factors. Sometimes we'll use a benchmark tool or the tools which are available in the market.
Performance tests also require a test bed, which can be as simple as a single PC or as complex as hundreds of network clients and servers
Evaluate the performance of system using above mentioned tools and identifies key factors which are degrading system performance.

LEVELS OF PERFORMANCE EVALUATION

Performance analysis is done at three levels which are:-


The first level of performance analysis involves evaluating the results for a single actor or use-case instance and comparing the results across several test executions This first-level analysis can help identify trends that could indicate contention among system resources, which may affect the validity of the conclusions drawn from other performance test results.

A second level of analysis examines the summary statistics and actual data values for specific actor or use-case execution, and the target-of-test's performance behavior. Summary statistics include standard deviations and percentile distributions for the response times, which provide an indication of the variability in system responses as seen by individual actors. A third level of analysis can help in understanding the causes and significance of performance problems. This detailed analysis takes the low-level data and uses statistical methods to help testers draw correct conclusions from the data.