Load Testing Web Perform

Load Testing to Predict Web Performance

White Paper

Mercury Interactive Corporation 1325 Borregas Avenue Sunnyvale, CA 94089 408-822-5200 www.mercuryinteractive.com

Load Testing White Paper

Abstract Businesses that are leveraging the Web to conduct daily transactions need to provide customers with the best possible user experience in order to be successful. Often, however, these businesses lose customers because their sites are not able to handle surges in Web traffic. For example, a successful promotion that drives Web traffic can radically impact system performance. Online customers, tired of waiting, will simply click to competitors’ sites, and any opportunities for revenue growth will be lost. Whether a business is a brick-and-mortar or a dot.com, the challenges of successfully conducting business online are the same: high user volumes, slow response times for Web requests and ensuring the overall reliability of the service. This paper illustrates how maintaining Web application performance is key to overcoming these e-business challenges and generating revenue. The paper then discusses the importance of maintaining a Web application to ensure customer satisfaction and why load testing is critical to successfully launching and managing Web sites. In addition, it examines various types of load testing and provides a detailed discussion on the load testing process and the attributes of a reliable testing tool. In closing, this paper provides an overview of Mercury Interactive’s LoadRunner® load testing solution.

3

Load Testing White Paper

Table of Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6 Ensuring Optimal End-user Experience—A Complex Issue . . . . . . . . . . . . . . . . .6 Application Load Testing Prior to Going Live . . . . . . . . . . . . . . . . . . . . . . . . . . . .7 Challenges of Automated Load Testing Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . .9 The Process of Automated Load Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10 Mercury Interactive’s LoadRunner . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .22 About Mercury Interactive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .23

5

Load Testing White Paper

Introduction In the last few years, e-business has grown at an accelerated rate. Today, analysts estimate that 260 million people use the Internet—and there is little sign that this growth will slow down. In fact, the International Data Corporation expects the number of online users to reach 500 million within the next two years. E-business has become a popular commercial medium for two reasons: it enables businesses to share information and resources worldwide, and it offers them an efficient channel for advertising, marketing and e-commerce. By using the Internet, businesses have been able to improve their sales and marketing reach, increase their quality assurance to customers and conduct multimedia conversations with customers. More important, businesses are realizing the challenges—and rewards—of providing customers with a positive end-user experience. After all, customers who are satisfied with their online experience are likely to engage in repeat business and provide a steady stream of revenue. As a result, businesses have become more focused on providing positive end-user experiences.

Ensuring Optimal End-user Experience—A Complex Issue In addition to being fast-growing, e-business is also very complex. According to a December 1999 report by the IBM High-Volume Web site team, commercial Web sites can be classified into four categories based on the types of business transactions that they perform: publishing/subscribers, online shopping, customer self-service and trade/auction sites. By understanding these categories, businesses can better predict their level of user volume and understand how users prefer to access the site. Following is an overview of the different commercial Web site categories: Publishing/subscribers sites provide the user with media information, such as magazine and newspaper publications. Although the total number of concurrent users is generally low on these sites, the number of individual transactions performed on a per user basis is relatively high, resulting in the largest number of page views of all site categories. Online shopping sites allow users to browse and shop for anything found in a traditional brick-and-mortar store. Traffic is heavy, with daily volumes ranging between one and three million hits per day. Customer self-service sites include banking and travel reservation sites. Security considerations (e.g., privacy, authentication, site regulation, etc.) are high. Trade/auction sites allow users to buy and sell commodities. This type of site is volatile, with very high transaction rates that can be extremely time-sensitive. No matter the transaction type, Web sites must enable customers to conduct business in a timely manner. For this reason, a scalable architecture is essential. A well-structured Web environment, however, consists of an extremely complex multi-tier system. Scaling this infrastructure from end-to-end means managing the performance 6

Load Testing White Paper

and capacities of individual components within each tier. Figure 1 illustrates the complexity of these components.

Internet

Clients

Routers

Switches

Web Servers

Internet Firewall

Load Balancers

Application Servers

Database Servers & Other Database Sources

Fig. 1. Schematic of a complex Web infrastructure

This complexity prompts many questions about the integrity and performance capabilities of a Web site. For instance, will the response time experienced by the user be less than 8 seconds? Will the Web site be able to sustain a given number of users? Will all the pieces of the system, in terms of interoperability, co-exist when connected together? Is communication between the application server and the database server fast enough? Is there sufficient hardware on each tier to handle high volumes of traffic? To eliminate these performance issues, businesses must implement a method for predicting how Web applications will behave in a production environment, prior to deployment.

Application Load Testing Prior to Going Live To accommodate the growth of their sites, Web developers can optimize software or add hardware to each component of the system. However, to ensure optimal performance, businesses must load test the complete assembly of a system prior to going live. Application load testing is the measure of an entire Web application’s ability to sustain a number of simultaneous users and/or transactions, while maintaining adequate response times. Because it is comprehensive, load testing is the only way to accurately test the end-to-end performance of a Web site prior to going live. Application load testing enables developers to isolate bottlenecks in any component of the infrastructure. Two common methods for implementing this process are manual and automated testing. Manual testing, however, has several built-in challenges, such as determining how to: • Emulate hundreds of thousands of manual users that will interact with the application to generate load • Coordinate the operations of users • Measure response times • Repeat tests in a consistent way • Compare results

7

Load Testing White Paper

Because load testing is iterative in nature, testers must identify performance problems, tune the system and retest to ensure that tuning has had a positive impact—countless times. For this reason, manual testing is not a very practical option. With automated load testing tools, tests can be easily rerun and the results automatically measured. In this way, automated testing tools provide a more cost-effective and efficient solution than their manual counterparts. Plus, they minimize the risk of human error during testing. Today, automated load testing is the preferred choice for load testing a Web application. The testing tools typically use three major components to execute a test: • A control console, which organizes, drives and manages the load • Virtual users, which are processes used to imitate the real user performing a business process on a client application • Load servers, which are used to run the virtual users Using these components, automated load testing tools can: • Replace manual testers with automated virtual users • Simultaneously run many virtual users on a single load-generating machine • Automatically measure transaction response times • Easily repeat load scenarios to validate design and performance changes This advanced functionality in turn allows testers to save time and costly resources.

Internet

Virtual Users

Web Server

Application Server

Database

Manual Testers Control Console

Fig. 2. Manual testers are replaced by a single console controlling several thousand virtual users

Automated testing tools recently demonstrated their value in a report by the Newport Group. The report, published in 1999, revealed that 52 percent of Web-based businesses did not meet their anticipated Web-based business scalability objectives. Of this group, 60 percent did not use any type of automated load testing tool. In contrast, nearly 70 percent of businesses that met their scalability expectations had used an automated load testing tool.

8

Load Testing White Paper

©Newport Group Inc. 1999

Fig. 3. Automated load testing enables businesses to meet scalability expectations

Challenges of Automated Load Testing Tools Primary challenges of load testing tools include the ability to be accurate and scalable and to isolate performance problems. To isolate performance problems, load testing tools monitor key system-level components and identify bottlenecks during the run of a load test. Accuracy is defined by how closely an automated tool can emulate real user behavior. Scalability relates to the product’s ability to generate the maximum load using the minimum amount of resources. Automated load testing tools must address all aspects of accuracy and scalability and be able to pinpoint problems in order to ensure reliable end-to-end testing. Listed below are some key attributes of accuracy and scalability. Accuracy:

Scalability:

• Recording ability against a real client application • Capturing protocol-level communication between the client application and the rest of the system • Providing flexibility and the ability to define user behavior configuration (e.g., think times, connection speeds, cache settings, iterations) • Verifying that all requested content returns to the browser to ensure a successful transaction • Showing detailed performance results that can be easily understood and analyzed to quickly pinpoint the root cause of problems • Measuring end-to-end response times • Using real-life data • Synchronizing virtual users to generate peak loads • Monitoring different tiers of the system with minimal intrusion

• Generating the maximum number of virtual users that can be run on a single machine before exceeding the machine’s capacity • Generating the maximum number of hits per second against a Web server • Managing thousands of virtual users • Increasing the number of virtual users in a controlled fashion

Fig. 4. Key attributes of accuracy and scalability in load testing

9

Load Testing White Paper

The Process of Automated Load Testing By taking a disciplined approach to load testing, businesses can optimize resources, as well as better predict hardware and software requirements, and set performance expectations to meet end-user service level agreements (SLAs). Repeatability of the testing process is necessary to verify that changes have taken place. Following is a step-by-step overview of the automated load testing process:

Step 1: System Analysis This step is critical to interpreting the user’s testing needs and is used to determine whether the system will scale and perform to the user’s expectations. Testers essentially translate existing requirements of the user into load testing objectives. A thorough evaluation of the requirements and needs of a system, prior to load testing, will provide more realistic test conditions. First, the tester must identify all key performance goals and objectives before executing any testing strategies. Examples include identifying which processes/transactions to test, which components of a system architecture to use in the test and the number of concurrent connections and/or hits per second to expect against the Web site, as well as clarifying which processes are to be tested. By referring to the four models of Web sites (see page 6), developers can easily classify their site’s process/transaction type, allowing users to conduct transactions in a more timely fashion. For example, a business-to-consumer model can implement an online shopping process in which a customer browses through an online bookstore catalog, selects an item and makes a purchase. This process could be labeled “buy book” for the purposes of the test. Defining these objectives will provide a concise outline of the SLAs and mark the goals that are to be achieved with testing. Second, the tester needs to define the input data used for testing. The data can be created dynamically. For example, auctioning bids may change every time a customer sends in for a new request. Random browsing also may be used to obtain the data. Examples include any non-transactional process, such as browsing through a brochure or viewing online news. Emulating data input can avoid potential problems with inaccurate load test results. Third, testers must determine the appropriate strategy for testing applications. They can select from three strategy models: load testing, stress testing and capacity testing. Load testing is used to test an application against a requested number of users. The objective is to determine whether the site can sustain this requested number of users with acceptable response times. Stress testing, on the other hand, is load testing over extended periods of time to validate an application’s stability and reliability. The last strategy is capacity testing. Capacity testing is used to determine the maximum number of concurrent users an application can manage. For example, businesses would use capacity testing to benchmark the maximum loads of concurrent users their sites can sustain before experiencing system failure.

10

Load Testing White Paper

Fourth, testers need to cultivate a solid understanding of the system architecture, including: • Defining the types of routers used in the network setup • Determining whether multiple servers are being used • Establishing whether load balancers are used as part of the IP networks to manage the servers • Finding out which servers are configured into the system (Web, application, database) Last, developers must determine which resources are available to run the virtual users. This requires deciding whether there is a sufficient number of load generators or test machines to run the appropriate number of virtual users. It also requires determining whether the testing tool has multithreading capabilities and can maximize the number of virtual users being run. Ultimately, the goal is to minimize system resource consumption while maximizing the virtual user count.

Step 2: Creating Virtual User Scripts A script recorder is used to capture all the business processes into test scripts, often referred to as virtual user scripts or virtual users. A virtual user emulates the real user by driving the real application as a client. It is necessary to identify and record all the various business processes from start to finish. Defining these transactions will assist in the breakdown of all actions and the time it takes to measure the performance of a business process.

Step 3: Defining User Behavior Run-time settings define the way that the script runs in order to accurately emulate real users. Settings can configure think time, connection speed and error handling. Think times can vary in accordance with different user actions and the user’s level of experience with Web technology. For example, novice users require more time to execute a process because they have the least experience using the Web. Therefore, a tester will need to emulate more think time in the form of pauses. Advanced users, however, have much more experience and can execute processes at an accelerated level, often by using shortcuts. System response times also can vary because they are dependent on connection speed, and all users connect to the Web system at different speeds (e.g., modem, LAN/WAN). This feature emulates dial-up connections over PPP at varying modem speeds (e.g., 28.8 Kbps, 56.6 Kbps, etc.) and is useful for measuring application response times based on the connection speed. Error handling is another setting that requires configuration. Errors arise throughout the course of a scenario and can impede the test execution. The tester can configure virtual users to handle these errors so that the tests can run uninterrupted.

Step 4: Creating a Load Test Scenario The load test scenario contains information about the groups of virtual users that will run the scripts and the load machines that the groups are running on.

11

Load Testing White Paper

In order to run a successful scenario, testers must first define individual groups based on common user transactions. Second, they need to define and distribute the total number of virtual users. A varying number of virtual users can be assigned to individual business processes to emulate user groups performing multiple transactions. Third, testers must determine which load generating machines the virtual users will run on. Load generator machines can be added to the client side of the system architecture to run additional virtual users. Last, testers need to specify how the scenario will run. Virtual user groups can either run in staggered or parallel formation. Staggering the virtual users allows testers to examine a gradual increase of the user load to a peak.

Step 5: Running the Load Test Scenario and Monitoring the Performance Real-time monitoring allows testers to view the application’s performance at any time during the test. Every component of the system requires monitoring: the clients, the network, the Web server, the application server, the database and all server hardware. Real-time monitoring allows for early detection of performance bottlenecks during test execution. Testers then have the ability to view the performance of every single tier, server and component of the system during testing. As a result, testers can instantly identify performance bottlenecks during load testing. They then can accelerate the test process and achieve a more stable application.

Step 6: Analyzing Results This is the most important step in collecting and processing the data to resolve performance bottlenecks. The analysis yields a series of graphs and reports that help summarize and present the end-to-end test results. For example, Figure 5 uses generic data to display a standard performance under load graph that shows the total number of virtual users against the response time. This can be used to determine the maximum number of concurrent users until response times become unacceptable. Figure 6 shows a transaction overview revealing the total number of transactions that passed in a scenario. Analysis of these types of graphs can help testers isolate bottlenecks and determine which changes are needed to improve system performance. After these changes are made, the tester must rerun the load test scenarios to verify the adjustments.

Fig. 5. This is a generic graph showing performance under load. This graph is useful in pinpointing bottlenecks. For example, if a tester wants to inquire about the user threshold at 2 seconds, the results above show a maximum of 7,500 concurrent users.

12

Load Testing White Paper

Fig. 6. This graph is a generic display of the number of transactions that passed or failed. In the above example, if the goal is to obtain a 90-percent passing rate for the number of transactions, then transaction 2 fails. Approximately 33 percent out of 100 transactions failed.

Mercury Interactive’s LoadRunner Mercury Interactive’s LoadRunner is a load testing tool that predicts system behavior and performance. It exercises an entire enterprise infrastructure by emulating thousands of users to identify and isolate problems. Able to support multiple environments, LoadRunner can test an entire enterprise infrastructure, including e-business, ERP, CRM and custom client/server applications, thereby enabling IT and Web groups to optimize application performance. By emulating the behavior of a real user, LoadRunner can test applications communicating with a wide range of protocols, such as HTTP(s), COM, CORBA, Oracle Applications, etc. LoadRunner also features a seamless integration with Mercury Interactive’s Web performance monitoring tool, Topaz™. Therefore, the same tests created during testing can be reused to monitor the application once it is deployed. LoadRunner enhances every step of the load testing process to ensure that users reap the maximum return on their investment in the tool. The remainder of this paper discusses how LoadRunner offers support for each segment of the load testing process:

Step 1: System Analysis LoadRunner advocates the same system analysis as mentioned previously in this paper. In emulating a test environment, it is necessary to identify all testing conditions, including system architecture components, the processes being tested and the total number of virtual users to test with. A good system analysis will enable customers to convert their goals and requirements into a successful, automated test script.

Step 2: Creating Virtual User Scripts You begin by recording the business processes to create a test script. Script recording is done using LoadRunner’s Virtual User Generator (VUGen). VUGen is a component that runs on a client desktop to capture the communication between the real client application and the server. VUGen can emulate the exact behavior of a real browser by

13

Load Testing White Paper

sending various e-business protocol requests to the server. VUGen also can record against Netscape or Internet Explorer browsers—or any user-defined client that provides the ability to specify a proxy address. After the recording process, a test script is generated.

Fig. 7. The Virtual User Generator allows testers to capture business processes to create virtual users

You can then add logic to the script to make it more realistic. Intelligence can be added to the scripts so that they emulate virtual user reasoning while executing a transaction. LoadRunner executes this stage using the transactions, as well as its verification and parameterization features. • Transactions. Transactions represent a series of operations that are required to be measured under load conditions. A transaction can be a single URL request or a complete business process leading through several screens, such as the online purchase of a book.

• Verification. VUGen allows insertion of verification checkpoints using ContentCheck™. ContentCheck verifies the application functionality by analyzing the returned HTML Web page to ensure a successful transaction. If the verification fails, LoadRunner will log the error and highlight the reasons for the failure (e.g., broken link, missing images, erroneous text). • Parameterization. To accurately emulate real user behavior, LoadRunner virtual users use varying sets of data during load testing, replacing constant values in the script with variables or parameters. The virtual user can substitute the parameters with values from a data source, such as flat files, random numbers, date/time, etc. This allows a common business process, such as searching for or ordering a book, to be performed many times by different users.

14

Load Testing White Paper

Step 3: Defining User Behavior LoadRunner provides comprehensive run-time settings to configure scripts that emulate the behavior of real users.

Fig. 8. The run-time settings are used to emulate the real user as closely as possible. In this example, think time is randomly generated to simulate the speed at which the user interacts with the system.

Examples of run-time settings include: Think time:

Controls the speed at which the virtual user interacts with the system by including pauses of think times during test execution. By varying think times for users, LoadRunner can emulate the behaviors of different users—from novice to expert users.

Dial-up speed:

Emulates a user connected to the system using a modem and/or LAN/WAN connections. Modem speeds range from 14.4 Kbps to 56.6 Kbps. This is useful for controlling user behavior in order to accurately emulate response times for each request.

Emulate cache:

Emulates a user browsing with a specific cache size. Caching can be turned off based on server requirements.

Browser emulation:

Enables the tester to specify which browser the virtual user emulates. LoadRunner supports both Netscape and Internet Explorer, as well as any custom browser.

Number of connections:

Allows the virtual user to control the number of connections to a server, like a real browser, for the download of Web page content.

IP spoofing:

Tests the performance impact of IP-dependent components by assigning virtual users their own IP addresses from the same physical machine.

15

Load Testing White Paper

Iterations:

Commands repetition of virtual user scripts. Also paces virtual users, instructing how long to wait between intervals. Iterative testing defines the amount of work a user does based on the number of times a process is performed using varying data.

Error handling:

Regulates how a virtual user handles errors during script execution. LoadRunner can enable the Continue on Error feature when the virtual user encounters an error during replay.

Log files:

Stores information about virtual user server communication. Standard logging maps all transactions, rendezvous and output messages. Extended logging also tracks warnings and other messages.

Step 4: Creating a Load Test Scenario LoadRunner’s Controller is used to create scenarios. As a single point of control, it provides complete visibility of the tests and the virtual users.

Fig. 9. LoadRunner’s Controller is an interactive environment for organizing, driving and managing the load test scenario

The Controller facilitates the process of creating a load test scenario by allowing users to: • Assign scripts to individual groups • Define the total number of virtual users needed to run the tests • Define the host machines that virtual users are running on In addition, LoadRunner offers a Scenario Wizard, a Scheduler and TurboLoad™ to enhance the tester experience.

16

Load Testing White Paper

Scenario Wizard. LoadRunner’s Scenario Wizard is a feature that enables testers to quickly compose multi-user load test scenarios. Using five easy-to-follow screens, the Scenario Wizard steps the tester through a process of selecting the workstations that will host the virtual users as well as the test scripts to run. During this step-by-step process, testers also create simulation groups of virtual users. (The steps for creating a scenario are the same as those described on page 13.) Scheduler. LoadRunner Scheduler is used to ramp virtual user numbers up/down in order to position virtual users in both the ready state and the running state. For example, the tester may want to gradually increase the load of users logging into a site with a fixed batch size. This is referred to as the ready state. This method is useful for avoiding unnecessary strain on the system. The schedule also manages scheduling and features an automated process that allows the user to run the script without being present. In real time this would be analogous to running a script during off-peak hours of Internet traffic—6:00 p.m. to 6:00 a.m. To schedule a test, the user simply has to click the Run Scenario button and enter the desired starting time. TurboLoad. TurboLoad is a patent-pending technology that provides for maximum scalability. TurboLoad can minimize CPU consumption for each virtual user, thereby enabling more users to run on each load generator machine. In recent customer benchmarks, using 10 Windows-based load servers (4 CPU, 500 MHz Xeon processors, 4 GB RAM), LoadRunner generated 3 billion Web hits per day against the Web system (or 3,700 hits/sec per machine). Moreover, the load generators were running at less than 40 percent CPU utilization. TurboLoad also can generate more hits/sec for a given machine. LoadRunner’s replay speed can thereby generate more throughput against the server using a minimum amount of resources.

Step 5: Running the Load Test Scenario and Monitoring the Performance Once the scenario is built, the tester is ready to run the test. LoadRunner’s Controller provides a suite of performance monitors that can monitor each component of a multitier system during the load test. By capturing performance data over the entire system, testers can then correlate this information with the end-user loads and response times in order to pinpoint bottlenecks. LoadRunner provides performance monitors for the network, network devices and the most common Web servers, application servers and database servers. The performance monitoring is done in a completely non-intrusive manner to minimize performance impact. Additionally all of these monitors are hardware and OS independent, as no agents are required to be installed on the remotely monitored servers.

17

Load Testing White Paper

Fig. 10. LoadRunner online monitors help identify and isolate performance bottlenecks in real time

LoadRunner supports a number of environments to provide for more accurate online monitoring, including: Runtime Graphs:

Virtual User Status, User Defined Data Points

Transaction Graphs:

Response Time, Transactions (pass/fail)

Web Server Resource Graphs:

Hits per Second, Throughput, Apache, MS IIS, Netscape

System Resource Graphs:

Server resource, SNMP, Tuxedo

Web Application Server Graphs:

BroadVision, ColdFusion, MS Active Server Pages, SilverStream, WebLogic

Database Server Resource Graphs: SQL Server, Oracle

Step 6: Analyzing Results Evaluating results is the most important step in the load testing process. Up until now the tester has been able to record and play the actions of a real user with extreme precision, while conducting multiple processes on the Web. In addition, the performance monitoring feature offers an accurate method for pinpointing bottlenecks while running the scripts. To fix these problems, testers can follow several steps. First, a network specialist (DBA, consultants) can be used to make the necessary adjustments to the system. As a next step, testers need to rerun the scripts, to verify that the changes have taken place. Last, a comparison of the results from before and after enables the tester to measure the amount of improvement that the system has undergone. LoadRunner’s Analysis component provides a single integration environment that collectively gathers all the data generated throughout the testing cycle. Because this tool is powerful and easy to use, testers can create cross-scenario comparisons of the graphs and thereby enrich the data analysis process. For example, Figure 11a shows the results of a dot.com after testing the maximum number of concurrent users that its existing system can handle. Based on these results, the dot.com plans to improve its infrastructure to allow more user traffic. Figure 11b provides a comparison of a repeated test after adjustments had been made to the Web architecture to optimize server software. 18

Load Testing White Paper

Fig. 11a. Results of a dot.com using LoadRunner Analysis before making any adjustments to the system infrastructure. The site reached a peak of 4,000 Vusers before any performance degradation.

Fig. 11b. Results of the same company using LoadRunner Analysis after making enhancements to the performance of the system. The number of Vusers increased five fold to a peak of 20,000.

LoadRunner Analysis provides advanced, high-level drill-down capabilities that enable testers to locate bottlenecks in these scenarios. In addition, LoadRunner Analysis uses a series of sophisticated graphs and reports that answer such questions as: What was the Web server’s CPU memory when the system was under a load of 5,000 concurrent users? How many total transactions passed or failed after the completion of the load test? How many hits per second did the Web server uphold? What were the average transactions times for each virtual user? Below are sample graphs that LoadRunner Analysis provides testers as it solves complex bottleneck issues. • Running virtual users. Displays running virtual users during each second of a scenario. • Rendezvous. Indicates when and how virtual users were released at each point. • Transaction/sec (passed). Displays the number of completed, successful transactions performed per second. • Transaction/sec (failed). Displays the number of incomplete, failed transactions performed per second.

Fig. 12. This activity graph displays the number of completed transactions (successful and unsuccessful) performed during each second of a load test. This graph helps testers determine the actual transaction load on their system at any given moment. The results show that after six minutes an application is under a load of two hundred transactions per second.

19

Load Testing White Paper

LoadRunner provides a variety of performance graphs: • Percentile. Analyzes percentage of transactions that were performed within a given time range. • Performance under load. Indicates transaction times relative to the number of virtual users running at any given point during the scenario. • Transaction performance. Displays the average time taken to perform transactions during each second of the scenario run. • Transaction performance summary. Displays the minimum, maximum and average performance times for all the transactions in the scenario. • Transaction performance by virtual user. Displays the time taken by an individual virtual user to perform transactions during the scenario. • Transaction distribution. Displays the distribution of the time taken to perform a transaction.

Fig. 13. This performance graph displays the number of transactions that passed, failed, aborted or ended with errors. For example, these results show the “Submit_Search” business process passed all its transactions at a rate of approximately 96 percent.

Fig. 14. This performance graph displays the minimum, average and maximum response times for all the transactions in the load test. This graph is useful in comparing the individual transaction response times in order to pinpoint where most of the bottlenecks of a business process are occurring. For example, the results of this graph show that “FAQ” business process has an average transaction response time of 1.779 seconds. This would be an acceptable statistic in comparison to the other processes.

20

Load Testing White Paper

LoadRunner offers two types of Web graphs: • Connections per second. Shows the number of connections made to the Web server by virtual users during each second of the scenario run. • Throughput. Shows the amount of throughput on the server during each second of the scenario run.

Fig. 15. This Web graph displays the number of hits made on the Web server by Vusers during each second of the load test. This graph helps testers evaluate the amount of load Vusers generate in terms of the number of hits. For instance, the results provided in this graph indicate an average of 2,200 hits per second against the Web server.

Fig. 16. This Web graph displays the amount of throughput (in bytes) on the Web server during the load test. This graph helps testers evaluate the amount of load Vusers generate in terms of server throughput. For example, this graph reveals a total throughput of over 7 million bytes per second.

LoadRunner’s Analysis includes a Correlation of Results feature to enhance the user analysis process of the data. Correlation enables the tester to custom design a graph beyond the basics, using any two metrics. As a result, the tester can pinpoint and troubleshoot performance problems more quickly.

21

Load Testing White Paper

Fig. 17. This graph correlates the relationship of the system behavior to the number of users, using a compilation of results from other graphs. This enables the tester to view the CPU consumption based on the total number of users.

Summary In a short time, e-business has proven to be a viable business model for dot.coms and brickand-mortars alike. With the number of Internet users growing exponentially, it is therefore critical for these businesses to prepare themselves for high user volumes. Today’s businesses can leverage load testing practices and tools to ensure that Web application performance keeps pace with end-user demand. Moreover, by using automated load testing tools, businesses can quickly and cost-effectively assess the performance of applications before they go live, as well as analyze their performance after deployment. As a result, businesses can confidently stay one step ahead of performance issues and focus on initiatives to drive Web traffic and revenues. Mercury Interactive’s LoadRunner is the leading tool for predicting scalability, reliability and performance issues of an e-business application; identifying system bottlenecks; and displaying results. LoadRunner emulates various types of transactions using a highly scalable number of users. This is essential for understanding an application’s limitations while planning for growth and reducing business risk. LoadRunner also tests system behavior under real-time conditions and converts this data into easy-to-use, yet sophisticated, graphs and reports. With this information, businesses can more quickly and efficiently resolve problems, thereby ensuring a positive end-user experience and providing the opportunity for increased revenue.

22

Load Testing White Paper

About Mercury Interactive Mercury Interactive Corporation is the worldwide leader in Web performance management solutions. Mercury Interactive solutions turn Web application performance, scalability and user experience into competitive advantage. The company’s performance management products and hosted services are open and integrated to best test and monitor business-critical Web applications. Together with our world-class partners and award-winning service and support, Mercury Interactive provides the industry’s best Web application performance solutions capable of supporting both traditional and wireless e-business applications. More than 10,000 e-commerce customers, Internet service providers, applications service providers, systems integrators and consultants use Mercury Interactive solutions. Mercury Interactive is headquartered in Sunnyvale, California, and has 40 offices worldwide. For more information on Mercury Interactive, visit our Web site at www.mercuryinteractive.com.

Topaz, TurboLoad and ContentCheck are trademarks, and LoadRunner, Mercury Interactive and the Mercury Interactive logo are registered trademarks of Mercury Interactive Corporation. All other company, brand and product names are marks of their respective holders. © 2000 Mercury Interactive Corporation. Patents pending. All rights reserved. 384-BR-LOADTEST

23