Web Stress Testing

Web Stress Testing

Matt Odhner Program Manager Application Center Microsoft

Topics to be Covered











Important web hosting statistics What stress testing solves & common stress testing issues Web Application Stress tool demonstration Understanding the report data What a commercial stress tool needs Application Center Test (ACT) demonstration Visual Studio integration The importance of bandwidth testing Other ACT features Capacity planning

Web Hosting Statistics







Popular web sites receive over 40,000,000 requests per day Web hosting is expected to be a $14.6 billion industry by 2003 Newport Group study revealed 52% of web applications failed to scale, 60% had no performance testing, only 25% used a testing tool Zona Research found that $4 billion a year is lost due to slow and failed web pages

Stress Testing Solves …







Stability issues - unexpected downtime and poorly written COM objects Performance problems – locate bottlenecks and whether the application will handle peak loads Capacity planning - how many machines are needed to support usage Plus, it avoids financial losses and ensures customer satisfaction

Stress Testing Locates Bottlenecks








Memory Processor Network Hard disk COM component

Common Stress Test Issues








Invalid test platform Invalid script Thread safety issues with unstable server COM components Active Server Page script errors and GLOBAL.ASA issues Insufficient processor power and/or scripts are too complex for the hardware

Traditional Test Bed Web server

Member Stress Client

Member Stress Client Controller Stress Client

Approaches to Stress Testing







Confirm that the application functions under load Find the maximum requests per second my application can handle Determine the maximum number of concurrent connections my application can handle Test the application with x number of unique users

Creating the Script Scripts view - create, edit, delete, and run stress scripts.

Creating the Script (cont’d) Record using your browser – this method is recommended because all page elements are captured.

Script Configuration - Settings

Concurrent browser connections

Script configuration - Users Users view - manage user names and passwords.

Script Configuration - Clients Clients view - add and remove the client machines used to stress the web server(s)

Script Configuration - Script Item Details Script item details – modify query string, edit POST data, use SSL, query RDS.

Performance Monitor Data Collect performance monitor counters from within the tool

Scriptable Object Model

Sub AddNewScript() Set objWAS = CreateObject("WAS.EngControl.1") Set objScripts = objWAS.Scripts Set objScript = objScripts.Add objScript.sName = "A New Script" objScript.ScriptItems.sServer = "www.targetserver.com" MsgBox "The new ScriptID is " & objScript.ScriptID End Sub

Session Trace Option

=h= Selected new user: User2, password, ASPSESSIONIDGQQGQNYG=MPBBLMOAFBPBPDFAJLMCCFCO; Account=6001 =h=

closed socket

=h=

linger is on, linger timeout is 0

=h=

connected to the server.

=h=

sending request to server (470 bytes)

172.30.170.217:80

GET /fmstocks/_NewAccount.asp HTTP/1.1 Host: localhost

ASP Client

Reports – Tabular Report Data Reports view - analyze the results of a test.

Time to First and Last Byte

0 ms Request 9 ms 6 ms Response Time To First Byte

Time To Last Byte

Calculating ASP Performance MHz Cost = N * S * avg (Pt) / avg(Rps) Where:
N = Number of Processors
S = Speed of Processors
Pt = System: % Total Processor Time
Rps = ASP: Requests per secondReports view analyze the results of a test.

Calculating ASP Performance Example: a test using a 2 processor web server achieved 350 requests per second, with the processors 70% utilized. So, this works out to …
2 processors * 400 MHz => 800 MHz
70% processor utilization => (800) * (0.70) => 560 MHz used
350 ASP requests per second 560/350 => 1.6 million cycles per ASP request

More About Web Application Stress Tool
Designed

to meet internal test needs
http://webtool.rte.microsoft.com contains: –Searchable knowledge base –Tutorial of main features –Latest download –Object model sample scripts
Peer support on the discussion alias: [email protected]
Online

help contains samples
The tool respects ROBOTS.TXT

What is Needed in a Commercial Stress Test Tool? All of the features in Web application Stress, Plus:
Accurate simulation of client connection speeds
Running a script a specific number of iterations
Testing of servers in multiple domains
Handle complex applications that require conditional responses
Capable of changing subsequent requests based on the response of a previous request
Functional and performance testing
In-depth reporting and troubleshooting

Microsoft Application Center Test

Significant Features

1. 2. 3. 4. 5. 6. 7.

Integration with Visual Studio IP router technology Data collector to replay actual web site traffic Completely scriptable, even as the test is running Graphical and tabular reporting Capacity planning Can import Web Application Stress scripts

1. Visual Studio Integration

Visual Studio Integration (cont’d)

Bandwidth Statistics
58% of web site users said download speed was a primary factor that influences them to return to a site (Forrester Research)
Web surfers give a site about 10 to 15 seconds to display before they hit the stop button and go to a different site
Bandwidth is not expected to improve soon Wireless 6%

Chart shows forecast for the year 2003

Cable 16% ISDN 5% xDSL 13%

Analog 60%

2. IP Router Technology

Web farm

56 K

T1

T3

28.8

Client computers

IP Router Technology (cont'd)

ACT

Local Area Network

Web cluster

Delay prior to sending each packet so that 1,588 bits per second is maintained

IP Router Technology (cont'd)

3. Data Collector

Internet traffic

Replays exact traffic

Client computers

4. Completely Scriptable If IsObject(Session) = False Then Set Session = CreateObject("ACT.Session") End If

Set oRequest = Session.CreateRequest oRequest.Path = "/samples/cookie.asp" oRequest.Headers.Add "Connection", "Close" oRequest.Headers.Add "Cookie", "FirstName=Matt&Lastname=Odhner" Set oConnection = Session.CreateConnection("MATTOD15") Set fs = CreateObject("Scripting.FileSystemObject") Set writeToFile = fs.OpenTextFile("c:\test.txt", 8, TRUE) thePath = oRequest.Verb & " " & oRequest.Path & " " & _ oConnection.Send(oRequest).ResultCode & chr(13) & chr(10) writeToFile.Write thePath writeToFile.Close

Reporting - Tabular Reports

Reporting - Connections vs. TTFB & TTLB

Reporting - Comparing Scripts

6. Capacity Planning 1. Manual capacity planning

Capacity Planning (cont'd) 2. Historical capacity planning

Internet

Filter

Filter

Filter

Web cluster

Capacity Planning (cont'd) 3. Calibrated capacity planning Web farm

Client computers

Summary Web Stress tools give answers to the problems of Web application stress, via:





Modeling Prediction Identification of bottlenecked resources Provision of performance Metrics to help developers decide what content to deploy

So - employ one today to improve your web application's performance!

Resources - Further Reading http://www.microsoft.com/TechNet/iis/sol.asp http://msdn.microsoft.com/library/techart/windnami stakes.htm http://msdn.microsoft.com/workshop/server/asp/server1 02599.asp http://msdn.microsoft.com/workshop/server/asp/server0 92799.asp http://msdn.microsoft.com/workshop/server/asp/server1 22799.asp http://www.microsoft.com/siteserver/commerce/support/ highcapacity.htm

Resources - Other Web Stress Tools Available Mercury Interactive, Loadrunner/Winrunner http://www.merc-int.com/products/ Radview, Webload - http://www.radview.com/webload/ RSW, eLoad - http://www.rswsoftware.com/ Segue, Silkperformer http://www.segue.com/html/s_solutions/silk/s_performe r.htm Bluecurve, Dynameasure http://www.bluecurve.com/products/products.htm Rational, Rational Suite / PerformanceStudio http://www.rational.com/products/rs/pstudio/index.jtm pl

Questions?

Matt Odhner Program Manager Application Center Microsoft