Loadrunner Correlation

Correlation? What’s that? If you think correlation has something to do with the fit of data points to a function curve on a graph, and the word has no meaning to you in the context of Mercury’s tool, then this document is for you. It explains what correlation is, why you have to do it, how to do it, and what to do when it goes wrong. If this is the first time that you have used VuGen or if you have been using it a little but are not a guru, then read on.

Introduction VuGen, when recording a script, simply listens to the client (browser) talking to the server (Web server) and writes it all down. The complete transcript of everything that was said, the dates and time, content, requests, and replies can be found in the Recording Log (View
Output Window
Recording Log). The script is an easier to read version of this. The main difference is that the script only contains the client’s communication. If you imagine that VuGen is an impersonator pretending to be the client (browser), the script tells VuGen what to say to the server to successfully fool it. You want the server to believe that VuGen is a real client, and thus send it the requested information. This script has the hard coded information of the original conversation (Browser session) that occurred between the client and server. This hard coded information may not be enough to fool the server during replay however; it may have to be correlated.

What is correlation? Correlation is where the script is modified so that some of the hard coded values in the script are no longer hard coded. Rather then sending the original value to the server, we may need to send different values. For example, the original recorded script may have included the server sending the client a session identification number, something to identify the client during that particular session. This Session ID was hard coded into the script during recording. During replay, the server will send the client a new Session ID. You need to capture this value, and incorporate it into the script so you can send it back to the server to correctly identify yourself for this new session. If you leave the script unmodified, you will send the old hard coded Session ID to the server. The server will look at it and think it invalid, or unknown, and so will not send the pages that have been requested. Replaying script with old will Session ID not have successfully fooled the server into believing it is a client. Correlation is the capturing of dynamic values passed from the server to the client and back. You save this captured value into a parameter, and then use this parameter in the script in place of the original value. During replay, the replay engine will now listen to what the server sends to it, and when it makes requests of the server, send this new, valid value back to the server; thus fooling the server into believing it is talking to a real client.

Why do I have to correlate? If you try to replay a script without correlating first, then most likely the script will fail. The requests it sends to the server will not be replied to. Either the Session ID is invalid so the server will not allow you into the site, or it will not allow you to create new records because they are the same as existing ones, or the server will not understand your request because it is not what it is expecting. Any value that changes every time you connect to the server is a candidate for correlation. A correlated script will send the server the information it is looking for, and so allow the script to replay. This will allow many Vusers to replay the script many times, placing load on your server.

What errors mean I have to correlate? There are no specific errors that are associated with correlation, but there are errors that could be caused because a value has not been correlated. For example, if an invalid Session ID is sent to a Web server, how that server responds depends on the implementation of that server. It might send a page specifically stating the Session ID is invalid and ask you to log in again. It might send an HTTP 404 Page not found error because the requesting user did not have permissions for the specified page, and so the server could not find the page. In general, any error message returned from the server after the script makes a request that complains about permissions can point to a hard coded value that needs to be correlated.

The tools (functions) used to correlate There are 3 functions that you can use for correlation. A list of them, along with documentation and examples, can be found in the on-line documentation. From VuGen, go to Help
Function reference
Contents
Web and Wireless Vuser Functions
Correlation Functions. The first function, web_reg_save_param, is the latest function with some extra features for a more powerful usage. The second and third function, web_create_html_param and web_create_html_param _ex are for backward compatibility. They are essentially the same, and will be talked about together. The last function is associated with the second and third function, and is not directly a correlation function but rather a setting. It will be talked about later in a different section.

Web_reg_save_param web_reg_save_param ( “Parameter Name” , , LAST ); Each of these parameters is a pointer to a string. That means that if they are entered as literal text, they need to be enclosed in quotes, with each parameter separated by a comma. The supported attributes include LB (Left boundary), RB (Right boundary), RelFrameID, ORD, Search, SaveOffset, SaveLen, and Convert. These attributes can appear in any order because they contain within them what they are. Detail information about each attribute can be found on the function reference online.

Web_create_html_param web_create_html_param ( “Parameter Name”, “Left Boundary”, “Right Boundary” ); This function is for backward compatibility only. It was a standard for LoadRunner6.5 and below. This function takes three parameters. Each of these parameters is a pointer to a string. That means that if they are entered as literal text, they need to be enclosed in quotes, with each parameter separated by a comma. Parameter Name - This is the name of the parameter, or placeholder variable for saving the captured value. After successfully capturing the value, the parameter name is used in the script in place of the original value. The replay engine will identify the parameter/placeholder, and substitute the captured value for the placeholder during replay. This name should have no spaces, but apart from that limitation, it is entirely up to you what name to give it.

Left Boundary - This is where you tell the replay engine how to find the dynamic value you are looking for. In the Left Boundary, you specify the text that will appear to the left of the changing value. Right Boundary - This is where you tell the replay engine how to identify the end of the dynamic value you are looking for. You use the text that will appear after the value.

Web_create_html_param_ex web_create_html_param_ex ( “Parameter Name”, “Left Boundary”, “Right Boundary”, “Instance”); This function is the same as the web_create_html_param function, except it does not look for the first instance of the boundaries, but rather the nth instance of those boundaries. The first three parameters (Parameter Name, Left Boundary, and Right Boundary) are the same, the last parameter is a pointer to a string, and so it must be enclosed in double quotes. It is the number of the occurrence. If you place the number 1 here, the function behaves exactly as the web_create_html_param function; it looks for the first occurrence. If you put the number 3 here, it will look for the third occurrence of the left and right boundaries, and place what appears in-between into the parameter.

Identifying values to correlate So you have the tools, and you know why you need to use them, but how do you know what to use them on? What values in the script need to be correlated? The simplest answer is “Any value that changes between sessions required for the script to replay.” A hypothetical example You are logging onto a website. When you send the server your user name and password, it replies with a Session ID that is good for that session. The Session ID needs to be correlated for replay. You need to capture this value during replay to use in the script in place of the hard coded value. To identify values to correlate, record the script, and save it. Open a new script, and record the same actions and business process again. As much as possible, during recording, enter the same values in both scripts, for example, user ID, password, and fields and edit selections. Save the second script, and run it with Extended log (Vuser
Run-Time Settings
Log
Extended log. Check all three options). Go to Tools → Compare with Vuser, and choose the first recorded script. WinDiff will open and display the two scripts side by side. Lines with differences in them will be highlighted in yellow. Differences within the line will be in red (If not, go to Options
View
Show inline differences.)

If WinDiff gives an error here, dismiss the error. WinDiff will be minimized in the task bar, right-click on it and choose “Restore.” Go to File → Select Files/Directories and manually select the action sections for the two scripts. Differences like "lr_think_time" can be ignored. They are pacing functions, and do not represent data sent to the server. Locate the first difference, take note of it, and search the script open in VuGen for that difference. That is the original value hard coded into the script that was different in the second script. Highlight it and copy it.

Go to the recording log and place your cursor at the top. Press Control-F (CTRL+F) to do a search and paste in the original value. You are looking for the first occurrence of this value in the recording log. If you do not find the value in the recording log, verify you are looking in the correct script’s recording log. Remember you have two almost identical scripts here. If you find the value, scroll up in the log, and make sure the value was sent as part of a response from the server. The first header you come across while looking up the script should be prefaced by a receiving response. This indicates that the server sent the value to the client. If the value first appears as part of a sending request, then the value originated on the client side and does not need to be correlated, but rather parameterized.

That is a different topic all together. The response will have a comment before it that looks like this: *** [tid=b9

Action1 2] Receiving response from host astra.merc-int.com:80 ( 25/11/2002 12:04:00 )

So, you have a value that is different between subsequent recordings, it was sent from the server to the client; this value most likely needs to be correlated. If the value is not different between recordings and not originating on the server and sent to the client then the value probably does not need to be correlated.

How to correlate Step 1 After confirming that the first occurrence was part of a received response from the server, you need to figure out where to place the web_reg_save_param( ) function. The web_reg_save_param statement needs to go immediately before the request that fetched the dynamic value from the server. In order to find this request or URL in the script, you need to replay the script once with extended log and all the three options (In Vuser → Runtime Settings → Log) turned on. In the Recording log, pick up the text that is before the dynamic value. This text should remain constant no matter how many times you replay the script and highlight it and copy it. This is the text that will identify where to find the start of the value you are capturing.

Now, go to the Execution log and search for the text that you just copied from the Recording log. You should see a corresponding Action1.c() at the beginning of that line with a number in the brackets. That is the number of the line in the script where you need to put web_reg_save_param( ) function. The function should go right above that line in the script.

So, add a couple of blank lines to your script before the function at that line, and then type in web_reg_save_param(“UserSession”, but give it a name that means more to you than UserSession.

Step 2 Go back to the recording log, highlight the text to the left of the dynamic value and copy it. This should be some of the same text you searched for in the Execution log. The amount of text you highlight should be sufficient so that it is unique in this reply from the server. It is suggested that you copy as much as possible without copying any special characters. These show in the Execution log as black squares, and the actual character they represent is uncertain. After selecting a boundary, go to the top of the Servers reply, and press CTRL+F and do a search for that boundary. You want to make certain what you have selected is the first occurrence in the Servers reply. If it is not select, use more text to make it unique, or consider using the ORD parameter of the web_reg_save_param function. Once you have finalized the static text that represents the left boundary, copy it into the web_reg_save_param (or web_create_html[_ex]) statement. If it contains any carriage returns, place it all on one line. If there are any quotes (“) in the text, place the escape character before it so that the replay does not incorrectly think it is the end of the parameter, but rather a character to search for. For example, if the left boundary was 'input type=hidden name=userSession value=' (without the single quotes) and you are using the web_reg_save_param statement, then the function you have so far would be web_reg_save_param (“UserSession”, “LB=input type=hidden name=userSession value=”,

Step 3 You are now going to tell the replay engine how to identify the end of the value you are trying to capture, that is, the right boundary of what you are looking for. Again, look in the Execution log and copy the static text that appears to the right of the dynamic value you are looking for. For example, let’s say the Execution log contained the following: … userSession value=75893.0884568651DQADHfApHDHfcDtccpfAttcf>… Then, the example so far to save the number into the parameter UserSession would be web_reg_save_param(“UserSession”, “LB=input type=hidden name=userSession value=”, “RB=>”,LAST);

In choosing a right boundary, make sure you choose enough static text to specify the end of the value. If the boundary you specify appears in the value that you are trying to capture, then you will not capture the whole value.

Recap That was a lot of looking through the Recording and Execution logs and checking of values. Let’s just recap what you have done. You have identified a value that you think needs to be correlated. You then identified in the script where to place the statement that would ultimately capture and save the value into a parameter. You then placed the statement, and gave the text strings that appear on either side of the value that you are looking for so that it can be found. The flow of logic for this is the correlation functions tell the replay engine, what to look for in the next set of replies from the server. The replay engine makes a request of the server. The server replies. The replay engine looks thorough the replies for the left and right boundaries. If it finds them, what is in-between is then saved to a parameter of the name specified. Remember, the parameter cannot have a value until after the next statement is executed. The correlation statement only tells the replay what to look for; it does not assign a value to the parameter. Assignment of a value to the parameter does not happen until after generating request to the server and looks in the reply. If you have a case where a correlation statement is followed by a function that attempts to use the parameter, the statement is in the wrong place, and the script will fail. This is always incorrect: web_reg_save_param(…); Web_submit_data (… {Parameter}…); In-between the two, there needs to be a request of the server that causes it to reply with the value you are trying to capture.

Replacing the hard coded value in the script with the parameter Once you have created the parameter, the next step is to replace the hard coded occurrences with the parameter. Look through the script for the original value. Where you find it, delete the value and replace it with the parameter. Note, only the value you want replaces is deleted. The characters around it remain. EXAMPLE: Change: … "Name=userSession", "Value=75893.0884568651DQADHfApHDHfcDtccpfAttcf", , …

.....

To: … "Name=userSession", "Value={UserSession}", ENDITEM, …

At this point, you are ready to run the script to test if it works, or, if it needs further correlation, more work on this correlation.

Common errors when correlating When the replay fails to find the boundaries for a web_reg_save_param statement, it will print a warning message in the execution log like this: Action1.c(x): Error: No match found for the requested parameter "Parameter Name". If the data you want to save exceeds 256 bytes, use web_set_max_html_param_len to increase the parameter size Action1.c(x): web_url highest severity level was "ERROR", 47489 body bytes, 1518 header bytes By default, web_reg_save_param function issues an error if the boundaries cannot be found. If you need to use this function for other purposes, and would like to avoid errors, use the "Notfound=warning" attribute so that the replay will just issue a warning. Now, the replay actually suggested 2 possible causes for the error, 1. The boundaries defined cannot be found 2. The data you want to save exceeds 256 bytes The advice given are just recommendation that you must take into consideration. Was the value you were trying to capture more than 256 characters long? In the above example it was only 40 characters long. Have a look at the Recording log and see how long the original value is. Have a look at the second recording made earlier and see how long it was in that script. Turn on the extended log (Run-Time Settings → Log → Extended log → All data returned from server) and have a look at how long it is in the Execution log. If all occurrences of the script were much shorter that the maximum parameter length, then the problem is either with the web_reg_save_param in the wrong place, or that the boundaries are incorrect. Go back and look at the boundaries that you selected, look at the placement of the web_reg_save_param. Is it immediately before the statement that causes the server to reply with the data you are looking for? If, at any time, any of these values was close to being around 200 characters, or exceeded 200 characters, then add a web_set_max_html_param_len statement to the start of the script to make the maximum longer than 256 characters. For example, web_set_max_html_param_len ( “length” ); This statement tells the replay engine to look for larger matches for the left and right parameter. When it finds the left boundary, it will look ahead the max parameter length for the right boundary. This setting is script wide and takes

effect from when it is executed; it only needs appear in the script once. Looking for longer matches uses up more memory and CPU processing to search through the returned text from the server. For this reason, do not set it too high, or you will be making your script less scalable. That is you will reduce the number of Vusers that can run it on a given machine. Try to have the maximum parameter length no more than 100 characters greater that what you are expecting.

Special cases for the boundaries: There are some special characters and cases when specifying the boundaries. Double quotes should be preceded by a “\” so that it can be recognizes as part of the string to look for. If your text includes any carriage returns, that are part of the http, and not just part of the wrap around in the recording log, these need to be specified as a “\r\n” character set. If the “\” character is part of the text, it too needs to be preceded by a “\” to indicate it is a literal. Recording log Left boundary Right boundary Value=”57685” “Value=\”” “\”” Value_”\item\”value’7875’ “Value_\”\\item\\\”value’ “’” Value= “7898756”

“Value=\r\n\””

“\””

Debug help Sometimes you want to print out the value that was assigned to a parameter. To do this, use the lr_eval_string function and the lr_output_message function. For example, to print the value of the parameter to the Execution log: lr_output_message(“ Value Captured = %s", lr_eval_string("{ParameterName}")); If you find that the value being substituted is too long, too short, or completely wrong, printing out the value will help identify the changes you need to make to the correlation function. If you have extra characters at the start of the value, you need to add them to the end of the left boundary, if you have extra characters at the end of the parameter value, you need to add them to the start of the right boundary. If you are getting the wrong value all together, do searches in the Recording log for the left boundary, make sure that you have a unique boundary and that LoadRunner is not picking up an earlier occurrence. You can then use the ORD=x attribute to specify which occurrence you want to capture if the boundary is not unique.

Other Correlation help resources The examples in the Function reference contain a lot of data and examples on how to use these functions. I would recommend looking over them. The Customer Support site has a video for download that goes over correlation. You can get it from http://support.merc-int.com. After logging in, go to Downloads → Browse. Select the Mercury Interactive downloads radio button, choose “LoadRunner” from the product selection dropdown box, and click on the “Retrieve” button. Under “Training,” select the “LoadRunner Web Script Correlation Training” link.