Performance Testing, LoadRunner Tips&Tricks

This site is moving to a bigger space @ LoadRunner TnT

There are couple of times when an individual starts to pick up LoadRunner ask this question. I've written the explanation here for the convenience of the newbies which I had previously responded in the Yahoo! group discussion.


Max. Running Vuser

It defines at any point of point of time, the maximum number of vusers running together concurrently (in Run state). This is the "state" or usually the requirement of a load test to reach "X" number of concurrent users. If a load test were to required to run 100 concurrent users, then the Max. Running Vuser must be 100. This is different from Vuser Quantity explained in the following.


Vuser Quantity
In the Controller, the Vuser Quantity is the total number of participating in the load test but it is different from Max. Running Vuser which is explained above. To view the total number of Vuser participating the load test (Vuser Quantity), open the Vuser Summary graph. Please take note that the Summary Report in the Analysis session displays the maximum number of vusers running in the scenario and not the total vuser particiapting.
Related Topics

Labels: , ,


Bookmark this article now! AddThis Social Bookmark Button



technorati del.icio.us reddit digg

Protocols: Java Error - java.lang.NoClassDefFoundError

I had a question from a visitor with regards to a Java error. The error message is as followed.

java.lang.NoClassDefFoundError: jhook/HookMetaData
at java.rmi.Naming.lookup(Naming.java)
at pie.engine.atp.ClientRMI.getRemoteModuleImpl(ClientRMI.java:256)
at pie.engine.atp.ClientATPRemoteEx.getRemoteModule(ClientATPRemoteEx.java:89)
at pie.engine.atp.ClientATPRemoteEx.getRemoteModule(ClientATPRemoteEx.java:64)
at pie.engine.atp.VirtualRemoteServer.setupPingRemote(VirtualRemoteServer.java:102)
at pie.engine.atp.VirtualRemoteServer.pingServer(VirtualRemoteServer.java:166)
at pie.engine.atp.CacheTimer$1.run(CacheTimer.java:47)
at java.util.TimerThread.mainLoop(Timer.java:432)
at java.util.TimerThread.run(Timer.java:382)


I did not have straight answer for him initially but, he was able to resolve the problem by himself with the following command script (inclusion of the Xbootclasspath).

-Xbootclasspath/p:"D:\Program Files\LoadRunner\classes;D:\Program Files\LoadRunner\classes\srv"

Once that was added, the script worked for him! Well hope this is useful to anyone that is encountering the same problem.


Related Topic

Working with CORBA-Java
Working with RMI-Java
Working with Java - Introduction

Labels: , , ,


Bookmark this article now! AddThis Social Bookmark Button



technorati del.icio.us reddit digg

Monitors: The Other Way? II

The organisation that I worked in have frequent port clearance to perform prior the load test and this is often a troublesome matter to do. As you know that for monitoring to be successful, it requires certain ports to be opened as described in the article titled, "Monitors: How does the monitoring work?". So for this requirement, port-clearance becomes part and parcel of the performance tester job (or for me in particular). This kind of situation exists for some other organisations too.
With that, I introduce you to monitor in an alternative way by monitoring locally on the machine and logging the data. The monitoring fundamental for LoadRunner is almost the same for all monitors, as such, you can collect the performance on the machine locally then port it to Analysis to be imported. Analysis provide an import feature that allows external monitor to be ported into the results of the load test.
The import feature is straight forward in the usage. More information (a short one) can be found in the Analysis User Guide that comes along with every installation of LoadRunner. I would advise the data to be stored into a .csv file for easier import as Analysis can extract the data easily.
An example of collecting data on the local machine for Windows can be found in the article, "How to create a log using System Monitor in Windows". Another article titled, "How to create a Performance Monitor log in Windows NT". I will come up an example for UNIX in my future post.
Somethings to note, the timing of the graphs may deviate more than often. What I suggest is try to synchronize the timing of the local machine to the load test results for an accurate reading whenever possible.
Related Topics

Labels: , , , ,


Bookmark this article now! AddThis Social Bookmark Button



technorati del.icio.us reddit digg

General: Maximum amount of Vusers generated by a single load generator

How many vusers can a Load Generator generate?
There is a straight forward answer in this KB article, KB 11808, "How many Vusers can run on a given computer" from Mercury/HP. Also, you can refer to the following article, KB 3629, "Recommended hardware and OS tuning for optimal Vuser performance" to get information of recommended hardware for vuser performance. As per metioned in the articles, the question of maximum load relates closely to the CPU and Memory on a given hardware.

Before we dwell deeper into the technical details, it is best to ensure that common blunders are removed, such as, (1) the Controller have sufficient license to run the amount of vusers stated in the Scenario. (2) Furthermore, ensure that there are no limits enforced on the Load Generators with are configured in the Scenario Settings.

To start with, we must understand what happens during a scenarion execution. Refer to General: Scenario Execution for an understanding of the scenarion execution. Furthermore, when vusers are executed in a scenario, the vusers are actually running in memory instead of the disk. That is why memory is an important factor to consider.
Also, the article, KB 10291, "Vuser Memory requirements - memory footprints" provides a list of memory footprint of different types of protocols. This is only a measurement of a simple script of its kind but it can be used to estimate roughly the amount of load generated on a given hardware specification.
For Citrix, SAP GUI and RTE, it is closely related to GDI resources. GDI resources is a hard-coded value in the OS and cannot be changed. For more information, you can refer to the Microsoft article on GDI Objects. To monitor GDI resource, you can refer to KB 30684, "How to monitor the GDI resources used".
If you like to maximize the amount of load that can be generated, you can refer to the following article, KB 3645, "How to increase the number of Vusers you can run on a Windows Machine" by Mercury/HP. However, this is not applicable to GDI resources-related protocols.
WIth all suggestions above, the best bet to determine the amount of load generated, is to monitor the Load Generators itself (instead of the machines under load in a load test).

Take note that the above Mercury/HP links required a valid login to their support site. Please apply an account with them first before you click on them.
Related Topics

Labels: , , , ,


Bookmark this article now! AddThis Social Bookmark Button



technorati del.icio.us reddit digg

Basics: Transaction Response TIme

What is Transaction Response Time?

Transaction Response Time represents the time taken for the application to complete a defined transaction or business process.


Why is important to measure Transaction Response Time?

The objective of a performance test is to ensure that the application is working perfectly under load. However, the definition of “perfectly” under load may vary with different systems.
By defining an initial acceptable response time, we can benchmark the application if it is performing as anticipated.

The importance of Transaction Response Time is that it gives the project team/ application team an idea of how the application is performing in the measurement of time. With this information, they can relate to the users/customers on the expected time when processing request or understanding how their application performed.


What does Transaction Response Time encompass?

The Transaction Response Time encompasses the time taken for the request made to the web server, there after being process by the Web Server and sent to the Application Server. Which in most instances will make a request to the Database Server. All this will then be repeated again backward from the Database Server, Application Server, Web Server and back to the user. Take note that the time taken for the request or data in the network transmission is also factored in.


To simplify, the Transaction Response Time comprises of the following:
  1. Processing time on Web Server
  2. Processing time on Application Server
  3. Processing time on Database Server
  4. Network latency between the servers, and the client
The following diagram illustrates Transaction Response Time.

Figure 1 Transaction Response Time = (t1 + t2 + t3 + t4 + t5 + t6 + t7 + t8 + t9) X 2
Note:
Factoring the time taken for the data to return to the client.


How do we measure?

Measuring of the Transaction Response Time begins when the defined transaction makes a request to the application. From here, till the transaction completes before proceeding with the next subsequent request (in terms of transaction), the time is been measured and will stop when the transaction completes.


Differences with Hits Per Seconds

Hits per Seconds measures the number of “hits” made to a web server. These “hits” could be a request made to the web server for data or graphics. However, this counter does not represent well to users on how well their applications is performing as it measures the number of times the web server is being accessed.

How can we use Transaction Response Time to analyze performance issue?

Transaction Response Time allows us to identify abnormalities when performance issues surface. This will be represented as slow response of the transaction, which differs significantly (or slightly) from the average of the Transaction Response Time.

With this, we can further drill down by correlation using other measurements such as the number of virtual users that is accessing the application at the point of time and the system-related metrics (e.g. CPU Utilization) to identify the root cause.

Bringing all the data that have been collected during the load test, we can correlate the measurements to find trends and bottlenecks between the response time, the amount of load that was generated and the payload of all the components of the application.


How is it beneficial to the Project Team?

Using Transaction Response Time, Project Team can better relate to their users using transactions as a form of language protocol that their users can comprehend. Users will be able to know that transactions (or business processes) are performing at an acceptable level in terms of time.

Users may be unable to understand the meaning of CPU utilization or Memory usage and thus using a common language of time is ideal to convey performance-related issues.


Related Topics:


Labels: , ,


Bookmark this article now! AddThis Social Bookmark Button



technorati del.icio.us reddit digg

Products: SiteScope

SiteScope is another Mercury/HP product that provides agentless monitoring to the users. Agentless monitoring in this context means that there is no need to have additional installation of components to the target servers to perform monitoring.

Similar to the LoadRunner Controller, SiteScope requires the designated server to be able to perform monitoring of its own which SiteScope will be able to draw data from it. Depending on the user, they have a choice of using SiteScope to perform the monitoring or use the Controller.

Below is a list of links written by Charles Laudia that provides SiteScope-related information.

What is SiteScope?

What are SiteScope components?


Related Topics

Labels: , ,


Bookmark this article now! AddThis Social Bookmark Button



technorati del.icio.us reddit digg

Working with Web (HTTP/HTML) V: File Download/Export

Frequently, I've observed that there are always questions relating to file download functionalities or file export not being recorded by Vugen for Web (HTTP/HTML) protocol in Web applications, such as clicking a link and the file will be downloaded.
In this article, I will describe the possibilities of the problem based on my experiences. We will elaborate using a Winword file.
When you click on the link or button to access the file, the file gets loaded in the browser. From here, the business process requires you to click on a link to export it or save it to the local machine.
There are two possibilities that will happen. The Winword is either saved over the network or at the OS level.
In the context of network level, the client makes a request to the server (as per norm) to download the file which translates to network traffic going to and fro between the client and server. This will be recorded.
In the context of OS level, the client issues a OS command to "save-as" the file from a temporary location (e.g. Temporary Internet Folder) to a defined location. Why is that so? Sometimes, the file is actually saved in the temporary folder in the first file download request which is viewed in the browser and the subsequent "export" command is actually an OS command to retrieve the file from the temporary location. This usually happens for Web applications that requires another application, in this example (Winword Viewer) to be launched prior the file to be viewed in the browser.
Therefore, in the two situations, you may want to know when the file is actually downloaded. And for this, I would recommend turing on the Full Extended Log to dig further. Usually for file downloads, the data should be represented in mixed chunk of characters (or what we defined them as binaries) in the Full Extended Log.
However, before you start with digging the logs, I would also recommend you to look at this previous article, "Protocols: Working with Web (HTTP/HTML)" that have recording solutions for Web (HTTP/HTML) protocols. This will at least scope the problem to application related or configuration setup in Vugen.
Related Topics

Labels: , , , , ,


Bookmark this article now! AddThis Social Bookmark Button



technorati del.icio.us reddit digg

General: Virtualization with LoadRunner

There was the question posted in the Yahoo discussion group on the topic of virtualization with LoadRunner. That is to use virtualization tools such as VmWare and its support and compatibility with LoadRunner.
James Pulley who often posts in the group came together a list of recommendations with regards to his experience in such a topic here. The recommendations consist of what could be installed on the virtual machines. I personally appreciated that he put in these information.
I believed that the recommendations will be very useful for individuals or organisations looking for some answers with the proliferation of the virtualization tools and therefore extracted the content of the discussion and publish here.
The recommendation by James were in order of the least influenced to the most influenced by virtualization.
Analysis
The core issue associated with virtualization is the state of the virtualized system clock which is inconsistent in operation, so much so that time critical events do not illustrate reliable behavior Mercury Analysis. This is a very easily virtualized tool. There are no time critical components associated with the analysis engine, since all of the data has been collected by other hosts during the execution of the performance test. This virtualized instance works best when the data store for the testr esults is in a Microsoft SQL Server database as opposed to the default datastore of a Microsoft Access formatted data file. With the change inl icensing with 8.1, this item can be promiscuous in its distribution and is no longer limited to just a single copy with a controller purchase.
LoadRunner Virtual User Generator
As the workbench for the script development, there are very few time critical events associated with the development and the testing of scripts. Virtualized forms of VUGEN work best on platforms which have hardwarevirtual machine extensions, such as the latest versions of the Intel CoreDuo processor and their AMD equivalents. Hyperthreaded machines seem to show in consistent performance as well - There is a knowledge base article on this hyperthreaded issue in the Mercury knowledge base. Timing may be a little off in the log due to the clock issue noted above, but other than thisissue, the VUGEN operates very well in a virtualized form. As with the Analysis engine, with the change in licensing with 8.1, this item can be promiscuous in its distribution and is no longer limited to just a singlecopy with a controller purchase.
LoadRunner Controller
As we move into the controller timing becomes a more critical item. The controller itself can be virtualized without an issue, but because thetiming record information collected is as an offset to the beginning of thetest, an inconsistent clock on the controller can become an issue formonitor data with offset timestamps may not reflect the actual system clock.
Load Generator/Injector
This is a layer which suffers from the most influence from the inconsistent clock issue for time critical events within a virtual machine and of all ofthe layers, this is the one which should be virtualized the least. The best test design would dictate that all of your generators have an identical hardware and software profile, a very difficult item to achieve invirtualized form which suffers from a virtualized CPU and a virtualized system clock.This time critical items is not something which would just impact LoadRunner. It also impacts other time critical processes, such as inability to run some functions for conferencing in virtualized PBXs.
Related Topics:
References extracted from the discussion topic:

Labels: , , , ,


Bookmark this article now! AddThis Social Bookmark Button



technorati del.icio.us reddit digg

Protocols: Working with CORBA-Java

The blog by Stuart Moncrieff has great resources on load testing and particularly one of them is CORBA-Java. It comes really handy with a downloadable document at the end of the article and when Java-related protocol resources are hard to find.

The article "Tips for Load Testing CORBA Applications with LoadRunner" covers recording, replaying, workarounds and scripting. Click on the link here to be redirected.

Related Topics

Content Page - Protocols


Labels: , , ,


Bookmark this article now! AddThis Social Bookmark Button



technorati del.icio.us reddit digg



Powered by Google

Enter your email address:

Delivered by FeedBurner


Add to Technorati Favorites


XML

Powered by Blogger

make money online blogger templates

Powered by FeedBurner

Blog Directory

Top Blogs

Software Blogs -  Blog Catalog Blog Directory





© 2007 Performance Testing, LoadRunner Tips&Tricks | Blogger Templates by GeckoandFly.
No part of the content or the blog may be reproduced without prior written permission.
Learn how to make money online | First Aid and Health Information at Medical Health