In a virtualized environment, the load generating capacity can be degraded significantly
for throughput, CPU utilization, and measurement accuracy. For instance, in a cloud environment,
measuring the response time can vary, depending on factors such as the location of data center, type
of host, and life-span of agent virtual machines. It is difficult to get an accurate response time
every time because not all factors can be controlled by IBM® Rational® Performance Tester. But, you
can do a statistical comparison of the reported measurements against a trusted control.
Important: For a cloud run, it is very important to audit
measurement results using the control techniques described in this
topic.
IBM Rational Performance Tester provides
two options to audit the reported response time accuracy:
- Apply a known fixed delay with the unknown response times and
observe deviations from the known correct value as a 'proxy' for response
time measurement accuracy.
- Distribute a subset of the full load on a non-compromised controlled
agent virtual machine that runs in parallel and measure the deviations.
The first approach automatically generates an audit test in the cloud schedule. You can assign
extra audit users to run the audit test on every control and load agent locations. These audit users
are 'dummies' that do not apply load to the system under test. You do not have to account for them
when filling out the Stage page of the wizard.
The second approach is recommended for auditing measurement accuracy because a control agent is
expected to be run on a dedicated computer. For a control agent that is provisioned with BareMetal,
the internet latency and congestion will be the same as the other cloud agents from the same data
center. So, the differences will be only due to CPU utilization, memory, and Network Interface Card.
Note: - Cloud-based control agents cannot detect data center or physical host bandwidth
bottlenecks.
Consider using an on-premises location for the control agent. For a control agent that is
on-premises the internet latency and congestion factors count for response time. The number of
virtual users in a control agent should be low enough to be able to have data to audit measurement,
typically 10-20% of the total capacity. The load should not be more than 30% of the capacity. At the
same time, the number of users should be high enough to get sufficient statistical sampling for each
stage, typically 100 samples for every counter of interest.
Note: If some of the user groups have low
percentage Group sizes, the number of control users running the tests is
reduced. You can, however, counterbalance the reduction by running longer stage durations to get
more samples for each user.