Purpose

Test Trend is a metric originated by IBM to help teams understand whether their test activities are falling behind plan. 

Definition

Count for each unit of time monitored:

• total number of planned test cases
• total number of attempted test cases
• total number of completed test cases

Because not all test cases are equally important, you can assign weight to the test cases using a relative point system. Monitor test points for this metric instead of the number of test cases. IBM uses a 10 point scale where 10 is the most important and 1 is the least.  Points are assigned during test planning.

Test cases can be grouped into functional and system tests categories.

Analysis

Create a Test Trend report, plotting units of time on the X axis and test cases or test points (cumulative) on the Y axis.  Plot a line for  the number of planned test cases, the number of attempted test cases, and the number of completed test cases.

By tracking against the plan, it is easy to identify when testing activity is falling behind. This typically happens to projects under schedule pressure. The expected trend is an S-curve. However, if the S-curve is steep, the team should challenge the feasibility of the plan

Establish a recommended 15% threshold between successful and planned test cases to trigger corrective actions as necessary.

For a large project with hundreds or thousands of test cases it is helpful to monitor at the component level for better visibility. Consider using a tabular format to report on this metric.

Watch out for cases where overall testing activity progress is on track, but a particular component is falling behind schedule.

Frequency and reporting

Monitor Test Trend weekly to ensure that testing activity issues are discovered quickly. Frequent monitoring can reduce the risk of schedule slippage.

Collection and reporting tools

Test activity is captured in IBM® Rational® Quality Manager® and reported in the Test Execution Trends report.