Artifact: Test Suite |
This artifact defines a collection of related tests.
Domains: Test
Purpose
To manage and sequence the execution of tests. To provide a useful and related set of test log information from which test results can be determined.
Relationships
Roles | Responsible: Test Designer | Modified By: Test Designer
Tasks | Input To: Analyze Test Failure Run Tests Structure the Test Implementation | Output From: Structure the Test Implementation
Description
Main Description | The Test Suite provides a means of managing the complexity of the test implementation. Many system test efforts fail because the team gets lost in the minutia of all of the detailed tests, and subsequently loses control of the test effort. Similar to UML packages, Test Suites provide a hierarchy of encapsulating containers to help manage the test implementation. They provide a means of managing the strategic aspects of the test effort by collecting tests together in related groups that can be planned, managed, and assessed in a meaningful way.
Brief Outline | Each Test Suite needs to consider various aspects, including the following: Compatibility and relevance of the individual tests to be executed by the Test Suite, especially in terms of test objective and scope Points from which the Test Suite can be recovered or resumed if execution is halted Required configuration settings for the Test Suite, of both hardware and software (for example, video-display resolution, resource allocation, environment variables, and so on) Pre-existing consumables required by the Test Suite, such as populated data sets, receipt printer dockets, and so on
Illustrations
Reports | Test Survey
Tailoring
Impact of not having | Without this artifact no test execution timing and sequencing information will be captured. This might lead to inconclusive tests and a lack of consistency in the way tests are performed.
Reasons for not needing | This artifact is recommended for most projects, with the exception of very special cases where the test cases and scripts are completely independent and there are not used in combination.
Representation Options | This work product represents a container for organizing arbitrary collections of related tests. This may be realized (implemented) as one or more automated regression Test Suites, but the Test Suite can also be a work plan for the implementation of a group of related manual tests. Note also that Test Suites can be nested hierarchically; therefore, one Test Suite may be enclosed within another. Sometimes these groups of tests relate directly to a subsystem or other system design element, but other times they relate directly to things such as quality dimensions, core mission critical functions, requirements compliance, standards adherence, and many others concerns that cut across (or are not directly related to) the internal system elements. You should consider creating Test Suites that arrange the available Test Scripts (in addition to other Test Suites) in many different combinations: the more variations you have, the more you will increase coverage and the potential for finding errors. Give thought to a variety of Test Suites that will cover the breadth and depth of the target test items. Remember the corresponding implication that a single Test Script (or Test Suite) may appear in many different Test Suites. Some test automation tools provide the ability to automatically generate or assemble Test Suites. There are also implementation techniques that enable automated Test Suites to dynamically select all or part of their component Test Scripts for each test cycle run.
More Information
Concepts | Testing Qualitative Requirements
Guidelines | Maintaining Test Suites
© Copyright IBM Corp. 1987, 2009. All Rights Reserved. © Copyright ABC Corp. 2009. All Rights Reserved. Includes materials licensed from IBM and Eclipse. See IBM copyright and Eclipse copyright .