A Sample Test Summary Report

Test Summary Report

A test summary report is a Quality work product / Test Document that formally summarizes the results of all testing.

Test Lead or Test Manager prepares this document at end of the Testing, means in Test Closure phase (Last phase in STLC/Software Test Process.

Purpose:

To enable Project management and Customer to know the status of testing status of the project and Application Quality Level.

Advantages:

All stake holders of the Project able to get project test status, Application Quality status and they can take corrective actions (if required)

Guidelines:

•    A Test summary report can be generated for every software release
•    It should have metrics, charts and table forms, if possible
•    It has to summarize all test activities based on Quality work products.

Test Summary Report Template

Introduction:

Test Items:

Reference documents

Target Audience

Test Summary

       a) Test Suite Information:
           •    Number of test suites planned.
           •    Number and percentage of test suites implemented.
           •    Number and percentage of test suites executed.
       
       b) Test Case Information:
           •    Number of test cases planned.
           •    Number and percentage of test cases implemented.
           •    Number and percentage of test cases executed.
           •    Number and percentage of test cases passed.
           •    Number and percentage of test cases failed (total and by severity).

     c) Defect Report Information

Number of defects found in comprehensive Testing

Number of Test Cases selected for Regression Testing Cycle1
Number of defects found in Regression Testing Cycle1

Number of Test Cases selected for Regression Testing Cycle 2
Number of defects found in Regression Testing Cycle 2

Number of Test Cases selected for Regression Testing Cycle 3
Number of defects found in Regression Testing Cycle 3
.
.
.
Number of Test Cases selected for Final Regression.
Number of opened defects in this release.

Approvals
------------------------------------------------------------------------------------------

A Sample Test Summary Report

Introduction:
It is for Internet Banking System Application Version 2.0, IBS (Internet Banking System) has Information, Personal banking, Corporate Banking, Master Data, User Management and Reports modules already. Now in this release some features added to Personal Banking Module and One New module “Small Business” added.

Test Items:

Personal Banking
New Business
Reports

Reference documents:

Test Plan, Test Case documents, Opened and Closed Defect Reports, Metrics docs, Review Reports.

Target Audience:

Project Manager, Release Team, Maintenance Team and Customer (End Users).

Test Summary

a) Test Suite Information:

•    Test suites planned: 42
•    Test suites implemented: 39
•    Test suites executed: 38

b) Test Case Information:

•    Test cases planned: 983
•    Test cases implemented: 978
•    Test cases executed: 967
•    Test cases passed: 940 (Final)
•    Test cases failed: 14 (Final)
•    Test Cases pending: 3

c) Defect Report Information

Defects found in comprehensive Testing: 148
Test Cases selected for Regression Testing Cycle1: 320
Defects found in Regression Testing Cycle1: 96
Test Cases selected for Regression Testing Cycle 2: 213
Defects found in Regression Testing Cycle 2: 42
Test Cases selected for Regression Testing Cycle 3: 107
Defects found in Regression Testing Cycle 3: 26
.
.
.
Test Cases selected for Final Regression: 512
Number of opened defects in this release: 14

Software Test Documentation Templates

I) Test Policy (Company Level Document)

1) Test Definition             
(Verification & Validation)

2) Testing Process
(Proper planning before starts testing)

3) Testing Standards

4) Testing Measurements
(QAM, TTM, PCM)

5) Risks

6) Approvals

7) Comments

8) Revision History
--------------------------------------------------------
II) Test Strategy (Company Level Document)
  Table of Contents
1.0    Introduction
2.0    Objective
3.0    Scope of the Document
4.0    Test Approach
4.1              Introduction
4.2              Functional and Usability Issues
5.0        Phases of Testing
5.1              Documentation Review
5.2              Unit /Module and Integration Testing
5.3              System Testing
5.4              Pre – Production Testing
5.5              Responsibilities
6.0      Test Management
6.1              Test Procedures
6.2              Test Report
6.3              Test Phase Acceptance
6.4              Timescales
7.0       Risks and Assumptions
7.1               Risks
7.2               Issues
7.3               Assumptions
7.4               Related Documents
8.0      Revision History -------------------------------------------------------- III) Business Requirements Specification
  Table of Contents
1.0    Introduction
1.1    Purpose
1.2    Scope
1.3    Background
1.4    Assumptions
1.5    Constraints
1.6    Dependencies
1.7    Users of the product
2.0    Functional Requirements
3.0    Usability Requirements
3.1    Help
3.2    Navigation
3.3    Feedback
3.4    Look and Feel
4.0    Performance Requirements
4.1    Capacity
4.2    Availability
4.3    Latency
5.0    Security Requirements
5.1    Protection
5.2    Authorization and Authentication

6.0    Revision History -------------------------------------------------------- IV) Software Requirements Specification

Table of Contents
1.0    Introduction
1.1    Purpose
1.2    Scope of the project
1.3    References
2.0    System Requirements Specification
3.0    Functional Requirements Specification
3.1    Use Cases
3.2    Functional Requirements – Online Catalog
3.3    Functional Requirements – Administration
4.0    Non Functional Requirements
4.1    Usability Requirements
4.2    Performance Requirements
4.3    Compatibility Requirements
4.3.1    Operating Systems
4.3.2    Browsers
4.4    Security Requirements

5.0    Revision History  -------------------------------------------------------- V) System Test Plan
  1) Test Plan ID:
Some type of unique company generated number to identify this test plan.

2) Introduction:
Describe the purpose of the Plan, possibly identifying the level of the plan (System Test Plan etc.). This is essentially the executive summary part of the plan.

3) Test Items:
These are things you intend to test within the scope of this test plan.

4) References:
List all documents that support this test plan. Refer to the actual version/release number of the document as stored in the configuration management system.

5) Features to be Tested:
This is a listing of what is to be tested from the Users viewpoint of what the system does. This is not a technical description of the software, but a Users view of the functions.

6) Features not to be tested:
This is a listing of what is NOT to be tested from both the Users viewpoint of what the system does and a configuration management/version control view. This is not a technical description of the software, but a Users view of the functions.

7) Test Approach:
This is your overall test strategy for this test plan; it should be appropriate to the level of the plan (master, acceptance, etc.) and should be in agreement with all higher and lower levels of plans. Overall rules and processes should be identified.

8) Entry Criteria:
It describes when to start Testing

9) Exit Criteria:
It describes when to stop testing

10) Suspension Criteria:
It describes when to stop testing temporarily.

11) Roles & Responsibilities
Team Lead or Test Lead and Team members Roles and Responsibilities.

12) Schedule:
Schedule for all Test activities in this Software Test Process.

13) Training:
Training on the application/system (Domain Training)

Training for any test tools to be used.

14) Test Environment / Lab:
It describes Require Hardware and software for setting-up Test Environment or Test Lab.

15) Test Deliverables:
Lists out that what is to be delivered as part of this plan?

16) Approvals
Who can approve the process as complete and allow the project to proceed to the next level.

17) Glossary:
Define terms and acronyms used in the document, it can be used to understood the terms used in this plan.
--------------------------------------------------------
VI) Requirements Traceability Matrix
  1) Serial No
2) Type of Requirement
3) Requirement Source
4) Requirement ID
5) Requirement Description
6) Test Case ID
7) Test Case Description
8) Test Log
9) Defect ID
10) Defect Description
11) Requirement Status
12) Software Version
13) Comments --------------------------------------------------------
VII) Test Scenario
  1) Scenario ID

2) Scenario Description

3) Module Name/ID

4) Version Control

5) Requirement/s Reference

6) Comments
-------------------------------------------------------- VIII) Test Case
  1) Test Case Id: a Unique name/number (Alfa-numeric)

2) Test Case Name: Name of Test Case

3) Test Suite ID: Unique name/number (Alfa-numeric)

4) Pre-Condition: Status before Test Case Execution

5) Steps: Steps for Executing the Test Case

6) Post-Condition: Status After Test Case Execution

7) Expected Result: Expected Result as per Requirements

8) Actual Results:

9) Test Results: Pass / Fail

10) Remarks: Comments (Optional)

-------------------------------------------------------- Note 1: You prepare this Test Case Template in Excel Sheet

Note 2: Test Case Template may vary from one company from another and one project to another.

Note 3: In the above template Actual Results and Test Results fields can be filled in Test Execution phase, Remaining fields in Test Design phase. -------------------------------------------------------- IX) Test Data

X) Test Metrics
-------------------------------------------------------- XI) Defect Report

1) Defect Id: any unique name for Identifying the Defect (Alfa numeric)

2) Defect Description: Details about the Defect

3) Test Case Id: Corresponding Test Case Id for tracking

4) Tester: Tester's name (who found the Defect)

5) Product Version: Version of the Product on which defect was found

6) Build Version: Version of the Build on which defect was found

7) Priority: Importance of the Defect based on Business /Customer

8) Severity: Importance of the Defect based on Functionality

9) Status: Status of Defect

10) Reproducible or not: Yes / No

    If Reproducible:

      Steps:

    If not Reproducible:

      Attachments

11) Reporting to: Corresponding Developer

12) Remarks : Comments (Optional) -------------------------------------------------------- XII) Test Summary Report
  1) Introduction:

2) Test Items:

3) Reference documents

4) Target Audience

5) Test Summary

       5.1) Test Suite Information:

           •    Number of test suites planned.

           •    Number and percentage of test suites implemented.

           •    Number and percentage of test suites executed.

       5.2) Test Case Information:

           •    Number of test cases planned.

           •    Number and percentage of test cases implemented.

           •    Number and percentage of test cases executed.

           •    Number and percentage of test cases passed.

           •    Number and percentage of test cases failed (total and by severity).

     5.3) Defect Report Information

    Number of defects found in comprehensive Testing

    Number of Test Cases selected for Regression Testing Cycle1

    Number of defects found in Regression Testing Cycle1

    Number of Test Cases selected for Regression Testing Cycle 2

    Number of defects found in Regression Testing Cycle 2

    Number of Test Cases selected for Regression Testing Cycle 3

    Number of defects found in Regression Testing Cycle 3

    .

    .

    .

    Number of Test Cases selected for Final Regression.

    Number of opened defects in this release.

6) Approvals