Testing Framework
This document was uploaded by user and they confirmed that they have the permission to share it. If you are author or own the copyright of this book, please report to us by using this DMCA report form. Report DMCA
Overview
Download & View Testing Framework as PDF for free.
More details
- Words: 3,118
- Pages: 22
Controlled copy
Framework for Testing
Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
C3: Protected
Controlled copy
Testing Framework
Table of Contents 1.0 Introduction............................................................................... .............................4 2.0 Definitions and Acronyms............................................................. ........................5 2.1 Acronyms...................................................................................................... .....................5 2.2 Definitions........................................................................................................ ..................5
3.0 Purpose...................................................................................................... .............6 What is not the intended use for the Framework?................................... ........................6 What then defines the process for the projects?....................................................... .......6 How should the Framework be used?................................................... ..........................6
4.0 Organization of the Framework.................................................................. ...........7 4.1 Section A – Process Disciplines.................................................................................. .....7 4.2 Section B – Standard PSPH ................................................................................ .............7
5.0 Section A - Process Disciplines................................................. ..........................9 5.1 Test Requirements Study ........................................................................................ .........9 5.1.1 Test Requirements Study (AWR)............................................................................ ......9 Scoping........................................................................................................................ ....9 Requirements Gathering ............................................................................................ .....9 Identification of Estimation Methodology............................................... ..........................9 Estimation ....................................................................................................... ............10 Business Process Understanding ................................................................... ..............10 Application Understanding.................................................................................. ...........10 Tools Understanding......................................................................................... .............10 Customer Testing Process Understanding........................................................... ..........11 Infrastructure requirements Understanding............................................................. .......11 Customer Performance Baseline Collection.................................................................. .11 Traceability matrix for Test Requirements........................................................... ...........12 Knowledge Transition to Offshore........................................................................ ..........12 5.2 Test Strategy.............................................................................................. .....................12 5.2.1 Test Strategy (AWR)........................................................................................... ........12 Tool Evaluation and Selection (in case of automated testing).................................... ....12 Testing Tools Installation (in case of automated testing).......................................... ......12 Proof Of Concept (in case of automated testing).............................................. .............13 Test Strategy formulation................................................................................ ...............13 5.2.2 Test Planning (AWR)............................................................................... ...................14 Test Planning................................................................................................. ................14 Receive requests from Customer......................................................... .........................14 Revisit Test Strategy ........................................................................................ .............15 5.3 Test Design................................................................................................................... ....15 5.3.1 Test Environment Setup (AWR)............................................................................ ......15 Test Environment Setup.............................................................................................. ...15 5.3.2 Test Design (AWR)........................................................................................ .............15 Test Design ......................................................................................................... ..........15 Test Data Preparation / Customization / Sampling / Manipulation.................................16
Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
C3: Protected
Page 2 of 22
Controlled copy
Testing Framework
5.4 Test Execution...................................................................................... ...........................17 5.4.1 Test Execution (AWR)...................................................................... ..........................17 Pilot Execution of Test cases......................................................................... ................17 Test Execution......................................................................................... ......................17 5.5 Delivery....................................................................................................................... ......18 5.5.1 Delivery (AWR)...................................................................................................... .....18 Delivery..................................................................................................................... .....18 Verification by Onsite team............................................................................. ...............18 5.6 Acceptance Testing................................................................................... ......................19 5.6.1 Acceptance Testing (AWR)............................................................................... ..........19 Acceptance Test Plan................................................................................. ...................19 Acceptance Test Procedure................................................................................. ..........19 Assist in UAT Data Preparation........................................................... ..........................19 Acceptance Testing....................................................................................................... .19 Acceptance Sign – Off................................................................................... ................20
6.0 Section B – Process Handbook (ETVX)......................................... ...................21 7.0 Section C – Measurements Applicable.......................................... .....................22
Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
C3: Protected
Page 3 of 22
Controlled copy
Testing Framework
1.0Introduction This handbook describes the Cognizant Testing framework and applies to Projects executed by Cognizant delivery team as part of 1.
Standalone projects taken up for any new/existing customer.
2.
ODC engagement with existing customers, This framework is organized as follows:
−
SECTION A – TESTING WORKFLOW
−
SECTION B – STAGES OF TESTING
−
ANNEXURES
Section A – Testing Workflow The testing workflow refers to the logical group of tasks performed during the testing process. Logical grouping aims at grouping tasks that fulfill a common objective like System testing, Delivery etc. A similar workflow exists for the Project Management process, which happens in parallel / overlaps with the testing workflow. (Refer to the Delivery Management Framework for more details) Section B –Testing Process (ETVX Model/ OSSP) Tasks in Testing workflow and Project Management workflow are fitted into the different stages of Testing operational model
Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
C3: Protected
Page 4 of 22
Controlled copy
Testing Framework
2.0Definitions and Acronyms 2.1Acronyms NFR
Non functional requirement
RFP
Request For Proposal
SOW
Statement of work
AWR
Activity, Work Product, Responsibility
ODC
Offshore Development Center
POC
Proof of Concept
2.2Definitions ROLE
The roles depicted here identify the skill background of the people doing the tasks. A Project team member could be playing more than one role. It is suggested that projects do a mapping of the roles to the team members accordingly
Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
C3: Protected
Page 5 of 22
Controlled copy
Testing Framework
3.0Purpose The primary purpose of this framework is to describe the testing related practices that are applicable to projects executed by Cognizant. Where special guidelines are available, they have been referenced in the framework. It is recommended that such guidelines be read and understood entirely to have a complete understanding of the Testing Framework. The practices that are mandatory for all projects and the practices that are optional for the projects are listed in separate boxes throughout the framework. An explanation of the legends is provided below:
Mandatory / Optional Discussions Guidelines
Notes
What is not the intended use for the Framework? The Framework cannot be considered to be the PSPH for the project. Understanding this is important, as the framework is not organized by Stages. It is rather organized by process disciplines. The Section B of the Framework contains references to PSPH, which defines the process to be adopted by the projects.
What then defines the process for the projects? The Section B of the Framework contains references to PSPH, which defines the process to be adopted by the projects.
How should the Framework be used? 1. The Framework provides the basis on which all Operational life-cycle models are defined. 2. Additional operational life-cycle models needs to be arrived at using the Framework as the basis. 3. The Framework splits each of the tasks in the PSPH into further activities thus explaining how a task needs to be performed. Further references are provided to Testing guidelines that describe the Inputs, Outputs, and Tools & Techniques to be performed for each of these tasks.
Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
C3: Protected
Page 6 of 22
Controlled copy
Testing Framework
4.0Organization of the Framework This framework is organized as follows: −
SECTION A – Standard Cognizant Process Disciplines
−
SECTION B – Standard PSPH for QSM defined Operational Life-cycle models
−
SECTION C – Measurements Applicable. This provides a reference to the DM Framework where Metrics Plan and Guidelines have been consolidated across Solution Offerings Framewor k Standard Cognizant Process
Standard Cognizant PSPH
Tasks
Stages
Activitie s
Tasks
4.1Section A – Process Disciplines This refers to the logical groups of tasks performed during the testing process. Tasks from a logical group could be split across several stages and the process discipline provides an approach to view information from a logical view. If the project wished to understand the total effort spent on Acceptance Testing (for all the activities done as part of Acceptance Testing), it may not be possible only using stages as the plan for Acceptance Testing is done upfront in the project. During the start of the project, the tasks are mapped to the chronological stages in the projects while the Process Discipline mapping is fixed, abstracted but available for reporting purposes.
4.2Section B – Standard PSPH Here the stages of Application development life cycle are depicted. Tasks from the various Process Disciplines are fitted into the different stages
Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
C3: Protected
Page 7 of 22
Controlled copy
Testing Framework
depending on the life-cycle model chosen for the project Advantage of distinguishing Process Disciplines And Stages 1. Tasks can be looked upon as components, which can be plugged and played into different Stages of the project. The work products are tied to the tasks and the entry criteria and exit criteria of the stages would vary depending on how the tasks are interweaved. 2. Organizing tasks by process discipline and stages also helps in slicing and dicing project metrics and help in comparing one versus the other.
Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
C3: Protected
Page 8 of 22
Controlled copy
Testing Framework
5.0Section A - Process Disciplines 5.1Test Requirements Study 5.1.1Test Requirements Study (AWR) Scoping Activity Define scope of testing (In Scope and Out of Scope)
Work Products
Test Requirements Document
Responsibility Account Manager / Senior Business Analysts
Requirements Gathering Activity Requirement gathering discussions and clarifications Study of system Requirements Documentation of software requirements and sign-off
Work products Discussion notes / CRAD
Test Requirements Document
Responsibility Business Analysts, Project Manager / Project Leader Business Analysts, Project Manager / Project Leader Business Analysts, Project Manager / Project Leader
Identification of Estimation Methodology Activity Evaluate methodologies
Estimation
Define Estimation methodology for testing
Work Products
Responsibility
Estimation Sheet
Project Manager / Project Leader
Estimation Sheet
Project Manager / Project Leader
Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
C3: Protected
Page 9 of 22
Controlled copy
Testing Framework
Estimation Activity Estimate for the total volume of work
Work Products Estimation Sheet (If Test Case Estimation: Test Case Estimation Document)
Responsibility Project Leader
Business Process Understanding Activity
Work products
Responsibility
Understand and document the Testing KT Document Application process
Project Leader
Understand and document Technical aspects of the Testing KT Document application
Project Leader
Identify and customer baselines of system
Project Leader
define the performance Customer the existing Baseline
Performance
Application Understanding Activity
Work products
Responsibility
Understand the applications to Testing KT document be tested
Project Leader
Understand and document the interfaces to the application, if Testing KT document any
Project Leader
Understand the functionality impacts on other external Testing KT document interfaces
Project Leader
Tools Understanding Activity
Work products
Understand if there are any tools used for testing in client’s Testing KT Document place
Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
C3: Protected
Responsibility Project Leader
Page 10 of 22
Controlled copy
Testing Framework
Understand the test script coding guidelines of the tool, if Testing KT Document any
Project Leader
Obtain Test Script Coding Guidelines and User Manual if Testing KT Document any
Project Leader
Customer Testing Process Understanding Activity
Work products
Responsibility
Understand and document the Testing KT Document testing strategies used
Project Leader
Understand and document the Testing KT Document testing techniques used
Project Leader
Understand and document the various outputs of the testing Testing KT Document process (like Test Results, Test Cases etc.)
Project Leader
Understand the gaps in the Testing KT Document existing Test Process
Project Leader
Infrastructure requirements Understanding Activity
Work products
Responsibility
Understand and document the hardware / software Testing KT document environment that the application is running
Project Leader
Identify if there are any Testing KT document connectivity to be established
Project Leader
Customer Performance Baseline Collection Activity
Work products
Identify if there are any Customer Customer Performance Baseline Baselines set
Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
Performance
C3: Protected
Responsibility Project Leader
Page 11 of 22
Controlled copy
Testing Framework
Traceability matrix for Test Requirements Activity
Work products
List down the test Requirements requirements to trace it matrix throughout the testing life cycle
Responsibility
Traceability
Project Leader
Knowledge Transition to Offshore Activity
Work Products
KT of application systems, existing test process, infrastructure setup etc. to Offshore
-
Responsibility Project Leader
5.2 Test Strategy 5.2.1Test Strategy (AWR) Tool Evaluation and Selection (in case of automated testing) Activity Identify tools to be evaluated Identify various evaluation parameters
Work products Test Tools Evaluation and Selection Report
Responsibility Team Leads Team Leads Team members / Team Leads
Evaluate tools on the various parameters
Team members
Report findings Review findings
Test Tools Evaluation and Selection review report
Team Leads
Finalize tool to be used
Test Tools Evaluation and Selection Report
Team Leads
Refer Black-Box testing questionnaire and Performance testing questionnaire for evaluating various tools on Black-Box testing and Performance testing
Testing Tools Installation (in case of automated testing) Activity
Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
Work products
C3: Protected
Responsibility
Page 12 of 22
Controlled copy
Testing Framework
Installation, Setup and customization of tools that are finalized for testing
Tools ready to use
Project Leader / Team Leads
Proof Of Concept (in case of automated testing) Activity Proof of Concept in case of automated testing
Work products Proof Of Concept Document
Responsibility Team Leads
Test Strategy formulation Activity Determine the types of testing to be performed (Eg: Unit / Integration / System testing, Load Testing, Link Testing, Performance Testing etc.)
Work Product Test Strategy / Approach Document
Responsibility Project Leader / Team Leads
Define testing methodology (Eg: Top-Down / Bottom-Up) Define the risks involved and assumptions made in the project Identify testing tools, if applicable (This section can be filled once the testing tool is finalized) Define the testing approach / methodology to be followed in case of automated testing
1. Refer Test Strategy Guidelines for various types of testing that could be performed and various testing techniques 2. Refer Manual Testing Guidelines and Automated Testing Guidelines for the difference in the testing process
Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
C3: Protected
Page 13 of 22
Controlled copy
Testing Framework
5.2.2Test Planning (AWR) Test Planning Activity Types of Testing to be done are identified
Work products Test Strategy
Responsibility Project Leader / Team Leads
The tasks for Test Planning are identified
Project Leader / Team Leads
Test schedule is defined
Project Leader / Team Leads
Specific responsibilities are assigned to team members along with clear-cut deadlines Preparatory activities are identified (Eg: acquiring special testing utility, hiring and training engineers etc.)
Project Leader / Team Leads Test Plan
Project Leader / Team Leads Project Leader / Team Leads
Test items are identified Project Leader / Team Leads
Additional needs, if any, are identified Review Test Plan
Test Plan review report
Log Test Requests Identify the Deadlines (Start Date and End Date) for the Test Request Update Requirements Traceability matrix
Team Leads Team members
Test Request Sheet and Tracking Sheet
Updated Requirements Traceability matrix
Team members
Receive requests from Customer Activity Receive requests from Customer
Work products Test Request and Tracking Sheet
Responsibility Team Leads / Team members
Requirements could come in two ways 1. Upfront Defined requirements 2. Adhoc requests which needs to be tracked Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
C3: Protected
Page 14 of 22
Controlled copy
Testing Framework
Revisit Test Strategy Activity Identify if Test Planning has brought in any changes to the Test Strategy
Work products Updated Test Strategy
Responsibility Project Leader
5.3Test Design 5.3.1Test Environment Setup (AWR) Test Environment Setup Activity
Work products
Responsibility
Manual Testing Setup System / Integration / Unit Test environment as applicable
Updated Test Strategy document
Setup System / Integration databases
Team Leads / Test DBA
Run scripts and populate the Populated System / databases Integration database Review System / Integration / Unit test environment
Team members
Team Leads
Test Environment review Team Leads report
Automated Testing Setting up the automated testing environment
Automated test bed
Team Leads / Team members
5.3.2Test Design (AWR) Test Design Activity
Work products
Responsibility
Manual Testing
Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
C3: Protected
Page 15 of 22
Controlled copy
Testing Framework
Identify the type of testing
Team Leads/ Team members
Identify the scope of testing Identify the pre-requisites for testing Identify the test scenarios for generating the test cases Generate test cases based on the type and scope of testing
Test Procedure with test cases
Team Leads/ Team members
Define execution and verification procedures Finalize environment data requirements Build test suites, test drivers as Test Suites / Test Drivers Team members needed for testing Review Test Design
Test Procedure review report
Update Requirements Updated Requirements Traceability Matrix to include Traceability Matrix test specifications
Team Leads
Team members
Automated Testing Prepare Test Scripts
Test Scripts / Test Procedure
Team Leads / Team members
Review Test Scripts
Reviewed Test Scripts
Team Leads / Team members
Test Data Preparation / Customization / Sampling / Manipulation Activity
Work products
Responsibility
Manual Testing Prepare necessary test data Test environment ready for System / Integration / Unit with Test Data Testing
Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
C3: Protected
Team members
Page 16 of 22
Controlled copy
Testing Framework Test environment ready with validated Test Data
Validate test data
Team Leads
5.4Test Execution 5.4.1Test Execution (AWR) Pilot Execution of Test cases Activity
Work products
Responsibility
Manual Testing Perform Pilot Run Test Logs
Team members
Log Results Analyze Results
Pilot Run Results Report Project Leader / Client
Define Corrective action, if necessary Revisit Test Strategy, if necessary
-
Project Leader
Updated Test Strategy document
Project Leader
Test Execution Activity
Work products
Identify the type of testing to be done
-
Identify number of test runs from the test plan.
-
Responsibility
Team members / Team Leads
Test logs /As run In every test run, Run test cases as per the test logs (in case of procedure and records results. Automated Testing) If a test run fails altogether, because of an abortive functionality, then record the failure of the run as a whole (E.g. RUN N: ABORT M)
Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
Record of disruption of Cycle completion in the test logs /As Team members run logs (in case of Automated Testing)
C3: Protected
Page 17 of 22
Controlled copy
Testing Framework
Once the abortive failure is corrected, restart Test logs /As run the Test Run and repeat for all the planned Test logs (in case of Runs Automated Testing) Update Requirements Traceability Matrix to include test results
Team members
Updated Requirements Team members Traceability Matrix
5.5Delivery 5.5.1Delivery (AWR) Delivery Activity Work products Responsibility Zip / Pack the testing documents Zipped file containing Test Plans Project Leader / (typically Test Plans, Test Procedures / / Test Procedures / Test Logs for Team Leads Test Cases and Test Logs) for the the tested module tested module / requirements Prepare summary report for testing
Summary Report
Project Leader / Team Leads
Verification by Onsite team Activity Deliverables verification by Onsite team
Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
Work products
Responsibility
Verification mail from Onsite coordinator
Onsite Coordinator
C3: Protected
Page 18 of 22
Controlled copy
Testing Framework
5.6Acceptance Testing 5.6.1Acceptance Testing (AWR) Acceptance Test Plan Activity
Work products
Plan for Acceptance Testing
Responsibility
Acceptance Test Plan
Customer and Cognizant
Acceptance Test Procedure Activity
Work products
Responsibility
Test Requirements Document
Determine Acceptance Criteria Identify Acceptance test scenarios
Customer and Cognizant Customer and Cognizant
Acceptance Test Procedure
Define Acceptance Test Cases
Customer and Cognizant
Assist in UAT Data Preparation Activity
Work products
Responsibility
Acceptance testing environment ready with test data
Prepare Acceptance Test Data
Customer and Cognizant
Acceptance Testing Activity
Work products
Responsibility
Review and communicate any changes to the acceptance test plan
Updated Acceptance test plan
Client / Cognizant (S)
Get the acceptance test cases and review the same
Acceptance test cases
Cognizant Delivery Manager
Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
C3: Protected
Page 19 of 22
Controlled copy
Testing Framework
Support during acceptance testing
Issue Resolution, Test logs from the client
Onsite coordinator / Delivery manager
Acceptance Sign – Off Activity
Work products
Get Acceptance sign-off from Formal mail from Client customer
Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
C3: Protected
Responsibility Customer
Page 20 of 22
Controlled copy
Testing Framework
6.0 Section B – Process Handbook (ETVX) Testing Project Software Process HandBook
Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
C3: Protected
Page 21 of 22
Controlled copy
Testing Framework
7.0Section C – Measurements Applicable Metrics Plan for Testing
Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
C3: Protected
Page 22 of 22
Framework for Testing
Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
C3: Protected
Controlled copy
Testing Framework
Table of Contents 1.0 Introduction............................................................................... .............................4 2.0 Definitions and Acronyms............................................................. ........................5 2.1 Acronyms...................................................................................................... .....................5 2.2 Definitions........................................................................................................ ..................5
3.0 Purpose...................................................................................................... .............6 What is not the intended use for the Framework?................................... ........................6 What then defines the process for the projects?....................................................... .......6 How should the Framework be used?................................................... ..........................6
4.0 Organization of the Framework.................................................................. ...........7 4.1 Section A – Process Disciplines.................................................................................. .....7 4.2 Section B – Standard PSPH ................................................................................ .............7
5.0 Section A - Process Disciplines................................................. ..........................9 5.1 Test Requirements Study ........................................................................................ .........9 5.1.1 Test Requirements Study (AWR)............................................................................ ......9 Scoping........................................................................................................................ ....9 Requirements Gathering ............................................................................................ .....9 Identification of Estimation Methodology............................................... ..........................9 Estimation ....................................................................................................... ............10 Business Process Understanding ................................................................... ..............10 Application Understanding.................................................................................. ...........10 Tools Understanding......................................................................................... .............10 Customer Testing Process Understanding........................................................... ..........11 Infrastructure requirements Understanding............................................................. .......11 Customer Performance Baseline Collection.................................................................. .11 Traceability matrix for Test Requirements........................................................... ...........12 Knowledge Transition to Offshore........................................................................ ..........12 5.2 Test Strategy.............................................................................................. .....................12 5.2.1 Test Strategy (AWR)........................................................................................... ........12 Tool Evaluation and Selection (in case of automated testing).................................... ....12 Testing Tools Installation (in case of automated testing).......................................... ......12 Proof Of Concept (in case of automated testing).............................................. .............13 Test Strategy formulation................................................................................ ...............13 5.2.2 Test Planning (AWR)............................................................................... ...................14 Test Planning................................................................................................. ................14 Receive requests from Customer......................................................... .........................14 Revisit Test Strategy ........................................................................................ .............15 5.3 Test Design................................................................................................................... ....15 5.3.1 Test Environment Setup (AWR)............................................................................ ......15 Test Environment Setup.............................................................................................. ...15 5.3.2 Test Design (AWR)........................................................................................ .............15 Test Design ......................................................................................................... ..........15 Test Data Preparation / Customization / Sampling / Manipulation.................................16
Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
C3: Protected
Page 2 of 22
Controlled copy
Testing Framework
5.4 Test Execution...................................................................................... ...........................17 5.4.1 Test Execution (AWR)...................................................................... ..........................17 Pilot Execution of Test cases......................................................................... ................17 Test Execution......................................................................................... ......................17 5.5 Delivery....................................................................................................................... ......18 5.5.1 Delivery (AWR)...................................................................................................... .....18 Delivery..................................................................................................................... .....18 Verification by Onsite team............................................................................. ...............18 5.6 Acceptance Testing................................................................................... ......................19 5.6.1 Acceptance Testing (AWR)............................................................................... ..........19 Acceptance Test Plan................................................................................. ...................19 Acceptance Test Procedure................................................................................. ..........19 Assist in UAT Data Preparation........................................................... ..........................19 Acceptance Testing....................................................................................................... .19 Acceptance Sign – Off................................................................................... ................20
6.0 Section B – Process Handbook (ETVX)......................................... ...................21 7.0 Section C – Measurements Applicable.......................................... .....................22
Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
C3: Protected
Page 3 of 22
Controlled copy
Testing Framework
1.0Introduction This handbook describes the Cognizant Testing framework and applies to Projects executed by Cognizant delivery team as part of 1.
Standalone projects taken up for any new/existing customer.
2.
ODC engagement with existing customers, This framework is organized as follows:
−
SECTION A – TESTING WORKFLOW
−
SECTION B – STAGES OF TESTING
−
ANNEXURES
Section A – Testing Workflow The testing workflow refers to the logical group of tasks performed during the testing process. Logical grouping aims at grouping tasks that fulfill a common objective like System testing, Delivery etc. A similar workflow exists for the Project Management process, which happens in parallel / overlaps with the testing workflow. (Refer to the Delivery Management Framework for more details) Section B –Testing Process (ETVX Model/ OSSP) Tasks in Testing workflow and Project Management workflow are fitted into the different stages of Testing operational model
Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
C3: Protected
Page 4 of 22
Controlled copy
Testing Framework
2.0Definitions and Acronyms 2.1Acronyms NFR
Non functional requirement
RFP
Request For Proposal
SOW
Statement of work
AWR
Activity, Work Product, Responsibility
ODC
Offshore Development Center
POC
Proof of Concept
2.2Definitions ROLE
The roles depicted here identify the skill background of the people doing the tasks. A Project team member could be playing more than one role. It is suggested that projects do a mapping of the roles to the team members accordingly
Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
C3: Protected
Page 5 of 22
Controlled copy
Testing Framework
3.0Purpose The primary purpose of this framework is to describe the testing related practices that are applicable to projects executed by Cognizant. Where special guidelines are available, they have been referenced in the framework. It is recommended that such guidelines be read and understood entirely to have a complete understanding of the Testing Framework. The practices that are mandatory for all projects and the practices that are optional for the projects are listed in separate boxes throughout the framework. An explanation of the legends is provided below:
Mandatory / Optional Discussions Guidelines
Notes
What is not the intended use for the Framework? The Framework cannot be considered to be the PSPH for the project. Understanding this is important, as the framework is not organized by Stages. It is rather organized by process disciplines. The Section B of the Framework contains references to PSPH, which defines the process to be adopted by the projects.
What then defines the process for the projects? The Section B of the Framework contains references to PSPH, which defines the process to be adopted by the projects.
How should the Framework be used? 1. The Framework provides the basis on which all Operational life-cycle models are defined. 2. Additional operational life-cycle models needs to be arrived at using the Framework as the basis. 3. The Framework splits each of the tasks in the PSPH into further activities thus explaining how a task needs to be performed. Further references are provided to Testing guidelines that describe the Inputs, Outputs, and Tools & Techniques to be performed for each of these tasks.
Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
C3: Protected
Page 6 of 22
Controlled copy
Testing Framework
4.0Organization of the Framework This framework is organized as follows: −
SECTION A – Standard Cognizant Process Disciplines
−
SECTION B – Standard PSPH for QSM defined Operational Life-cycle models
−
SECTION C – Measurements Applicable. This provides a reference to the DM Framework where Metrics Plan and Guidelines have been consolidated across Solution Offerings Framewor k Standard Cognizant Process
Standard Cognizant PSPH
Tasks
Stages
Activitie s
Tasks
4.1Section A – Process Disciplines This refers to the logical groups of tasks performed during the testing process. Tasks from a logical group could be split across several stages and the process discipline provides an approach to view information from a logical view. If the project wished to understand the total effort spent on Acceptance Testing (for all the activities done as part of Acceptance Testing), it may not be possible only using stages as the plan for Acceptance Testing is done upfront in the project. During the start of the project, the tasks are mapped to the chronological stages in the projects while the Process Discipline mapping is fixed, abstracted but available for reporting purposes.
4.2Section B – Standard PSPH Here the stages of Application development life cycle are depicted. Tasks from the various Process Disciplines are fitted into the different stages
Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
C3: Protected
Page 7 of 22
Controlled copy
Testing Framework
depending on the life-cycle model chosen for the project Advantage of distinguishing Process Disciplines And Stages 1. Tasks can be looked upon as components, which can be plugged and played into different Stages of the project. The work products are tied to the tasks and the entry criteria and exit criteria of the stages would vary depending on how the tasks are interweaved. 2. Organizing tasks by process discipline and stages also helps in slicing and dicing project metrics and help in comparing one versus the other.
Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
C3: Protected
Page 8 of 22
Controlled copy
Testing Framework
5.0Section A - Process Disciplines 5.1Test Requirements Study 5.1.1Test Requirements Study (AWR) Scoping Activity Define scope of testing (In Scope and Out of Scope)
Work Products
Test Requirements Document
Responsibility Account Manager / Senior Business Analysts
Requirements Gathering Activity Requirement gathering discussions and clarifications Study of system Requirements Documentation of software requirements and sign-off
Work products Discussion notes / CRAD
Test Requirements Document
Responsibility Business Analysts, Project Manager / Project Leader Business Analysts, Project Manager / Project Leader Business Analysts, Project Manager / Project Leader
Identification of Estimation Methodology Activity Evaluate methodologies
Estimation
Define Estimation methodology for testing
Work Products
Responsibility
Estimation Sheet
Project Manager / Project Leader
Estimation Sheet
Project Manager / Project Leader
Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
C3: Protected
Page 9 of 22
Controlled copy
Testing Framework
Estimation Activity Estimate for the total volume of work
Work Products Estimation Sheet (If Test Case Estimation: Test Case Estimation Document)
Responsibility Project Leader
Business Process Understanding Activity
Work products
Responsibility
Understand and document the Testing KT Document Application process
Project Leader
Understand and document Technical aspects of the Testing KT Document application
Project Leader
Identify and customer baselines of system
Project Leader
define the performance Customer the existing Baseline
Performance
Application Understanding Activity
Work products
Responsibility
Understand the applications to Testing KT document be tested
Project Leader
Understand and document the interfaces to the application, if Testing KT document any
Project Leader
Understand the functionality impacts on other external Testing KT document interfaces
Project Leader
Tools Understanding Activity
Work products
Understand if there are any tools used for testing in client’s Testing KT Document place
Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
C3: Protected
Responsibility Project Leader
Page 10 of 22
Controlled copy
Testing Framework
Understand the test script coding guidelines of the tool, if Testing KT Document any
Project Leader
Obtain Test Script Coding Guidelines and User Manual if Testing KT Document any
Project Leader
Customer Testing Process Understanding Activity
Work products
Responsibility
Understand and document the Testing KT Document testing strategies used
Project Leader
Understand and document the Testing KT Document testing techniques used
Project Leader
Understand and document the various outputs of the testing Testing KT Document process (like Test Results, Test Cases etc.)
Project Leader
Understand the gaps in the Testing KT Document existing Test Process
Project Leader
Infrastructure requirements Understanding Activity
Work products
Responsibility
Understand and document the hardware / software Testing KT document environment that the application is running
Project Leader
Identify if there are any Testing KT document connectivity to be established
Project Leader
Customer Performance Baseline Collection Activity
Work products
Identify if there are any Customer Customer Performance Baseline Baselines set
Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
Performance
C3: Protected
Responsibility Project Leader
Page 11 of 22
Controlled copy
Testing Framework
Traceability matrix for Test Requirements Activity
Work products
List down the test Requirements requirements to trace it matrix throughout the testing life cycle
Responsibility
Traceability
Project Leader
Knowledge Transition to Offshore Activity
Work Products
KT of application systems, existing test process, infrastructure setup etc. to Offshore
-
Responsibility Project Leader
5.2 Test Strategy 5.2.1Test Strategy (AWR) Tool Evaluation and Selection (in case of automated testing) Activity Identify tools to be evaluated Identify various evaluation parameters
Work products Test Tools Evaluation and Selection Report
Responsibility Team Leads Team Leads Team members / Team Leads
Evaluate tools on the various parameters
Team members
Report findings Review findings
Test Tools Evaluation and Selection review report
Team Leads
Finalize tool to be used
Test Tools Evaluation and Selection Report
Team Leads
Refer Black-Box testing questionnaire and Performance testing questionnaire for evaluating various tools on Black-Box testing and Performance testing
Testing Tools Installation (in case of automated testing) Activity
Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
Work products
C3: Protected
Responsibility
Page 12 of 22
Controlled copy
Testing Framework
Installation, Setup and customization of tools that are finalized for testing
Tools ready to use
Project Leader / Team Leads
Proof Of Concept (in case of automated testing) Activity Proof of Concept in case of automated testing
Work products Proof Of Concept Document
Responsibility Team Leads
Test Strategy formulation Activity Determine the types of testing to be performed (Eg: Unit / Integration / System testing, Load Testing, Link Testing, Performance Testing etc.)
Work Product Test Strategy / Approach Document
Responsibility Project Leader / Team Leads
Define testing methodology (Eg: Top-Down / Bottom-Up) Define the risks involved and assumptions made in the project Identify testing tools, if applicable (This section can be filled once the testing tool is finalized) Define the testing approach / methodology to be followed in case of automated testing
1. Refer Test Strategy Guidelines for various types of testing that could be performed and various testing techniques 2. Refer Manual Testing Guidelines and Automated Testing Guidelines for the difference in the testing process
Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
C3: Protected
Page 13 of 22
Controlled copy
Testing Framework
5.2.2Test Planning (AWR) Test Planning Activity Types of Testing to be done are identified
Work products Test Strategy
Responsibility Project Leader / Team Leads
The tasks for Test Planning are identified
Project Leader / Team Leads
Test schedule is defined
Project Leader / Team Leads
Specific responsibilities are assigned to team members along with clear-cut deadlines Preparatory activities are identified (Eg: acquiring special testing utility, hiring and training engineers etc.)
Project Leader / Team Leads Test Plan
Project Leader / Team Leads Project Leader / Team Leads
Test items are identified Project Leader / Team Leads
Additional needs, if any, are identified Review Test Plan
Test Plan review report
Log Test Requests Identify the Deadlines (Start Date and End Date) for the Test Request Update Requirements Traceability matrix
Team Leads Team members
Test Request Sheet and Tracking Sheet
Updated Requirements Traceability matrix
Team members
Receive requests from Customer Activity Receive requests from Customer
Work products Test Request and Tracking Sheet
Responsibility Team Leads / Team members
Requirements could come in two ways 1. Upfront Defined requirements 2. Adhoc requests which needs to be tracked Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
C3: Protected
Page 14 of 22
Controlled copy
Testing Framework
Revisit Test Strategy Activity Identify if Test Planning has brought in any changes to the Test Strategy
Work products Updated Test Strategy
Responsibility Project Leader
5.3Test Design 5.3.1Test Environment Setup (AWR) Test Environment Setup Activity
Work products
Responsibility
Manual Testing Setup System / Integration / Unit Test environment as applicable
Updated Test Strategy document
Setup System / Integration databases
Team Leads / Test DBA
Run scripts and populate the Populated System / databases Integration database Review System / Integration / Unit test environment
Team members
Team Leads
Test Environment review Team Leads report
Automated Testing Setting up the automated testing environment
Automated test bed
Team Leads / Team members
5.3.2Test Design (AWR) Test Design Activity
Work products
Responsibility
Manual Testing
Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
C3: Protected
Page 15 of 22
Controlled copy
Testing Framework
Identify the type of testing
Team Leads/ Team members
Identify the scope of testing Identify the pre-requisites for testing Identify the test scenarios for generating the test cases Generate test cases based on the type and scope of testing
Test Procedure with test cases
Team Leads/ Team members
Define execution and verification procedures Finalize environment data requirements Build test suites, test drivers as Test Suites / Test Drivers Team members needed for testing Review Test Design
Test Procedure review report
Update Requirements Updated Requirements Traceability Matrix to include Traceability Matrix test specifications
Team Leads
Team members
Automated Testing Prepare Test Scripts
Test Scripts / Test Procedure
Team Leads / Team members
Review Test Scripts
Reviewed Test Scripts
Team Leads / Team members
Test Data Preparation / Customization / Sampling / Manipulation Activity
Work products
Responsibility
Manual Testing Prepare necessary test data Test environment ready for System / Integration / Unit with Test Data Testing
Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
C3: Protected
Team members
Page 16 of 22
Controlled copy
Testing Framework Test environment ready with validated Test Data
Validate test data
Team Leads
5.4Test Execution 5.4.1Test Execution (AWR) Pilot Execution of Test cases Activity
Work products
Responsibility
Manual Testing Perform Pilot Run Test Logs
Team members
Log Results Analyze Results
Pilot Run Results Report Project Leader / Client
Define Corrective action, if necessary Revisit Test Strategy, if necessary
-
Project Leader
Updated Test Strategy document
Project Leader
Test Execution Activity
Work products
Identify the type of testing to be done
-
Identify number of test runs from the test plan.
-
Responsibility
Team members / Team Leads
Test logs /As run In every test run, Run test cases as per the test logs (in case of procedure and records results. Automated Testing) If a test run fails altogether, because of an abortive functionality, then record the failure of the run as a whole (E.g. RUN N: ABORT M)
Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
Record of disruption of Cycle completion in the test logs /As Team members run logs (in case of Automated Testing)
C3: Protected
Page 17 of 22
Controlled copy
Testing Framework
Once the abortive failure is corrected, restart Test logs /As run the Test Run and repeat for all the planned Test logs (in case of Runs Automated Testing) Update Requirements Traceability Matrix to include test results
Team members
Updated Requirements Team members Traceability Matrix
5.5Delivery 5.5.1Delivery (AWR) Delivery Activity Work products Responsibility Zip / Pack the testing documents Zipped file containing Test Plans Project Leader / (typically Test Plans, Test Procedures / / Test Procedures / Test Logs for Team Leads Test Cases and Test Logs) for the the tested module tested module / requirements Prepare summary report for testing
Summary Report
Project Leader / Team Leads
Verification by Onsite team Activity Deliverables verification by Onsite team
Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
Work products
Responsibility
Verification mail from Onsite coordinator
Onsite Coordinator
C3: Protected
Page 18 of 22
Controlled copy
Testing Framework
5.6Acceptance Testing 5.6.1Acceptance Testing (AWR) Acceptance Test Plan Activity
Work products
Plan for Acceptance Testing
Responsibility
Acceptance Test Plan
Customer and Cognizant
Acceptance Test Procedure Activity
Work products
Responsibility
Test Requirements Document
Determine Acceptance Criteria Identify Acceptance test scenarios
Customer and Cognizant Customer and Cognizant
Acceptance Test Procedure
Define Acceptance Test Cases
Customer and Cognizant
Assist in UAT Data Preparation Activity
Work products
Responsibility
Acceptance testing environment ready with test data
Prepare Acceptance Test Data
Customer and Cognizant
Acceptance Testing Activity
Work products
Responsibility
Review and communicate any changes to the acceptance test plan
Updated Acceptance test plan
Client / Cognizant (S)
Get the acceptance test cases and review the same
Acceptance test cases
Cognizant Delivery Manager
Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
C3: Protected
Page 19 of 22
Controlled copy
Testing Framework
Support during acceptance testing
Issue Resolution, Test logs from the client
Onsite coordinator / Delivery manager
Acceptance Sign – Off Activity
Work products
Get Acceptance sign-off from Formal mail from Client customer
Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
C3: Protected
Responsibility Customer
Page 20 of 22
Controlled copy
Testing Framework
6.0 Section B – Process Handbook (ETVX) Testing Project Software Process HandBook
Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
C3: Protected
Page 21 of 22
Controlled copy
Testing Framework
7.0Section C – Measurements Applicable Metrics Plan for Testing
Release ID: QFTS-FW.doc / 1.01 / 05.03.2004
C3: Protected
Page 22 of 22
Related Documents
Testing Framework
November 2019 16
Software Testing Framework
May 2020 5
Software Testing Framework
November 2019 11
Building A Performance Testing Framework
June 2020 4
Framework
June 2020 26