Testing Process
This document was uploaded by user and they confirmed that they have the permission to share it. If you are author or own the copyright of this book, please report to us by using this DMCA report form. Report DMCA
Overview
Download & View Testing Process as PDF for free.
More details
- Words: 5,409
- Pages: 25
Testing Process
Testing process
Srinivas polimera
CONTENTS
1. Introduction ......................................................................................................... ...............3 1.1 Purpose............................................................................................................. .............3 1.2 Scope........................................................................................................... ..................3 1.3 Definitions and Acronyms.................................................................................... ...........3 1.4 References....................................................................................................... ..............3 1.4.1 General................................................................................................................ ....3 1.4.2 Templates............................................................................................... .................3 1.4.3 Guidelines.................................................................................... ...........................4 1.4.4 Checklist........................................................................................................ ..........4 2. Testing Process.................................................................................................................. .5 2.1 Workflow....................................................................................................................... ..5 2.2 Process Description........................................................................... ............................6 2.2.1 Project Management Process..................................................... ............................6 2.2.2 Test Initiation phase....................................................................................... ..........6 2.2.3 Knowledge transfer phase............................................................................. ..........7 2.2.4 Test strategy Phase.................................................................................. ...............8 2.2.5 Test Planning Phase...................................................................................... ..........9 2.2.6 Test Design phase........................................................................... ......................11 2.2.7 Test Automation phase.................................................................... ......................14 2.2.8 Test execution and defect reporting phase.......................................... ..................17 2.2.9 Test certification........................................................................................... .........24 2.2.10 Test Closure Phase.................................................................. ...........................24
Page 2 of 25
Testing process
Srinivas polimera
1. Introduction 1.1Purpose Purpose of this document is to lay down a standard approach to testing which can be followed by all projects. The scope of the document starts with the initiation of the testing activities and ends with the test Closure Phase.
1.2Scope The process applies to all the members who directly/indirectly involved in testing activities in marlabs .
1.3Definitions and Acronyms PEG CMMI SQA QMS PCR PDSP HR PM TM PIF RCA DP NC KT
Process Engineering Group Capability Maturity Model Integration Software Quality Assurance Quality Management System Process Change Request Project Defined Software Process Human Resource Project Manager Test Manager Project Initiation Form Root Cause Analysis Defect Prevention Non Conformance Knowledge transfer
1.4References This section provides a complete list of all documents that need to be referenced for proper interpretation of process.
1.4.1 General Capability Maturity Model® Integration (CMMISM), Version 1.1 •
TM can perform the responsibilities defined in this document for PM.
•
The role “Customer Project Manager / Contact” is used throughout this document. The person playing this role will be defined in the test plan. The person could be from Customer Project Organization or from the Project Organization. This role is significant as he / she will be involved in making key decisions during the test life cycle.
•
It is recommended that a Tech Lead in the project play the role “Development Representative” referred through out the document.
1.4.2 Templates • •
Test Requirements Specification Template Test Strategy Template Page 3 of 25
Testing process
• • • • • • • •
Srinivas polimera
Test Plan Template Impacted External Systems Communication Plan Test case / Test report Template Traceability Matrix Automation Design Template (Project defined template) Test Status Report Test Certificate Test Project Post-mortem Report
1.4.3 Guidelines • •
Testing Guidelines. Measurement Guidelines.
1.4.4 Checklist • •
Test case review Checklist. Test Quality Index Checklist
Page 4 of 25
Testing process
Srinivas polimera
2. Testing Process 2.1Workflow Test Initiation Phase
Know ledge Transfer Phase
Test StrategyPhase
Test Planning Phase
TestDesignphase High-level scenarios preparation Identification of impacted external systems Detailed Test Cases preparation Test Data Preparation
Test Execution and Defect Reporting Phase Identification of test cases for manual testing Integration test environment validation Integration testing System test environment validation System Testing Ad-hoc testing Regression test environment validation Regression testing
Test Autom ationPhase
Test case identification for automation Test automation design Test automation development Customer supplied automated script selection Test automation execution
TestCertificationPhase
TestClosure Phase
Page 5 of 25
Testing process
Srinivas polimera
2.2Process Description 2.2.1 Project Management Process Refer to Project Management process document in PQMG portal for details
on
• • • • •
Project initiation Process Project Planning Project Execution and Tracking Risk Management Project Closure
All the process defined in the Project Management process are applicable for Testing projects also. In case of pure testing project, Test plan can be an appendix to Project Management Plan.
2.2.2 Test Initiation phase Once a specific project or a release is identified for testing, the test initiation phase will begin. The objective of this phase is to review the test requirements and prepare initial test plan.
2.2.2.1 Entry Criteria • • • •
Project Initiation phase completed Optionally Test Requirements document / Software requirements document exists. Test Manager / Lead identified to head the testing part of the project. Customer Project Manager / contact identified for coordinating testing part of the project.
2.2.2.2 Tasks 1. Review the Test Requirements document / Software requirements document. Use the “Guidelines for collecting Test Requirements” in Testing Guidelines document and prepare or update the test requirements document. 2. The test requirements document shall cover the requirements for manual testing, automation testing, performance testing, white box testing, etc. 3. Prepare the Initial test plan. The template will be same as detailed test plan. At least the sections “Introduction, Scope of testing, Knowledge Transfer plan” to be completed. The knowledge transfer plan shall address needs of manual testing, automation testing, performance testing, white box testing, etc.
4. All stakeholders relevant to testing activities have to be identified as part of test initiation and involved at appropriate way.
Page 6 of 25
Testing process
Srinivas polimera
2.2.2.3 Exit criteria • •
Updated Test Requirements document Initial test plan
2.2.2.4 Measurement •
Effort for preparing initial Test plan
2.2.2.5 Verification • •
Review by PM Review by SQA
2.2.2.6 Roles & Responsibilities Role PM Test Lead SQA
Responsibilities Review and approve Test requirements and initial test plan Prepare Test requirements and initial test plan Review initial test plan
2.2.3 Knowledge transfer phase The objective of this phase is to train the testing Team member(s) on requirements and functionalities that are part of the project or a release. This includes training needs to address manual testing, automation testing, performance testing, white box testing, etc.
2.2.3.1 Entry Criteria •
Test initiation phase completed
2.2.3.2 Tasks 1. Schedule meetings with identified persons for the knowledge transfer session as per Knowledge Transfer plan. 2. Prepare for the knowledge transfer. Identified Testing team member(s) to go through the Test requirements and other relevant documents prior to knowledge transfer session. If there are any special requirements, they will be identified in Knowledge Transfer plan section of Test Plan document. 3. Coordinate the Knowledge transfer sessions as planned in Knowledge Transfer plan section of Test Plan document.
4. Prepare a list of documents, which will be used as inputs during
subsequent phases. Send it to the Customer Project Manager / contact, for confirmation.
Page 7 of 25
Testing process
Srinivas polimera
2.2.3.3 Exit criteria •
List of documents that will be used for preparation of Test cases (No template, it can be a free form)
2.2.3.4 Measurement •
Effort spent on KT
2.2.3.5 Verification • • •
Review by PM Review by Customer Review by SQA
2.2.3.6 Roles & Responsibilities Responsibilities
Role Unit Deliver Head PM Test Lead Test team members SQA Customer
• • • • • • •
Resolve escalated issues during KT phase Review KT activities Co-ordinate all KT activities Confirms the completion of KT activities Understand the requirements during KT Audit KT activities Provide the KT inputs
2.2.4 Test strategy Phase The objective of this phase is to develop test strategy for a project or for a specific release. This phase is an optional phase depending the scope of the project.
2.2.4.1 Entry Criteria • • •
Test Initiation phase completed. Optionally Knowledge transfer is also completed, but not mandatory. Optionally Design document exists.
2.2.4.2 Tasks 1. Go through the test requirements document and other relevant documents to understand the project requirements. 2. Optionally schedule meetings with the customer and get additional information and resolve queries if any. 3. Prepare the test strategy document. Refer to the test strategy section of testing guidelines document. If needed, the test strategy document can be separate for manual testing, automation testing, performance testing, white box testing, etc. Page 8 of 25
Testing process
Srinivas polimera
4. Internally review the test strategy document and incorporate the review comments. 5. Optionally present / review the test strategy document with customer and incorporate the review comments.
2.2.4.3 Exit criteria •
Test Strategy document
2.2.4.4 Measurement •
Effort for designing test strategy
2.2.4.5 Verification • • • •
Review Review Review Review
by by by by
PM Development Representative (Optional) Customer SQA
2.2.4.6 Roles & Responsibilities Responsibilities
Role Test Architect / Test Manager / Test Lead
Prepare the Test Strategy
PM
Review and approve the test strategy
Development Representative
Optionally review the test strategy
Customer
Optionally present / review the test strategy with customer
SQA
Audit the Test Strategy activities
2.2.5 Test Planning Phase The objective of this phase is to prepare a detailed test plan for testing.
2.2.5.1 Entry Criteria •
Knowledge transfer phase completed
2.2.5.2 Tasks 1. Update the test plan (word document) prepared during Test Initiation phase and completes the remaining sections.
Page 9 of 25
Testing process
Srinivas polimera
2. If preparation of detailed test strategy document is not in the scope of the project, test strategy shall be a section within the test plan. The test strategy shall address the needs of manual testing, automation testing, performance testing, white box testing, etc. Refer to the test strategy section of the testing guidelines document for guidelines. 3. Metrics: Identify and Plan for collecting the metrics in the test plan. Refer to “Measurement guidelines” for details. 4. Plan for interim project reviews. Objective of this activity is to review the project progress. Testing Interim Project Review Template shall be used for this purpose. 5. Conduct reviews with identified team member(s) as per plan. For a specific project or a release, the name of the persons playing the role of internal reviewer, external reviewer and approver will be identified in the “Review Plan” section of the detailed test plan. 6. Test Plan/PMP shall mention the work product and type of verification (testing) to be performed. 7. Test Plan/PMP shall identify relevant resources to perform testing. 8. Configuration management activities have to be planned and performed as part of testing. All the relevant configurable items related to testing has to identified and controlled 9. Test manager/PM shall monitor and control the testing activities to achieve the project objectives and take corrective actions if any deviation found during execution.
2.2.5.3 Exit criteria •
Baselined test plan
2.2.5.4 Measurement •
Effort for preparing Test plan
2.2.5.5 Verification • • •
Review by PM Review by Customer Review by SQA
2.2.5.6 Roles & Responsibilities Responsibilities
ROLE PM Development Representative Test Manager / Test Lead Test team members SQA
• • • • • •
Review and approve test plan Resolve escalated issues Optional Provide inputs for test plan Review test plan Prepare test plan
• • •
Provide inputs for test plan Follow test plan Review test plan Page 10 of 25
Testing process
Srinivas polimera
Responsibilities
ROLE Customer
• • •
Optional Review test plan Approve test plan
2.2.6 Test Design phase The objective of this phase is to prepare for test execution activities. Planned tasks as part of this phase will include • • • •
High-Level scenarios preparation Identification of impacted external systems Detailed test cases preparation Test data preparation
2.2.6.1 High-level scenarios preparation Entry Criteria • • •
Test Strategy phase is completed (if in the scope). Detailed test planning phase completed. List documents, e.g. Requirements document, Design document, product specification etc. as agreed in Knowledge transfer phase for test design are available. Refer to Knowledge transfer phase for more details.
Tasks 1. Prepare “High-Level Scenarios / Initial test cases” as per plan, to address the needs of manual testing, automation testing, performance testing, etc. The template to be used is same Test case template. At least the columns “Test case description” and “Expected Result” columns to be filled up. Refer to the test case design section of testing guidelines document for guidelines.
2. Conduct reviews with identified team member(s) as per plan. For a specific project or a release, the name of the persons playing the role of internal reviewer, external reviewer and approver will be identified in the “Review Plan” section of the detailed test plan.
Outputs •
“High-Level scenarios / Initial test cases” document ready for all the different kinds of testing planned for the project or a release.
Exit criteria •
Approved “High-level scenarios / Initial test cases”.
Page 11 of 25
Testing process
Srinivas polimera
2.2.6.2 Identification of impacted external systems Entry Criteria •
High Level Scenarios preparation phase completed.
Tasks 1. Identify and prepare list of external systems that will be impacted
during test execution phase as per “Impacted external systems communication plan” template.
2. Initiate contact with the external system owners along with plan for support during test execution phase.
Outputs •
Impacted external systems communication plan
Exit criteria •
Confirmation from external system owners for support during test execution phase, if applicable
2.2.6.3 Detailed Test Cases preparation Entry Criteria •
High Level Scenarios preparation phase completed.
Tasks 1. Prepare Detailed Test Cases by updating the high level scenarios / initial test cases prepared during the HIGH LEVEL SCENARIOS PREPARATION PHASE. Refer to the test case design section of testing guidelines document for guidelines.
2. Identify the Sanity test cases for “Sanity testing”. 3. Identify the priority of the test cases as High, Medium or Low. 4. Update the traceability matrix in cases of development project (where one exists). In case of pure testing project create a new traceability matrix, tracing test cases to Requirements and vice versa. 5. Conduct reviews with identified team member(s) as per plan. For a specific release, the name of the persons playing the role of internal reviewer, external reviewer and approver will be identified in the “Review Plan” section of the detailed test plan.
Outputs •
Detailed test cases document Page 12 of 25
Testing process
•
Srinivas polimera
Traceability Matrix
Exit criteria •
Approved “Detailed Test Cases” document.
2.2.6.4 Test data preparation Entry Criteria •
Detailed Test Cases Preparation Phase completed.
Tasks 1. Prepare “Test data” for all applicable test types. 2. In case if customer supplies Test data, Review the test data and resolve queries if any, with appropriate contact. 3. Conduct reviews with identified team member(s) as per plan. For a specific release, the name of the persons playing the role of internal reviewer, external reviewer and approver will be identified in the “Review Plan” section of the detailed test plan.
Outputs •
Test data
Exit criteria •
Test Data is ready.
Measurement • • •
Effort for test case and data preparation. Effort for test case review. Review defects found in test cases review.
Verification • • •
Review by Development Representative (Optional) Review by Customer Review by SQA
Roles & Responsibilities Role PM Test team members
• •
SQA Customer
• •
Responsibilities Track the status and resolve issues Prepare the test deliverables as referenced in this section. Audit test preparation activities Review and approve test deliverables as referenced in this section (if applicable)
Page 13 of 25
Testing process
Srinivas polimera
2.2.7 Test Automation phase The objective of this phase is to identify the test cases for automation, design for automation, develop automation scripts as per plan, execute the automation scripts, analyze and report results. The test requirements for automation are already covered in the test initiation phase.
2.2.7.1 Identification of Test Cases for Automation The objective of this phase is to identify the test cases for automation in case of customer-supplied test cases or test cases developed in house. The automation includes functional, performance or API level.
Entry Criteria • •
Test Planning phase is completed Detailed Test Case Preparation phase is completed or Customer supplied test cases are available.
Tasks 1. Review all the test cases from automation perspective (either customer supplied test cases or in house developed test cases). Resolve queries if any, with appropriate contact. 2. Identify the test cases for automation. Please refer to the automation guidelines section in the testing guidelines document. 3. Conduct reviews with identified team member(s) as per plan. For a specific release, the name of the persons playing the role of internal reviewer, external reviewer and approver will be identified in the “Review Plan” section of the detailed test plan.
Outputs •
Test cases identified for automation
Exit criteria •
Identified test cases for automation are approved.
2.2.7.2 Test Automation Design The objective of this phase is to come out with design for the automation activities. The automation includes functional, performance or API level.
Entry Criteria •
“Identification of Test Cases for Automation” phase is completed
Tasks 1. Understand the automation test requirements and identify suitable tool for automation, if customer does not decide the tool. If needed, use Page 14 of 25
Testing process
Srinivas polimera
decision analysis and resolution (DAR) process for selecting a specific tool. 2. Prepare the high level and low level design including automation framework, if it is part of the project scope. Refer to automation guidelines section of testing guidelines document. 3. Conduct reviews with identified team member(s) as per plan. For a specific release, the name of the persons playing the role of internal reviewer, external reviewer and approver will be identified in the “Review Plan” section of the detailed test plan.
Outputs •
Automation Design document(s) (Template can be defined by the projects depending on the type of automation)
Exit criteria •
Automation design document(s) are approved
2.2.7.3 Test automation development The objective of this phase is to develop automation scripts as planned. The automation includes functional, performance or API level.
Entry Criteria • • • •
Test automation has been planned in the Test plan. Test automation infrastructure is ready. Identification of Test Cases for Automation phase is completed. Test Automation design phase is completed if in the scope.
Tasks 1. Automate the identified test cases as per design. 2. Conduct reviews with identified team member(s) as per plan. For a specific release, the name of the persons playing the role of internal reviewer, external reviewer and approver will be identified in the “Review Plan” section of the detailed test plan. 3. Unit test the automation.
Outputs •
Automation test scripts
Exit criteria •
Unit tested Automation test scripts
2.2.7.4 Customer supplied Automated Test Scripts Identification
Page 15 of 25
Testing process
Srinivas polimera
The objective of this phase is to identify the customer supplied automated test scripts for execution. The automation includes functional, performance or API level.
Entry Criteria • •
Test Planning phase is completed Customer supplied automated test scripts are available
Tasks 1. Review all the automated test scripts supplied by customer. Resolve queries if any, with appropriate contact. 2. Depending on the scope of the project, identify the specific automated test scripts for test execution. 3. Conduct reviews with identified team member(s) as per plan. For a specific release, the name of the persons playing the role of internal reviewer, external reviewer and approver will be identified in the “Review Plan” section of the detailed test plan.
Outputs •
Automated Test scripts identified for execution.
Exit criteria •
Identified Automated Test scripts are approved.
2.2.7.5 Test automation execution The objective of this phase is to execute the identified automated test scripts, analyze the results and report defects. The automation includes functional, performance or API level.
Entry Criteria •
TEST AUTOMATION DEVELOPMENT PHASE is completed Identification of Customer supplied Automated Test Scripts execution phase is completed
Or for
Tasks 1. Execute the identified automated test scripts as planned. 2. Analyze the test results. 3. If failures are related to script, correct the script, based on the scope of the project or a release. 4. If updates to the scripts are not in the scope of the project or if the failures are because of application under test, report the bugs in the bug tracking system.
Outputs • •
Automation Test results (Free Form) Updated test scripts.
Page 16 of 25
Testing process
Srinivas polimera
Exit criteria •
Valid bugs entered in to bug tracking system
Measurement • • • •
Effort for automation scripting Effort for automation scripts review Review defects in automation scripts Defects found on execution automation scripts
Verification • •
Review by SQA Review by Customer
Roles & Responsibilities Role PM
•
Test Lead
• • • •
Test members
SQA Customer
team
• • • • •
Responsibilities Identifies the scope and plans for automation Design the test automation framework as applicable Identifies the Test cases to automate along with the testers and customer Reviews the automation scripts and test execution report Reviews the defects to be logged into bug tracking system Prepares the automation script and reviews & unit tests them Executes the script and generate reports Logs the defects into bug tracking system Review test automation activities Reviews the automation scripts and test execution report as required
2.2.8 Test execution and defect reporting phase Test execution and defect reporting phase includes execution of identified test cases and report the defects. Test execution involve the following cycles: • • • • • • • •
Identification of test cases for manual testing Integration test environment validation Integration testing System test environment validation System Testing Ad-hoc testing Regression test environment validation Regression testing
2.2.8.1 Identification of Test Cases for manual testing The objective of this phase is to identify the test cases for manual testing in case of customer supplied test cases.
Entry Criteria Page 17 of 25
Testing process
• •
Srinivas polimera
Test Planning phase is completed Customer supplied test cases are available
Tasks 1. Review all the test cases supplied by customer. Resolve queries if any, with appropriate contact. 2. Depending on the scope of the project, identify the specific test cases for test execution from the customer-supplied test cases. 3. Identify the Sanity test cases for “Sanity testing”. 4. Identify the priority of the test cases as High, Medium or Low, 5. Conduct reviews with identified team member(s) as per plan. For a specific release, the name of the persons playing the role of internal reviewer, external reviewer and approver will be identified in the “Review Plan” section of the detailed test plan.
Outputs •
Test cases identified for test execution.
Exit criteria •
Identified test cases for test execution are approved.
2.2.8.2 Integration test environment validation Entry Criteria •
Communication from Test environment team to indicating readiness of integration test environment.
testing
Team,
Tasks 1. Validate integration test environment for readiness. 2. Create the test data on integration test environment as per plan. 3. In case of any issues, interact with Customer Project Manager / contact and resolve the issues.
Outputs •
Nil
Exit criteria •
Validated integration test environment ready for integration testing.
2.2.8.3 Integration testing Entry Criteria •
Test Preparation Phase completed. Page 18 of 25
Testing process
• •
Srinivas polimera
Integration test environment validation completed Optionally Unit test report from Development team for the modules under integration testing is available.
Tasks 1. Execute all the test cases identified for integration testing on integration test environment, as planned. 2. Send the Integration Test report (Test case wise Pass Fail report) and test status report at a frequency as planned in the test plan. The format of test report is same as Test case template but with result column is filled with appropriate results.
Outputs • •
Integration Test Report (Test case wise Pass Fail report) Test status report
Exit criteria •
Integration testing completed as planned.
Verification •
Test lead verifies the test execution on sampling basis.
2.2.8.4 System test environment validation Entry Criteria •
Communication from Test environment team indicating readiness of system test environment.
to
testing
Team,
Tasks 1. Validate system test environment for readiness. 2. Create the test data on system test environment as per plan. 3. In case of any issues, interact with Customer Project Manager / contact and resolve the issues.
Outputs •
Nil
Exit criteria •
Validated System test environment ready for system testing.
Verification •
Test lead verifies the test execution on sampling basis.
Page 19 of 25
Testing process
Srinivas polimera
2.2.8.5 System Testing Entry Criteria • • •
Test Preparation Phase completed. System test environment validation completed Optionally Unit test report from Development team is available.
Testing cycle System test cycle 1
-
System test cycle n
-
Entry criteria Execute all Sanity test cases and at least 80 % of the sanity test cases have passed (or as defined in the test plan) System test cycle (n – 1) exit criteria are met. Sanity test cases and at least 80 % of the sanity test cases have passed (or as defined in the test plan).
Tasks 1. For each of the test cycles Entry criteria are mentioned above. 2. If any of the test cases are not planned to be executed as part of any cycle, get the approval from Customer Project Manager / contact 3. Execute the above identified, approved test cases and verify the resolved bugs during all System test cycles. 4. Report all the bugs found. 5. Identify the Regression test cases for regression testing, as the system test cycle progresses. 6. Conduct review with identified team member(s) as per plan. For a specific release, the name of the persons playing the role of internal reviewer, external reviewer and approver will be identified in the “Review Plan” section of the detailed test plan. 7. Send the System Test report (Test case wise Pass Fail report) and test status report at a frequency as planned in the test plan. The format of test report is same as Test case template but with result column is filled with appropriate results.
Outputs • •
System Test report (Test case wise Pass Fail report) Test status report
Exit Criteria
Page 20 of 25
Testing process
Testing cycle System test cycle 1
Srinivas polimera
-
System test cycle n (Last cycle)
-
Exit criteria At least system test cases identified as High priority have been completed. For any other reasons, Approval from Customer Project Manager / contact is required to exit system test cycle 1. All the identified system test cases have been executed at least once.
-
Number of defects requiring action from Development or testing team should be as per acceptance criteria defined in the test plan.
-
If there are any deviations, approval from Customer Project Manager / contact is required to exit system test cycle n.
-
Approved Regression test cases document.
Verification •
Test lead verifies the test execution on sampling basis.
2.2.8.6 Ad-hoc Testing The objective of this phase is to perform ad-hoc testing on need basis. Hence this can be a planned or an unplanned activity.
Entry Criteria: •
Knowledge transfer phase completed.
Tasks 1. Ad-hoc testing can be performed at any phase. For example, if the defect trend in any system cycle indicates a specific module / functionality to be tested further, Ad-hoc testing can be planned 2. For all failed scenarios, if there are no test cases either create new test cases or update existing test cases.
Outputs •
Test report (Free Form)
Exit Criteria •
Report all the bugs found as per Defect tracking policy
Verification •
Test lead verifies the test execution on sampling basis. Page 21 of 25
Testing process
Srinivas polimera
2.2.8.7 Regression test environment validation Entry Criteria •
Communication from Test environment team to indicating readiness of regression test environment.
testing
Team,
Tasks 1. Validate regression test environment for readiness. 2. Create the test data on regression test environment as per plan. 3. In case of any issues, interact with Customer Project Manager / contact and resolve the issues.
Outputs •
Nil
Exit criteria • •
Validated regression test environment ready for regression testing.
2.2.8.8 Regression testing Entry Criteria Testing cycle Regression test cycle
Entry criteria - Regression test environment validation exit criteria is met - System test cycle n exit criteria are met.
Tasks 1. Execute all the sanity test cases. 100% of the sanity test cases should pass. In case of any issues / deviations, Test Manager / Lead to interact with Customer Project Manager / contact and resolve the issues / get the approval for deviations. 2. If any of the Regression test cases are not planned to be executed, get the approval from Customer Project Manager / contact 3. Execute all the test cases identified for regression testing as per plan and verify the resolved bugs. 4. Report all the bugs found. 5. Bugs found during regression testing will be fixed only with the approval of the Customer Project Manager / contact. It is assumed that this Regression testing cycle is the final cycle. 6. Send the Regression Test report (Test case wise Pass Fail report) and test status report at a frequency as planned in the test plan. The format of test report is same as Test case template but with result column is filled with appropriate results.
Outputs • •
Regression Test report (Test case wise Pass Fail report) Test status report Page 22 of 25
Testing process
Srinivas polimera
Exit Criteria • •
All the Identified Regression test cases have been executed. For any other reasons, approval from Customer Project Manager / contact is required to exit Regression test cycle.
Verification •
Test lead verifies the test execution on sampling basis.
2.2.8.9 Measurement • Effort for Test Execution • •
Defects Reports Reports from RCA & DP
2.2.8.10Verification • • •
Review by Test Lead Review by Customer Review by SQA
2.2.8.11Roles & Responsibilities
Role PM
Test Lead
Test team members
• • • • • • • • • •
SQA
• • • • •
Customer
• •
Responsibilities Prepares for various tests to be performed, identifies Validates various test environments Monitors test execution & reviews the status Reviews the defect reports Resolve issues if any Plans and conducts RCA & DP Along with PM validates various test environments Monitors test execution & reviews the defects Verifies defects logged into bug tracking system Prepares various reports, inputs required for RCA & DP Validates test environment Creates test data Executes various tests and report defects Participate in RCA & DP Reviews the test execution activities and defect reports Participate in RCA & DP as required Reviews the test execution status report, defects reported/ defect reports as required
Page 23 of 25
Testing process
Srinivas polimera
2.2.9 Test certification 2.2.9.1 Entry Criteria • •
The entire planned test for a specific project or a release has been completed. Acceptance criteria as agreed with customer have been met. For any deviations, approval from Customer Project Manager / contact is required, to enter this phase.
2.2.9.2 Tasks 1. Prepare the Test certificate, as per template. 2. Conduct review with identified team member(s) as per plan. For a specific release, the name of the persons playing the role of internal reviewer, external reviewer and approver will be identified in the “Review Plan” section of the detailed test plan. 3. Send the test certificate to the customer.
2.2.9.3 Outputs •
Test certificate
2.2.9.4 Exit Criteria •
Test certificate sent it to the customer.
2.2.9.5 Measurement •
Effort for preparing Test Certificate
2.2.9.6 Verification • • •
Review by PM, Test Lead Review by Customer Review by SQA
2.2.9.7 Roles & Responsibilities Role Delivery Unit Head PM, Test Lead
SQA
•
Responsibilities Reviews the overall status Verifies acceptance criteria as agreed with customer have been met Prepare the Test certificate
•
Reviews the certificate activities
• •
2.2.10Test Closure Phase Once all the testing tasks as planned for a project or a release are completed, a formal test project post-mortem report will be prepared and shared with project team. This will cover the following areas: •
What went well and What did not go well Page 24 of 25
Testing process
• •
Srinivas polimera
Analysis of metrics collected Lessons learnt
2.2.10.1Entry Criteria •
Test Execution and Defect Reporting Phase completed.
2.2.10.2Tasks 1. Prepare Test Project Post-mortem report. 2. Conduct post-mortem meeting with participation from Project team.
2.2.10.3Outputs: •
Test Project Post-mortem report
2.2.10.4Exit criteria •
Formal post-mortem session conducted.
2.2.10.5Measurement • •
Effort spent on test project closure activities No. Of best practices contributed to the organization repository
2.2.10.6Verification • • •
Review by PM Review by Customer Review by SQA
2.2.10.7Roles & Responsibilities Role Delivery Unit Head
•
Project Manager
• • •
Test Lead Developer Test team members SQA
• • • •
Responsibilities Reviews the overall Testing activities conducted Prepares the Test Project Post-mortem report Validates the data and inputs for Post-mortem Updates the project repository with lessons learnt & best practices Provides inputs of Post-mortem Participates in Post-mortem meeting Participates in Post-mortem meeting Verifies the post-mortem report
Page 25 of 25
Testing process
Srinivas polimera
CONTENTS
1. Introduction ......................................................................................................... ...............3 1.1 Purpose............................................................................................................. .............3 1.2 Scope........................................................................................................... ..................3 1.3 Definitions and Acronyms.................................................................................... ...........3 1.4 References....................................................................................................... ..............3 1.4.1 General................................................................................................................ ....3 1.4.2 Templates............................................................................................... .................3 1.4.3 Guidelines.................................................................................... ...........................4 1.4.4 Checklist........................................................................................................ ..........4 2. Testing Process.................................................................................................................. .5 2.1 Workflow....................................................................................................................... ..5 2.2 Process Description........................................................................... ............................6 2.2.1 Project Management Process..................................................... ............................6 2.2.2 Test Initiation phase....................................................................................... ..........6 2.2.3 Knowledge transfer phase............................................................................. ..........7 2.2.4 Test strategy Phase.................................................................................. ...............8 2.2.5 Test Planning Phase...................................................................................... ..........9 2.2.6 Test Design phase........................................................................... ......................11 2.2.7 Test Automation phase.................................................................... ......................14 2.2.8 Test execution and defect reporting phase.......................................... ..................17 2.2.9 Test certification........................................................................................... .........24 2.2.10 Test Closure Phase.................................................................. ...........................24
Page 2 of 25
Testing process
Srinivas polimera
1. Introduction 1.1Purpose Purpose of this document is to lay down a standard approach to testing which can be followed by all projects. The scope of the document starts with the initiation of the testing activities and ends with the test Closure Phase.
1.2Scope The process applies to all the members who directly/indirectly involved in testing activities in marlabs .
1.3Definitions and Acronyms PEG CMMI SQA QMS PCR PDSP HR PM TM PIF RCA DP NC KT
Process Engineering Group Capability Maturity Model Integration Software Quality Assurance Quality Management System Process Change Request Project Defined Software Process Human Resource Project Manager Test Manager Project Initiation Form Root Cause Analysis Defect Prevention Non Conformance Knowledge transfer
1.4References This section provides a complete list of all documents that need to be referenced for proper interpretation of process.
1.4.1 General Capability Maturity Model® Integration (CMMISM), Version 1.1 •
TM can perform the responsibilities defined in this document for PM.
•
The role “Customer Project Manager / Contact” is used throughout this document. The person playing this role will be defined in the test plan. The person could be from Customer Project Organization or from the Project Organization. This role is significant as he / she will be involved in making key decisions during the test life cycle.
•
It is recommended that a Tech Lead in the project play the role “Development Representative” referred through out the document.
1.4.2 Templates • •
Test Requirements Specification Template Test Strategy Template Page 3 of 25
Testing process
• • • • • • • •
Srinivas polimera
Test Plan Template Impacted External Systems Communication Plan Test case / Test report Template Traceability Matrix Automation Design Template (Project defined template) Test Status Report Test Certificate Test Project Post-mortem Report
1.4.3 Guidelines • •
Testing Guidelines. Measurement Guidelines.
1.4.4 Checklist • •
Test case review Checklist. Test Quality Index Checklist
Page 4 of 25
Testing process
Srinivas polimera
2. Testing Process 2.1Workflow Test Initiation Phase
Know ledge Transfer Phase
Test StrategyPhase
Test Planning Phase
TestDesignphase High-level scenarios preparation Identification of impacted external systems Detailed Test Cases preparation Test Data Preparation
Test Execution and Defect Reporting Phase Identification of test cases for manual testing Integration test environment validation Integration testing System test environment validation System Testing Ad-hoc testing Regression test environment validation Regression testing
Test Autom ationPhase
Test case identification for automation Test automation design Test automation development Customer supplied automated script selection Test automation execution
TestCertificationPhase
TestClosure Phase
Page 5 of 25
Testing process
Srinivas polimera
2.2Process Description 2.2.1 Project Management Process Refer to Project Management process document in PQMG portal for details
on
• • • • •
Project initiation Process Project Planning Project Execution and Tracking Risk Management Project Closure
All the process defined in the Project Management process are applicable for Testing projects also. In case of pure testing project, Test plan can be an appendix to Project Management Plan.
2.2.2 Test Initiation phase Once a specific project or a release is identified for testing, the test initiation phase will begin. The objective of this phase is to review the test requirements and prepare initial test plan.
2.2.2.1 Entry Criteria • • • •
Project Initiation phase completed Optionally Test Requirements document / Software requirements document exists. Test Manager / Lead identified to head the testing part of the project. Customer Project Manager / contact identified for coordinating testing part of the project.
2.2.2.2 Tasks 1. Review the Test Requirements document / Software requirements document. Use the “Guidelines for collecting Test Requirements” in Testing Guidelines document and prepare or update the test requirements document. 2. The test requirements document shall cover the requirements for manual testing, automation testing, performance testing, white box testing, etc. 3. Prepare the Initial test plan. The template will be same as detailed test plan. At least the sections “Introduction, Scope of testing, Knowledge Transfer plan” to be completed. The knowledge transfer plan shall address needs of manual testing, automation testing, performance testing, white box testing, etc.
4. All stakeholders relevant to testing activities have to be identified as part of test initiation and involved at appropriate way.
Page 6 of 25
Testing process
Srinivas polimera
2.2.2.3 Exit criteria • •
Updated Test Requirements document Initial test plan
2.2.2.4 Measurement •
Effort for preparing initial Test plan
2.2.2.5 Verification • •
Review by PM Review by SQA
2.2.2.6 Roles & Responsibilities Role PM Test Lead SQA
Responsibilities Review and approve Test requirements and initial test plan Prepare Test requirements and initial test plan Review initial test plan
2.2.3 Knowledge transfer phase The objective of this phase is to train the testing Team member(s) on requirements and functionalities that are part of the project or a release. This includes training needs to address manual testing, automation testing, performance testing, white box testing, etc.
2.2.3.1 Entry Criteria •
Test initiation phase completed
2.2.3.2 Tasks 1. Schedule meetings with identified persons for the knowledge transfer session as per Knowledge Transfer plan. 2. Prepare for the knowledge transfer. Identified Testing team member(s) to go through the Test requirements and other relevant documents prior to knowledge transfer session. If there are any special requirements, they will be identified in Knowledge Transfer plan section of Test Plan document. 3. Coordinate the Knowledge transfer sessions as planned in Knowledge Transfer plan section of Test Plan document.
4. Prepare a list of documents, which will be used as inputs during
subsequent phases. Send it to the Customer Project Manager / contact, for confirmation.
Page 7 of 25
Testing process
Srinivas polimera
2.2.3.3 Exit criteria •
List of documents that will be used for preparation of Test cases (No template, it can be a free form)
2.2.3.4 Measurement •
Effort spent on KT
2.2.3.5 Verification • • •
Review by PM Review by Customer Review by SQA
2.2.3.6 Roles & Responsibilities Responsibilities
Role Unit Deliver Head PM Test Lead Test team members SQA Customer
• • • • • • •
Resolve escalated issues during KT phase Review KT activities Co-ordinate all KT activities Confirms the completion of KT activities Understand the requirements during KT Audit KT activities Provide the KT inputs
2.2.4 Test strategy Phase The objective of this phase is to develop test strategy for a project or for a specific release. This phase is an optional phase depending the scope of the project.
2.2.4.1 Entry Criteria • • •
Test Initiation phase completed. Optionally Knowledge transfer is also completed, but not mandatory. Optionally Design document exists.
2.2.4.2 Tasks 1. Go through the test requirements document and other relevant documents to understand the project requirements. 2. Optionally schedule meetings with the customer and get additional information and resolve queries if any. 3. Prepare the test strategy document. Refer to the test strategy section of testing guidelines document. If needed, the test strategy document can be separate for manual testing, automation testing, performance testing, white box testing, etc. Page 8 of 25
Testing process
Srinivas polimera
4. Internally review the test strategy document and incorporate the review comments. 5. Optionally present / review the test strategy document with customer and incorporate the review comments.
2.2.4.3 Exit criteria •
Test Strategy document
2.2.4.4 Measurement •
Effort for designing test strategy
2.2.4.5 Verification • • • •
Review Review Review Review
by by by by
PM Development Representative (Optional) Customer SQA
2.2.4.6 Roles & Responsibilities Responsibilities
Role Test Architect / Test Manager / Test Lead
Prepare the Test Strategy
PM
Review and approve the test strategy
Development Representative
Optionally review the test strategy
Customer
Optionally present / review the test strategy with customer
SQA
Audit the Test Strategy activities
2.2.5 Test Planning Phase The objective of this phase is to prepare a detailed test plan for testing.
2.2.5.1 Entry Criteria •
Knowledge transfer phase completed
2.2.5.2 Tasks 1. Update the test plan (word document) prepared during Test Initiation phase and completes the remaining sections.
Page 9 of 25
Testing process
Srinivas polimera
2. If preparation of detailed test strategy document is not in the scope of the project, test strategy shall be a section within the test plan. The test strategy shall address the needs of manual testing, automation testing, performance testing, white box testing, etc. Refer to the test strategy section of the testing guidelines document for guidelines. 3. Metrics: Identify and Plan for collecting the metrics in the test plan. Refer to “Measurement guidelines” for details. 4. Plan for interim project reviews. Objective of this activity is to review the project progress. Testing Interim Project Review Template shall be used for this purpose. 5. Conduct reviews with identified team member(s) as per plan. For a specific project or a release, the name of the persons playing the role of internal reviewer, external reviewer and approver will be identified in the “Review Plan” section of the detailed test plan. 6. Test Plan/PMP shall mention the work product and type of verification (testing) to be performed. 7. Test Plan/PMP shall identify relevant resources to perform testing. 8. Configuration management activities have to be planned and performed as part of testing. All the relevant configurable items related to testing has to identified and controlled 9. Test manager/PM shall monitor and control the testing activities to achieve the project objectives and take corrective actions if any deviation found during execution.
2.2.5.3 Exit criteria •
Baselined test plan
2.2.5.4 Measurement •
Effort for preparing Test plan
2.2.5.5 Verification • • •
Review by PM Review by Customer Review by SQA
2.2.5.6 Roles & Responsibilities Responsibilities
ROLE PM Development Representative Test Manager / Test Lead Test team members SQA
• • • • • •
Review and approve test plan Resolve escalated issues Optional Provide inputs for test plan Review test plan Prepare test plan
• • •
Provide inputs for test plan Follow test plan Review test plan Page 10 of 25
Testing process
Srinivas polimera
Responsibilities
ROLE Customer
• • •
Optional Review test plan Approve test plan
2.2.6 Test Design phase The objective of this phase is to prepare for test execution activities. Planned tasks as part of this phase will include • • • •
High-Level scenarios preparation Identification of impacted external systems Detailed test cases preparation Test data preparation
2.2.6.1 High-level scenarios preparation Entry Criteria • • •
Test Strategy phase is completed (if in the scope). Detailed test planning phase completed. List documents, e.g. Requirements document, Design document, product specification etc. as agreed in Knowledge transfer phase for test design are available. Refer to Knowledge transfer phase for more details.
Tasks 1. Prepare “High-Level Scenarios / Initial test cases” as per plan, to address the needs of manual testing, automation testing, performance testing, etc. The template to be used is same Test case template. At least the columns “Test case description” and “Expected Result” columns to be filled up. Refer to the test case design section of testing guidelines document for guidelines.
2. Conduct reviews with identified team member(s) as per plan. For a specific project or a release, the name of the persons playing the role of internal reviewer, external reviewer and approver will be identified in the “Review Plan” section of the detailed test plan.
Outputs •
“High-Level scenarios / Initial test cases” document ready for all the different kinds of testing planned for the project or a release.
Exit criteria •
Approved “High-level scenarios / Initial test cases”.
Page 11 of 25
Testing process
Srinivas polimera
2.2.6.2 Identification of impacted external systems Entry Criteria •
High Level Scenarios preparation phase completed.
Tasks 1. Identify and prepare list of external systems that will be impacted
during test execution phase as per “Impacted external systems communication plan” template.
2. Initiate contact with the external system owners along with plan for support during test execution phase.
Outputs •
Impacted external systems communication plan
Exit criteria •
Confirmation from external system owners for support during test execution phase, if applicable
2.2.6.3 Detailed Test Cases preparation Entry Criteria •
High Level Scenarios preparation phase completed.
Tasks 1. Prepare Detailed Test Cases by updating the high level scenarios / initial test cases prepared during the HIGH LEVEL SCENARIOS PREPARATION PHASE. Refer to the test case design section of testing guidelines document for guidelines.
2. Identify the Sanity test cases for “Sanity testing”. 3. Identify the priority of the test cases as High, Medium or Low. 4. Update the traceability matrix in cases of development project (where one exists). In case of pure testing project create a new traceability matrix, tracing test cases to Requirements and vice versa. 5. Conduct reviews with identified team member(s) as per plan. For a specific release, the name of the persons playing the role of internal reviewer, external reviewer and approver will be identified in the “Review Plan” section of the detailed test plan.
Outputs •
Detailed test cases document Page 12 of 25
Testing process
•
Srinivas polimera
Traceability Matrix
Exit criteria •
Approved “Detailed Test Cases” document.
2.2.6.4 Test data preparation Entry Criteria •
Detailed Test Cases Preparation Phase completed.
Tasks 1. Prepare “Test data” for all applicable test types. 2. In case if customer supplies Test data, Review the test data and resolve queries if any, with appropriate contact. 3. Conduct reviews with identified team member(s) as per plan. For a specific release, the name of the persons playing the role of internal reviewer, external reviewer and approver will be identified in the “Review Plan” section of the detailed test plan.
Outputs •
Test data
Exit criteria •
Test Data is ready.
Measurement • • •
Effort for test case and data preparation. Effort for test case review. Review defects found in test cases review.
Verification • • •
Review by Development Representative (Optional) Review by Customer Review by SQA
Roles & Responsibilities Role PM Test team members
• •
SQA Customer
• •
Responsibilities Track the status and resolve issues Prepare the test deliverables as referenced in this section. Audit test preparation activities Review and approve test deliverables as referenced in this section (if applicable)
Page 13 of 25
Testing process
Srinivas polimera
2.2.7 Test Automation phase The objective of this phase is to identify the test cases for automation, design for automation, develop automation scripts as per plan, execute the automation scripts, analyze and report results. The test requirements for automation are already covered in the test initiation phase.
2.2.7.1 Identification of Test Cases for Automation The objective of this phase is to identify the test cases for automation in case of customer-supplied test cases or test cases developed in house. The automation includes functional, performance or API level.
Entry Criteria • •
Test Planning phase is completed Detailed Test Case Preparation phase is completed or Customer supplied test cases are available.
Tasks 1. Review all the test cases from automation perspective (either customer supplied test cases or in house developed test cases). Resolve queries if any, with appropriate contact. 2. Identify the test cases for automation. Please refer to the automation guidelines section in the testing guidelines document. 3. Conduct reviews with identified team member(s) as per plan. For a specific release, the name of the persons playing the role of internal reviewer, external reviewer and approver will be identified in the “Review Plan” section of the detailed test plan.
Outputs •
Test cases identified for automation
Exit criteria •
Identified test cases for automation are approved.
2.2.7.2 Test Automation Design The objective of this phase is to come out with design for the automation activities. The automation includes functional, performance or API level.
Entry Criteria •
“Identification of Test Cases for Automation” phase is completed
Tasks 1. Understand the automation test requirements and identify suitable tool for automation, if customer does not decide the tool. If needed, use Page 14 of 25
Testing process
Srinivas polimera
decision analysis and resolution (DAR) process for selecting a specific tool. 2. Prepare the high level and low level design including automation framework, if it is part of the project scope. Refer to automation guidelines section of testing guidelines document. 3. Conduct reviews with identified team member(s) as per plan. For a specific release, the name of the persons playing the role of internal reviewer, external reviewer and approver will be identified in the “Review Plan” section of the detailed test plan.
Outputs •
Automation Design document(s) (Template can be defined by the projects depending on the type of automation)
Exit criteria •
Automation design document(s) are approved
2.2.7.3 Test automation development The objective of this phase is to develop automation scripts as planned. The automation includes functional, performance or API level.
Entry Criteria • • • •
Test automation has been planned in the Test plan. Test automation infrastructure is ready. Identification of Test Cases for Automation phase is completed. Test Automation design phase is completed if in the scope.
Tasks 1. Automate the identified test cases as per design. 2. Conduct reviews with identified team member(s) as per plan. For a specific release, the name of the persons playing the role of internal reviewer, external reviewer and approver will be identified in the “Review Plan” section of the detailed test plan. 3. Unit test the automation.
Outputs •
Automation test scripts
Exit criteria •
Unit tested Automation test scripts
2.2.7.4 Customer supplied Automated Test Scripts Identification
Page 15 of 25
Testing process
Srinivas polimera
The objective of this phase is to identify the customer supplied automated test scripts for execution. The automation includes functional, performance or API level.
Entry Criteria • •
Test Planning phase is completed Customer supplied automated test scripts are available
Tasks 1. Review all the automated test scripts supplied by customer. Resolve queries if any, with appropriate contact. 2. Depending on the scope of the project, identify the specific automated test scripts for test execution. 3. Conduct reviews with identified team member(s) as per plan. For a specific release, the name of the persons playing the role of internal reviewer, external reviewer and approver will be identified in the “Review Plan” section of the detailed test plan.
Outputs •
Automated Test scripts identified for execution.
Exit criteria •
Identified Automated Test scripts are approved.
2.2.7.5 Test automation execution The objective of this phase is to execute the identified automated test scripts, analyze the results and report defects. The automation includes functional, performance or API level.
Entry Criteria •
TEST AUTOMATION DEVELOPMENT PHASE is completed Identification of Customer supplied Automated Test Scripts execution phase is completed
Or for
Tasks 1. Execute the identified automated test scripts as planned. 2. Analyze the test results. 3. If failures are related to script, correct the script, based on the scope of the project or a release. 4. If updates to the scripts are not in the scope of the project or if the failures are because of application under test, report the bugs in the bug tracking system.
Outputs • •
Automation Test results (Free Form) Updated test scripts.
Page 16 of 25
Testing process
Srinivas polimera
Exit criteria •
Valid bugs entered in to bug tracking system
Measurement • • • •
Effort for automation scripting Effort for automation scripts review Review defects in automation scripts Defects found on execution automation scripts
Verification • •
Review by SQA Review by Customer
Roles & Responsibilities Role PM
•
Test Lead
• • • •
Test members
SQA Customer
team
• • • • •
Responsibilities Identifies the scope and plans for automation Design the test automation framework as applicable Identifies the Test cases to automate along with the testers and customer Reviews the automation scripts and test execution report Reviews the defects to be logged into bug tracking system Prepares the automation script and reviews & unit tests them Executes the script and generate reports Logs the defects into bug tracking system Review test automation activities Reviews the automation scripts and test execution report as required
2.2.8 Test execution and defect reporting phase Test execution and defect reporting phase includes execution of identified test cases and report the defects. Test execution involve the following cycles: • • • • • • • •
Identification of test cases for manual testing Integration test environment validation Integration testing System test environment validation System Testing Ad-hoc testing Regression test environment validation Regression testing
2.2.8.1 Identification of Test Cases for manual testing The objective of this phase is to identify the test cases for manual testing in case of customer supplied test cases.
Entry Criteria Page 17 of 25
Testing process
• •
Srinivas polimera
Test Planning phase is completed Customer supplied test cases are available
Tasks 1. Review all the test cases supplied by customer. Resolve queries if any, with appropriate contact. 2. Depending on the scope of the project, identify the specific test cases for test execution from the customer-supplied test cases. 3. Identify the Sanity test cases for “Sanity testing”. 4. Identify the priority of the test cases as High, Medium or Low, 5. Conduct reviews with identified team member(s) as per plan. For a specific release, the name of the persons playing the role of internal reviewer, external reviewer and approver will be identified in the “Review Plan” section of the detailed test plan.
Outputs •
Test cases identified for test execution.
Exit criteria •
Identified test cases for test execution are approved.
2.2.8.2 Integration test environment validation Entry Criteria •
Communication from Test environment team to indicating readiness of integration test environment.
testing
Team,
Tasks 1. Validate integration test environment for readiness. 2. Create the test data on integration test environment as per plan. 3. In case of any issues, interact with Customer Project Manager / contact and resolve the issues.
Outputs •
Nil
Exit criteria •
Validated integration test environment ready for integration testing.
2.2.8.3 Integration testing Entry Criteria •
Test Preparation Phase completed. Page 18 of 25
Testing process
• •
Srinivas polimera
Integration test environment validation completed Optionally Unit test report from Development team for the modules under integration testing is available.
Tasks 1. Execute all the test cases identified for integration testing on integration test environment, as planned. 2. Send the Integration Test report (Test case wise Pass Fail report) and test status report at a frequency as planned in the test plan. The format of test report is same as Test case template but with result column is filled with appropriate results.
Outputs • •
Integration Test Report (Test case wise Pass Fail report) Test status report
Exit criteria •
Integration testing completed as planned.
Verification •
Test lead verifies the test execution on sampling basis.
2.2.8.4 System test environment validation Entry Criteria •
Communication from Test environment team indicating readiness of system test environment.
to
testing
Team,
Tasks 1. Validate system test environment for readiness. 2. Create the test data on system test environment as per plan. 3. In case of any issues, interact with Customer Project Manager / contact and resolve the issues.
Outputs •
Nil
Exit criteria •
Validated System test environment ready for system testing.
Verification •
Test lead verifies the test execution on sampling basis.
Page 19 of 25
Testing process
Srinivas polimera
2.2.8.5 System Testing Entry Criteria • • •
Test Preparation Phase completed. System test environment validation completed Optionally Unit test report from Development team is available.
Testing cycle System test cycle 1
-
System test cycle n
-
Entry criteria Execute all Sanity test cases and at least 80 % of the sanity test cases have passed (or as defined in the test plan) System test cycle (n – 1) exit criteria are met. Sanity test cases and at least 80 % of the sanity test cases have passed (or as defined in the test plan).
Tasks 1. For each of the test cycles Entry criteria are mentioned above. 2. If any of the test cases are not planned to be executed as part of any cycle, get the approval from Customer Project Manager / contact 3. Execute the above identified, approved test cases and verify the resolved bugs during all System test cycles. 4. Report all the bugs found. 5. Identify the Regression test cases for regression testing, as the system test cycle progresses. 6. Conduct review with identified team member(s) as per plan. For a specific release, the name of the persons playing the role of internal reviewer, external reviewer and approver will be identified in the “Review Plan” section of the detailed test plan. 7. Send the System Test report (Test case wise Pass Fail report) and test status report at a frequency as planned in the test plan. The format of test report is same as Test case template but with result column is filled with appropriate results.
Outputs • •
System Test report (Test case wise Pass Fail report) Test status report
Exit Criteria
Page 20 of 25
Testing process
Testing cycle System test cycle 1
Srinivas polimera
-
System test cycle n (Last cycle)
-
Exit criteria At least system test cases identified as High priority have been completed. For any other reasons, Approval from Customer Project Manager / contact is required to exit system test cycle 1. All the identified system test cases have been executed at least once.
-
Number of defects requiring action from Development or testing team should be as per acceptance criteria defined in the test plan.
-
If there are any deviations, approval from Customer Project Manager / contact is required to exit system test cycle n.
-
Approved Regression test cases document.
Verification •
Test lead verifies the test execution on sampling basis.
2.2.8.6 Ad-hoc Testing The objective of this phase is to perform ad-hoc testing on need basis. Hence this can be a planned or an unplanned activity.
Entry Criteria: •
Knowledge transfer phase completed.
Tasks 1. Ad-hoc testing can be performed at any phase. For example, if the defect trend in any system cycle indicates a specific module / functionality to be tested further, Ad-hoc testing can be planned 2. For all failed scenarios, if there are no test cases either create new test cases or update existing test cases.
Outputs •
Test report (Free Form)
Exit Criteria •
Report all the bugs found as per Defect tracking policy
Verification •
Test lead verifies the test execution on sampling basis. Page 21 of 25
Testing process
Srinivas polimera
2.2.8.7 Regression test environment validation Entry Criteria •
Communication from Test environment team to indicating readiness of regression test environment.
testing
Team,
Tasks 1. Validate regression test environment for readiness. 2. Create the test data on regression test environment as per plan. 3. In case of any issues, interact with Customer Project Manager / contact and resolve the issues.
Outputs •
Nil
Exit criteria • •
Validated regression test environment ready for regression testing.
2.2.8.8 Regression testing Entry Criteria Testing cycle Regression test cycle
Entry criteria - Regression test environment validation exit criteria is met - System test cycle n exit criteria are met.
Tasks 1. Execute all the sanity test cases. 100% of the sanity test cases should pass. In case of any issues / deviations, Test Manager / Lead to interact with Customer Project Manager / contact and resolve the issues / get the approval for deviations. 2. If any of the Regression test cases are not planned to be executed, get the approval from Customer Project Manager / contact 3. Execute all the test cases identified for regression testing as per plan and verify the resolved bugs. 4. Report all the bugs found. 5. Bugs found during regression testing will be fixed only with the approval of the Customer Project Manager / contact. It is assumed that this Regression testing cycle is the final cycle. 6. Send the Regression Test report (Test case wise Pass Fail report) and test status report at a frequency as planned in the test plan. The format of test report is same as Test case template but with result column is filled with appropriate results.
Outputs • •
Regression Test report (Test case wise Pass Fail report) Test status report Page 22 of 25
Testing process
Srinivas polimera
Exit Criteria • •
All the Identified Regression test cases have been executed. For any other reasons, approval from Customer Project Manager / contact is required to exit Regression test cycle.
Verification •
Test lead verifies the test execution on sampling basis.
2.2.8.9 Measurement • Effort for Test Execution • •
Defects Reports Reports from RCA & DP
2.2.8.10Verification • • •
Review by Test Lead Review by Customer Review by SQA
2.2.8.11Roles & Responsibilities
Role PM
Test Lead
Test team members
• • • • • • • • • •
SQA
• • • • •
Customer
• •
Responsibilities Prepares for various tests to be performed, identifies Validates various test environments Monitors test execution & reviews the status Reviews the defect reports Resolve issues if any Plans and conducts RCA & DP Along with PM validates various test environments Monitors test execution & reviews the defects Verifies defects logged into bug tracking system Prepares various reports, inputs required for RCA & DP Validates test environment Creates test data Executes various tests and report defects Participate in RCA & DP Reviews the test execution activities and defect reports Participate in RCA & DP as required Reviews the test execution status report, defects reported/ defect reports as required
Page 23 of 25
Testing process
Srinivas polimera
2.2.9 Test certification 2.2.9.1 Entry Criteria • •
The entire planned test for a specific project or a release has been completed. Acceptance criteria as agreed with customer have been met. For any deviations, approval from Customer Project Manager / contact is required, to enter this phase.
2.2.9.2 Tasks 1. Prepare the Test certificate, as per template. 2. Conduct review with identified team member(s) as per plan. For a specific release, the name of the persons playing the role of internal reviewer, external reviewer and approver will be identified in the “Review Plan” section of the detailed test plan. 3. Send the test certificate to the customer.
2.2.9.3 Outputs •
Test certificate
2.2.9.4 Exit Criteria •
Test certificate sent it to the customer.
2.2.9.5 Measurement •
Effort for preparing Test Certificate
2.2.9.6 Verification • • •
Review by PM, Test Lead Review by Customer Review by SQA
2.2.9.7 Roles & Responsibilities Role Delivery Unit Head PM, Test Lead
SQA
•
Responsibilities Reviews the overall status Verifies acceptance criteria as agreed with customer have been met Prepare the Test certificate
•
Reviews the certificate activities
• •
2.2.10Test Closure Phase Once all the testing tasks as planned for a project or a release are completed, a formal test project post-mortem report will be prepared and shared with project team. This will cover the following areas: •
What went well and What did not go well Page 24 of 25
Testing process
• •
Srinivas polimera
Analysis of metrics collected Lessons learnt
2.2.10.1Entry Criteria •
Test Execution and Defect Reporting Phase completed.
2.2.10.2Tasks 1. Prepare Test Project Post-mortem report. 2. Conduct post-mortem meeting with participation from Project team.
2.2.10.3Outputs: •
Test Project Post-mortem report
2.2.10.4Exit criteria •
Formal post-mortem session conducted.
2.2.10.5Measurement • •
Effort spent on test project closure activities No. Of best practices contributed to the organization repository
2.2.10.6Verification • • •
Review by PM Review by Customer Review by SQA
2.2.10.7Roles & Responsibilities Role Delivery Unit Head
•
Project Manager
• • •
Test Lead Developer Test team members SQA
• • • •
Responsibilities Reviews the overall Testing activities conducted Prepares the Test Project Post-mortem report Validates the data and inputs for Post-mortem Updates the project repository with lessons learnt & best practices Provides inputs of Post-mortem Participates in Post-mortem meeting Participates in Post-mortem meeting Verifies the post-mortem report
Page 25 of 25
Related Documents
Testing Process
November 2019 18
Testing Process
November 2019 16
Testing Process
October 2019 17
Coding And Testing Process
November 2019 14
Performance Testing Process
October 2019 18