Domain 10 Planning Process

PLANNING PROCESS CBOK Domain 10 By Marta Heinrich

Process Cycle Comprised of 4 parts: Plan (P): Devise a plan ■ Do (D): Execute the plan ■ Check (C): Check the results ■ Action (A): Take the necessary action ■

Process Cycle ■

The role of testing is to fulfill the ‘Check’ responsibilities

■

It is not to determine whether software can be placed into production

Process Cycle ■

90% of all defects are caused by process problems •

■

Dr. W. Edwards Deming

Anytime a process is used, defects of approximately the same type and frequency occur

Process Cycle ■

Testing, as performed in most organizations, is a process designed to compensate for an ineffective software development process.

Process Cycle ■

The test process must be designed to provide as much value as possible in the time available

Process Cycle ■

Testing is no longer an ‘after-thought’ to the software development life cycle, but rather is a key part of it.

Test Planning Activities Comprised of 3 parts: ■ Pre-Planning Activities ■ Test Planning ■ Post Planning Activities

Pre-Planning Activities ■

Success Criteria/Acceptance Criteria

■

Test Objectives

■

Assumptions

■

Entrance/Exit Criteria

Pre-Planning Activities Success Criteria/Acceptance Criteria ■ Criteria that must be validated through testing to provide the information needed to make an acceptance decision.

Pre-Planning Activities Success Criteria/Acceptance Criteria ■ Met during each phase of development by entrance/exit criteria ■ Pre-determined in test plan and/or project plan ■ Examples

Pre-Planning Activities Test Objectives ■ Objectives to be accomplished through testing

Pre-Planning Activities Test Objectives ■ Should restate the project objectives from the project plan ■ Should determine whether the projected objectives have been achieved ■ If not stated clearly in the project plan, test team must develop their own

Pre-Planning Activities Test Objectives Set objectives to minimize risks ■ Brainstorm to identify project objectives ■ 10 or fewer is a good number ■ Relate to the test policy ■

Pre-Planning Activities Test Objectives Itemize the objectives ■ Write in a measurable statement ■ Assign a priority ■ Define completion criteria ■

Pre-Planning Activities Assumptions ■ Establishing conditions that must exist for testing to be comprehensive and on schedule.

Pre-Planning Activities Entrance/Exit Criteria ■ Criteria that must be met prior to moving to the next level of testing or into production.

Pre-Planning Activities Entrance Criteria ■ Required conditions and standards for work product quality that must be present or met for entry into the next stage of the software development process

Pre-Planning Activities Exit Criteria ■ Standards for work product quality, which block the promotion of incomplete or defective work products to subsequent stages of the software development process

Pre-Planning Activities Entrance/Exit Criteria ■ Defined in Test Plan ■ Provides guidelines ■ Examples of different criteria

Test Planning ■

Test Plan

■

Staffing

■

Requirements and Traceability

■

Approach

■

Test Check Procedures

■

Estimating

■

Scheduling

Test Planning Test Plan ■ The deliverables to meet the test’s objectives ■ The activities to produce the deliverables ■ The schedule and resources to complete the activities

Test Planning Test Plan ■

Why create a test plan?

■

When do you start the test plan?

Test Planning Test Plan Documented tests should be:   

Repeatable Controllable Insure Adequate Coverage

Test Planning “The act of designing tests is one of the most effective error prevention mechanisms known…The thought process that must take place to create useful tests can discover and eliminate problems at every stage of development.” –

Boris Beizer

Test Planning Test Plan ■ IEEE standard defines a hierarchy of test documents – – – –

■

Test Plan Test Design Specification Test Case Specification Test Procedure Specification

View two sample plans

Test Planning Types of Tests ■ A key step in developing a test strategy is to determine the right tests to execute ■

Determined by the objectives of the test and the type of application

Test Planning ■ ■ ■ ■ ■ ■ ■ ■

Types of Tests ■ Vendor validation White Box (Glass Box) ■ Conversion Black Box ■ Stress/Load Functional ■ Performance Integration ■ Recovery Unit ■ Configuration System ■ Benefits realization Requirements-based (User Acceptance) Usability

Test Planning Requirements/Traceability ■ Defines the tests needed and relates those tests to the requirements

Test Planning Requirements ■ A description of a condition or capability of a system -- Also known as stakeholder needs. ■ Derived from user needs ■ Stated in a contract, standard, or specification

Test Planning Traceability ■ Ability to trace requirements is critical to ensure user needs have been delivered ■

Primary goal of software testing is to prove requirements are actually delivered in the final product

Test Planning Traceability Accomplished using any of the following:  Use cases  Activity diagrams  Class diagrams  Test cases  Scenarios

Test Planning Estimating ■ Determines the amount of resources required to accomplish the planned activities

Test Planning Scheduling ■ Establishes milestones for completing the testing effort

Test Planning Scheduling Four key objectives: ■ Provide predictability ■ Identify opportunities to improve or protect the project schedule ■ Be fair to the team ■ Maximize productivity

Test Planning Scheduling Tips to achieve goals ■ Measure performance and productivity ■ Identify and estimate every task ■ Identify tasks as fixed or recurring ■ Allow extra time

Test Planning Scheduling Tips to achieve goals (continued) ■ Recognize individuality of the team ■ Beware of late hiring in the schedule ■ Take deliberate shortcuts ■ Be careful of too many meetings

Test Planning Staffing ■ Selecting the size and competency of staff needed to achieve the test plan objectives ■

Also covered in CBOK Domain 7

Test Planning Staffing ■ What types of personnel should be considered for inclusion on the test team? ■ What attributes and skills make up a good tester? ■ What is a good ratio of testers to developers?

Test Planning Staffing Steps to form the best team: ■ Identify potential test team members ■ Recruit test team members and develop tentative test assignments ■ Define individual work assignments

Test Planning Approach ■ Methods, tools, and techniques used to accomplish test objectives ■

Covered in CBOK Domain 5 and 9

Test Planning Test Check Procedures ■ Test quality control ■ Set of procedures based on the test plan and test design ■ Incorporating test cases that ensure tests are performed correctly and completely

Test Planning Test Check Procedures Includes the following: – – –

■

Test coverage evaluation Product error evaluation Test effectiveness evaluation

Covered in CBOK Domain 16

Test Planning Test Check Procedures Test coverage evaluation ■ The process of assessing the thoroughness of the collective set of test cases for the product, and deciding whether or not to develop more tests.

Test Planning Test Check Procedures Product error evaluation ■ The process of assessing the quality of the product, with respect to the test execution, and deciding whether or not to develop more tests.

Test Planning Test Check Procedures Test effectiveness evaluation ■ The process of assessing the overall effectiveness of the current testing effort relative to the test completion criteria, and deciding whether to stop testing, or add more tests and continue.

Post Planning Activities ■

Change Management

■

Versioning

Post Planning Activities Change Management ■ Modifies and controls the plan in relationship to the actual progress and scope of the system development

Post Planning Activities Versioning ■ Methods to control, monitor, and achieve change Includes: – – –

Change Control Change Management Configuration Management

Post Planning Activities Versioning ■ The ability to maintain control over the changes made to ALL project artifacts is critical to the success of a project

Post Planning Activities Versioning The Configuration Management (CM) portion of the project facilitates: ■ Product baselines ■ Review ■ Change ■ Defect tracking

Post Planning Activities Versioning Other duties of CM: ■ Write the CM Plan ■ Report change request based progress statistics ■ Support product development activities ■ Allow all project artifacts to be available as required

Post Planning Activities Versioning  Once testing begins, the project must ensure that the correct versions of software components are being tested.  Migration and Back Out procedures must be developed

Planning Process -- Summary Consists of 3 parts: ■ Pre-Planning Activities ■ Test Planning ■ Post-Planning Activities ■

Testing covers the ‘C’ in the PDCA Continuous Process Cycle

Bibliography ■ ■

■

■

CSTE Study Guide, QAI, 2000 Kaner, Polk, and Nguyan. Testing Computer Software, Wiley Computer Publishing, 1999. Kit, Edward. Software Testing in the Real World, Addison Wesley, 1997. Perry, William E. Effective Methods for Software Testing, Wiley Computer Publishing, 2000.