Effective Methods Of Software Testing

  • October 2019
  • PDF

This document was uploaded by user and they confirmed that they have the permission to share it. If you are author or own the copyright of this book, please report to us by using this DMCA report form. Report DMCA


Overview

Download & View Effective Methods Of Software Testing as PDF for free.

More details

  • Words: 1,986
  • Pages: 70
Quality Assurance Institute’s (QAI) Effective Methods of Software Testing A Perspective Seattle Area Software Quality Assurance Group John Napier, CSQE, CSTE BEST Consulting 10/14/08

Purpose   

Value of Professional Associations Value of Software Certification programs QAI’s “Effective Methods of Software Testing”

Disclaimers 

The opinions expressed in this presentation are my own, and not those of: ◆ ◆ ◆ ◆

BEST Consulting The Boeing Company Quality Assurance Institute SASQAG

What its like in the trenches...     

No common language “In my last job…” No individual professional context No organizational professional context Can’t see the forest...

What’s a person to do?    

Establish a personal professional context Get to know “the forest” Help create a common language Help bring best practices into your organization

The Quality Assurance Institute What’s QAI?

10/14/08

The Quality Assurance Institute 

 

An international association of 1000 organizations banded together to share best practices, benchmark data and experiences Software quality focus Certification Programs ◆ Certified Quality Analyst ◆ Certified Software Quality Engineer

QAI Certified Software Test Engineer (CSTE) 

 Optional Skills Mandatory Skills ◆ Test Planning ◆ Risk Analysis ◆ Use of Test Tools ◆ Measurement ◆ Test Performance ◆ Test Case Design ◆ Defect Management ◆ Establishing and Maintaining a Test Environment

ASQC* Certified Quality Software Engineer (CQSE)   



Experience Professional Association Software Engineering Body of Knowledge ◆ Exam Many other ASQC Certification Programs

*American Society for Quality Control

Effective Methods of Software Testing Carolyn Fairbanks Q-Star Consultants 10-14 March 1997 Seattle 10/14/08

Effective Methods of Software Testing There are really only two issues in software testing:  

Requirements Organizational Culture 10/14/08

QAI Test Materials Organization  



Test Strategy Test Plan Supporting Materials ◆ Metrics ◆ Test Toolbox - Techniques ◆ Test Conditions - Techniques ◆ Test Case Design - Techniques

Test Strategy Definition      

1. Type of Project 2. Type of Software 3. When Testing will Occur 4. Critical Success Factors 5. Who will Conduct the Testing 6. Tradeoffs

1. Types of Project     

Traditional Client/Server Prototyping Object Oriented Purchased / Contracted

2. Types of Software Batch Process Control Graphic User Interface Object Oriented Procedure Control Expert System Multi-Media Advanced Math Models

Diagnostic Software Sensor and Signal Simulation DB Management Data Presentation Decision Analysis Pattern & Image Computer Operations Software Development Tools

3. When will Testing Occur?      

Requirements Phase Activities Design Phase Activities Build Phase Activities Test Phase Activities Installation Phase Activities Maintenance Phase Activities

4. Critical Success Factors Correctness Authorization Data Integrity Audit Trail Continuity of Processing Service Levels Access Control

Methodology Reliability Ease of Use Maintainable Portable Coupling Performance Ease of Operations

Testing Concerns Matrix Critical Success Factors

Correctness

Life Cycle Stages Requirements

Design

Complete, Clear, Correct,

Etc

Testable, Feasible

7. Tradeoffs (What can be given up?)     

Schedule Cost / Resources Scope Technology Quality / Critical Success Factors

Some QAI Philosophy Test Life Cycle, Test Workbench

10/14/08

Testing Relationship Definitions Testing Lifecycle Concept Acceptance Test

Operational or Business Need

System Test

Define Requirements

Integration Test

Design System Build System

Unit Test

Testing Relationship Definitions Continuous Life Cycle Testing Validate

Verify

Acceptance Test

Operational or Business Need

Validate

Verify

System Test

Define Requirements

Validate

Verify

Integration Test

Design System

Verify Build System

Validate Unit Test

Testing Relationship Definitions Test Support Activities TEST OBJECTIVES Validate

Verify

Acceptance Test

Operational or Business Need

TEST PLANNING

Validate

Verify

System Test

Define Requirements

Validate

Verify

Integration Test

Design System

TEST EXECUTION (STATIC)

Verify

Build System

Validate

Unit Test

TEST EXECUTION (DYNAMIC) & TEST REPORTING

QAI Definitions “Test Workbench” Concept Problems Products for Test

Test Processes

Test Quality Control

Test Toolbox

Test Measures

Validated Test Results

The QAI Test Process 1. 2. 3. 4.

Set Test Objectives Develop Test Plan Execute Tests Summarize and Report Results

Step 1 - Set Test Objectives    

Task 1 - Organize Test Team Task 2 - Perform Risk Assessment Task 3 - Set Test Objectives Task 4 - Quality Control

Set Test Objectives

Task 1 - Organize Test Team  

Define Test Responsibilities / Tasks Determine People to Perform Tasks

Worksheets 4,5

Set Test Objectives

2. Perform Risk Assessment 

Dimensions of Risk ◆ Criticality / Mission Impact ◆ System Size, Scale and Complexity ◆ System Environment and Stability ◆ Reliability and Integrity ◆ Technology Integration Simple or Complex Scoring Approach

Impact of Failure Likelihood of Failure

Set Test Objectives

3. Set Test Objectives 



Using: Prioritize Each◆ System Objectives Low ◆ Test Strategy Medium High ◆ Risk Score Analysis ◆ Team Assignments Worksheet 6 Set Objectives ◆ Brainstorming ◆ Relate Objectives to Function / Output ◆ Relate Objectives to Events / System Input

Set Test Objectives

Quality Control  

Objectives QC Checklist Objectives Incident Report (Change Request)

…lots of checklists!

Develop Test Plan         

1. Define Business Processes 2. Define Structural Attributes 3. Define Tests 4. Create Test / Function Matrix 5. Define Admin Requirements 6. Formulate System Test Plan 7. QC System Test Plan 8. Develop Unit Test Plan 9. QC Unit Test Plan

Develop Test Plan

1. Define Business Functions 

Describe what the system should do ◆ …stands alone as an activity ◆ …is controllable ◆ …can be a transition ◆ …can be described by a process

Business Function Matrix Function

Description

Evaluation Criteria

Worksheet

8

Develop Test Plan

2. Define Structural Attributes     

Reliability Efficiency Integrity Usability Maintainability

    

Testability Flexibility Portability Reusability Interoperability

Worksheet 9

Structural Attribute Matrix Attribute

Priority

Description

Evaluation Criteria

Develop Test Plan

3. Define Tests 



Using ◆ Test Objectives Worksheet ◆ Business Functions Worksheet ◆ Structural Attributes Worksheet Define Tests ◆ Associate test factors, techniques and test types ◆ Associate evaluation criteria with techniques and types of tests ◆ Determine type of test which satisfies each objective

Type of Test Techniques 

Functional ◆ Requirements ◆ Regression ◆ Error Handling ◆ Manual Support ◆ Intersystems ◆ Control ◆ Parallel



Structural ◆ Stress ◆ Execution ◆ Recovery ◆ Operations ◆ Compliance (to process) ◆ Security

Test Data Sources      

Test Data / Test Desks Production Data Simulation / Prototyping Review / Confirmation Instrumentation / Tracing Test Script

Test Definition 

Each Test is defined in terms of: ◆ Controls ◆ Inputs ◆ Outputs ◆ Procedures ...At a high level

Develop Test Plan

4. Create Function / Test Matrix Function

Tests

Test 1 Test 2

Function #1

25

17,30

Function #2

5,7

45

Ect.

Develop Test Plan

5. Define Admin Test Requirements       

Schedule Budget Resources Testing Materials Test Training Tests to be Conducted Defect Estimation Worksheets 13,14

Develop Test Plan

6. Formalize Test Plan    

Section I - General Information Section 2 - Plan Section 3 - Specifications and Evaluation Section 4 - Test Description

See Test Plan Example

Develop Test Plan

7. Test Plan Quality Control  

Quality Control Checklist Test Plan Incident Report (Change Request)

…even more checklists!

Develop Test Plan

8. Develop Unit Test Plan 



Using: ◆ System Test Plan ◆ Program Initiation Document ◆ Program Design / Specification Documents Develop ◆ Test Plan Administration ◆ Subfunction / Test Matrix ◆ Interface / Test Matrix Worksheets 20,21,22

Develop Test Plan

9. Unit Test Plan Quality Control 

Checklists

Step 3 - Execute Tests    

Task 1 - Select Test Tools Task 2 - Develop Test Cases Task 3 - Execute Tests Task 4 - Quality Control

QAI Tool Definition 

What’s a Test Tool? ◆ …any vehicle which assists in performing a test. ◆ Examples Decision

Tables  Automated capture / playback  Checklists  Program Analyzer

Execute Tests

Step 1 - Select Test Tools 

Using: ◆ System Test Plan ◆ Unit Test Plan ◆ Tool Selection Methodology

Execute Tests

Step 2 - Develop Test Cases Test Condition

Expected Result

Test Procedures

Worksheet 23

Execute Tests

Step 3 - Execute Tests  

Test Activities Log Regression Testing ◆ Full ◆ Partial

Worksheet 24

Step 4 - Summarize and Report Test Results     

Task 1 - Record Defects Task 2 - Perform Data Reduction Task 3 - Develop Findings and Recommendations Task 4 - Finalize Test Report Task 5 - Test Report Quality Control ◆ Test Defect Report ◆ Test Defect List ◆ Test Defect Log Worksheets 25,26,27,28

Supporting Material    

Metrics Toolbox Techniques Test Condition Analysis Test Design Analysis 10/14/08

Test Metrics Method Goal - Question - Metric

QAI Recommend Test Metrics    



Defect Reports Requirements Implemented Backlog of Defects (Not Corrected) Status of ◆ Project, Tests, Schedule, Budget Selected Test Metrics

QAI Test Metrics Selection Types 1 4 3 20 2

General Subject Involvement in Testing Extent of Testing Resources Consumed in Testing Effectiveness of Testing Assessment of Testing

The QAI Test Toolbox          

Data Modeling Boundary Value Analysis Capture/Playback Cause-Effect Graphing Change Control Trackers Checklists Checkpoint Review Code Comparison Compiler-based Analysis Complexity-based Analysis

         

Compliance Checkers Control Flow Analysis Correctness Proofs Coverage Based Analysis Data Dictionary Decision Tables Defect Trackers Desk Checking Equivalence Partitioning Error Guessing

The QAI Test Toolbox          

File Comparison Flowcharts Help Desk Trackers Inspections Instrumentation Integrated Test Facility Mapping Model Animation Model Balancing Network Analyzers

         

Performance / Timing Analyzers Population Analysis Risk Matrix Scoring Snapshots Symbolic Execution System Logs Test Case Generator Test Data Generator Test Scripts

The QAI Test Toolbox    

Test Stubs and Harnesses Test Suite Managers Tracing Walkthroughs

The QAI Test Toolbox Test Tools

Types of Functional and Structural Tests X

Boundary Analysis Population Analysis Checklists

X X

X

X

Test Condition Analysis 

Sources ◆ 1. Specification Decomposition ◆ 2. Population Analysis ◆ 3. Business Process Analysis ◆ 4. Structural Analysis ◆ 5. Test Transactions

Test Condition - Expected Result - Test Process

Identify Test Conditions  

1. System Specifications or Production Environment 2. Population Analysis ◆ Files and Tables ◆ Screens ◆ Field/Data Element

Test Condition - Expected Result - Test Process Worksheet 4

Identify Test Conditions 3. Test Transaction Types       

Field Record File Relationships Error Use (Outputs) Search

      

Match / Merge Stress Control Attributes States Procedures Control Flow

Test Condition - Expected Result - Test Process

Identify Test Conditions 



4. Business Cases ◆ …business scenarios… 5. Structural Analysis ◆ …decision points...

Test Condition - Expected Result - Test Process

Design Test Cases  

Design Test Cases Develop Scripts / Sets

Expand Test Conditions

Design Test Cases 

Expand Conditions into Test Cases using ◆ Boundary Value Analysis ◆ Equivalence Partitioning ◆ Decision Tables ◆ Code Coverage  Statement

Coverage  Condition Coverage Expand Test Conditions

Design Test Cases Test Scripts Actor/Scene …Sam scowls at the report...

Dialog “Where did this come from?”

Detailed Test Script Descriptions Script Reuse Expand Test Conditions

QAI Summary - Ten Key Test Concepts       

Use strategic test questions to set direction Develop Test Objectives Use a cross-functional “test team” Prepare a test plan, then execute it Utilize both verification and validation methods Collect and analyze defects Prepare test management reports

QAI Summary - Ten Key Test Concepts  



Make test efforts reusable Feed the continuous improvement process with information from testing Continuously improve the process

Incorporating the Process Models within Models …Make them your own

10/14/08

Questio ns?

Related Documents