Boundary Value Analysis

Functional Testing Part 1 Boundary Value Testing – Equivalence Class Testing

Agenda 

Boundary Value Testing 

Boundary Value Analysis  

   

Robustness Testing Worst Case Testing Special Value Testing Examples   





Generalizing Boundary Value Analysis Limitations of Boundary Value Analysis

Test Cases for the Triangle Problem Test Cases for the NextDate Problem Test Cases for the Commission Problem

Guidelines for Boundary value Testing

Equivalence Class Testing 

Equivalence Classes   

   

Weak Equivalence Class Testing Strong Equivalence Class Testing Traditional Equivalence Class Testing

Equivalence Class Test Cases for the Triangle Problem Equivalence Class Test Cases for the NextDate Function Equivalence Class Test Cases for the Commission Problem Guidelines and Observations

2

Agenda 

Boundary Value Testing 

Boundary Value Analysis  

   

Robustness Testing Worst Case Testing Special Value Testing Examples   





Generalizing Boundary Value Analysis Limitations of Boundary Value Analysis

Test Cases for the Triangle Problem Test Cases for the NextDate Problem Test Cases for the Commission Problem

Guidelines for Boundary value Testing

Equivalence Class Testing 

Equivalence Classes   

   

Weak Equivalence Class Testing Strong Equivalence Class Testing Traditional Equivalence Class Testing

Equivalence Class Test Cases for the Triangle Problem Equivalence Class Test Cases for the NextDate Function Equivalence Class Test Cases for the Commission Problem Guidelines and Observations

3

Boundary Value Analysis 

F(x1, x2), a ≤ x1 ≤ b, c ≤ x2 ≤ d



Boundary value analysis focuses on the boundary of the input space to identify test cases The rationale behind boundary value testing is that errors tend to occur near the extreme values of an input variable





e.g. loop conditions (< instead of ≤), counters 4

Boundary Value Analysis 





Basic idea: use input variable values at their minimum (min), just above the minimum (min+), a nominal value (nom), just below their maximum (max-), and at their maximum (max) “Single fault” assumption in reliability theory: failures are only rarely the result of the simultaneous occurrence of two (or more) faults. The boundary value analysis test cases are obtained by holding the values of all but one variable at their nominal values, and letting that variable assume its extreme values 5

Boundary Value Analysis F(x1, x2) x2

Input Space (domain)

d

d

c

c a

Test Cases (function of two variables)

x2

b

x1

a

<x1nom , x2min> <x1nom , x2min+> <x 1nom, x2nom > <x1nom , x2max-> <x1nom, x2max> <x1min, x2nom > <x1min+, x2nom > <x 1nom, x2nom > <x1max-, x2nom > <x1max, x2nom>

b

x1

6

Agenda 

Boundary Value Testing 

Boundary Value Analysis  

   

Robustness Testing Worst Case Testing Special Value Testing Examples   





Generalizing Boundary Value Analysis Limitations of Boundary Value Analysis

Test Cases for the Triangle Problem Test Cases for the NextDate Problem Test Cases for the Commission Problem

Guidelines for Boundary value Testing

Equivalence Class Testing 

Equivalence Classes   

   

Weak Equivalence Class Testing Strong Equivalence Class Testing Traditional Equivalence Class Testing

Equivalence Class Test Cases for the Triangle Problem Equivalence Class Test Cases for the NextDate Function Equivalence Class Test Cases for the Commission Problem Guidelines and Observations

7

Generalizing Boundary Value Analysis 

By the number of variables 



For n variables → 4n + 1 test cases

By the kinds of ranges, depends on the type (nature) of the variables 

Variables have discrete, bounded values 



Variables have no explicit bounds  



Create “artificial” bounds e.g. triangle problem

Boolean variables 



e.g. NextDate function, commission problem

Decision table-based testing

Logical variables (bound to a value or another logic variable) 

e.g. PIN and transaction type in SATM System

8

Agenda 

Boundary Value Testing 

Boundary Value Analysis  

   

Robustness Testing Worst Case Testing Special Value Testing Examples   





Generalizing Boundary Value Analysis Limitations of Boundary Value Analysis

Test Cases for the Triangle Problem Test Cases for the NextDate Problem Test Cases for the Commission Problem

Guidelines for Boundary value Testing

Equivalence Class Testing 

Equivalence Classes   

   

Weak Equivalence Class Testing Strong Equivalence Class Testing Traditional Equivalence Class Testing

Equivalence Class Test Cases for the Triangle Problem Equivalence Class Test Cases for the NextDate Function Equivalence Class Test Cases for the Commission Problem Guidelines and Observations

9

Limitations of Boundary Value Analysis 

Boundary value analysis works well when the program to be tested is a function of several independent variables that represent bounded physical quantities e.g. NextDate test cases are inadequate (little stress on February, dependencies among month, day, and year)  e.g. variables refer to physical quantities, such as temperature, air speed, load etc. 

10

Agenda 

Boundary Value Testing 

Boundary Value Analysis  

   

Robustness Testing Worst Case Testing Special Value Testing Examples   





Generalizing Boundary Value Analysis Limitations of Boundary Value Analysis

Test Cases for the Triangle Problem Test Cases for the NextDate Problem Test Cases for the Commission Problem

Guidelines for Boundary value Testing

Equivalence Class Testing 

Equivalence Classes   

   

Weak Equivalence Class Testing Strong Equivalence Class Testing Traditional Equivalence Class Testing

Equivalence Class Test Cases for the Triangle Problem Equivalence Class Test Cases for the NextDate Function Equivalence Class Test Cases for the Commission Problem Guidelines and Observations

11

Robustness Testing  



Simple extension of boundary value analysis In addition to the five boundary value analysis values of a variable, see what happens when the extrema are exceeded with a value slightly greater than the maximum (max+) and a value slightly less than the minimum (min-) Focuses on the expected outputs 



e.g. exceeding load capacity of a public elevator

Forces attention on exception handling 12

Robustness Testing Robustness Test Cases (function of two variables)

x2

<x1nom , x2min> <x1nom , x2min+> <x 1nom, x2nom > <x1nom , x2max-> <x1nom, x2max> <x1min, x2nom > <x1min+, x2nom > <x 1nom, x2nom > <x1max-, x2nom > <x1max, x2nom> <x 1min-, x2nom > <x1nom , x2max+> <x1max+, x2nom> <x 1nom, x2min- >

d

c a

b

x1

13

Agenda 

Boundary Value Testing 

Boundary Value Analysis  

   

Robustness Testing Worst Case Testing Special Value Testing Examples   





Generalizing Boundary Value Analysis Limitations of Boundary Value Analysis

Test Cases for the Triangle Problem Test Cases for the NextDate Problem Test Cases for the Commission Problem

Guidelines for Boundary value Testing

Equivalence Class Testing 

Equivalence Classes   

   

Weak Equivalence Class Testing Strong Equivalence Class Testing Traditional Equivalence Class Testing

Equivalence Class Test Cases for the Triangle Problem Equivalence Class Test Cases for the NextDate Function Equivalence Class Test Cases for the Commission Problem Guidelines and Observations

14

Worst Case Testing  

Worst case analysis: more than one variable has an extreme value Procedure:  

 

More thorough than boundary value analysis Represents more effort 

  

For each variable create the set <min, min+, nom, max-, max> Take the Cartesian product of these sets to generate test cases

For n variables → 5n test cases (as opposed to 4n+1 test cases for boundary value analysis)

Follows the generalization pattern Same limitations Robust worst case testing can be applied 15

Worst Case Testing Worst Case Test Cases (function of two variables)

x2

d

c a

b

x1

<x 1min, x2min> <x1min, x2min+> <x1min, x2nom > <x1min, x2max-> <x1min, x2max> <x1min+, x2min> <x1min+, x2min+> <x1min+, x2nom > <x1min+, x2max-> <x 1min+, x2max> <x1nom , x2min> <x1nom , x2min+> <x 1nom, x2nom > <x1nom , x2max-> <x1nom, x2max>

<x1max-, x2min> <x1max-, x2min+> <x1max-, x2nom > <x1max-, x2max-> <x 1max-, x2max> <x1max, x2min> <x 1max, x2min+> <x1max, x2nom> <x 1max, x2max-> <x1max, x2max>

16

Worst Case Testing Robust Worst Case Test Cases (function of two variables)

x2

d

c a

b

x1

<x 1min-, x2min- > <x1min- , x2min> <x1min- , x2min+> <x 1min-, x2nom > <x1min- , x2max-> <x1min-, x2max> <x1min- , x2max+> <x1min, x2min-> <x 1min, x2min> <x1min, x2min+> <x1min, x2nom > <x1min, x2max-> <x1min, x2max> <x 1min, x2max+>

<x1min+, x2min- > <x1min+, x2min> <x1min+, x2min+> <x1min+, x2nom > <x1min+, x2max-> <x 1min+, x2max> <x1min+, x2max+> <x 1nom, x2min- > <x1nom , x2min> <x1nom , x2min+> <x 1nom, x2nom > <x1nom , x2max-> <x1nom, x2max> <x1nom , x2max+>

<x1max-, x2min- > <x1max-, x2min> <x1max-, x2min+> <x1max-, x2nom > <x1max-, x2max-> <x 1max-, x2max> <x1max-, x2max+> <x1max, x2min-> <x1max, x2min> <x 1max, x2min+> <x1max, x2nom> <x 1max, x2max-> <x1max, x2max> <x1max, x2max+>

<x1max+, x2min-> <x 1max+, x2min > <x1max+, x2min+> <x1max+, x2nom> <x1max+, x2max-> <x1max+, x2max> <x 1max+, x2max+>

17

Agenda 

Boundary Value Testing 

Boundary Value Analysis  

   

Robustness Testing Worst Case Testing Special Value Testing Examples   





Generalizing Boundary Value Analysis Limitations of Boundary Value Analysis

Test Cases for the Triangle Problem Test Cases for the NextDate Problem Test Cases for the Commission Problem

Guidelines for Boundary value Testing

Equivalence Class Testing 

Equivalence Classes   

   

Weak Equivalence Class Testing Strong Equivalence Class Testing Traditional Equivalence Class Testing

Equivalence Class Test Cases for the Triangle Problem Equivalence Class Test Cases for the NextDate Function Equivalence Class Test Cases for the Commission Problem Guidelines and Observations

18

Special Value Testing   

 

The most widely practiced form of functional testing Most intuitive, least uniform, no guidelines The tester uses his/her domain knowledge, experience with similar programs, “ad hoc testing” It is dependent on the abilities of the tester Even though it is highly subjective, it often results in a set of test cases which is more effective in revealing faults than the test sets generated by the other methods 19

Agenda 

Boundary Value Testing 

Boundary Value Analysis  

   

Robustness Testing Worst Case Testing Special Value Testing Examples   





Generalizing Boundary Value Analysis Limitations of Boundary Value Analysis

Test Cases for the Triangle Problem Test Cases for the NextDate Problem Test Cases for the Commission Problem

Guidelines for Boundary value Testing

Equivalence Class Testing 

Equivalence Classes   

   

Weak Equivalence Class Testing Strong Equivalence Class Testing Traditional Equivalence Class Testing

Equivalence Class Test Cases for the Triangle Problem Equivalence Class Test Cases for the NextDate Function Equivalence Class Test Cases for the Commission Problem Guidelines and Observations

20

Test Cases for the Triangle Problem Boundary Value Analysis Test Cases

min = 1 min+ = 2 nom = 100 max- = 199 max = 200

Case 1 2 3 4 5 6 7 8 9 10 11 12 13

a 100 100 100 100 100 100 100 100 100 1 2 199 200

b 100 100 100 100 100 1 2 199 200 100 100 100 100

c 1 2 100 199 200 100 100 100 100 100 100 100 100

Expected Output Isosceles Isosceles Equilateral Isosceles Not a Triangle Isosceles Isosceles Isosceles Not a Triangle Isosceles Isosceles Isosceles Not a Triangle

21

Test Cases for the Triangle Problem Worst Case Test Cases (60 of 125) Case 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

a 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2

b 1 1 1 1 1 2 2 2 2 2 100 100 100 100 100 199 199 199 199 199 200 200 200 200 200 1 1 1 1 1

c 1 2 100 199 200 1 2 100 199 200 1 2 100 199 200 1 2 100 199 200 1 2 100 199 200 1 2 100 199 200

Expected Output Equilateral Not a Triangle Not a Triangle Not a Triangle Not a Triangle Not a Triangle Isosceles Not a Triangle Not a Triangle Not a Triangle Not a Triangle Not a Triangle Isosceles Not a Triangle Not a Triangle Not a Triangle Not a Triangle Not a Triangle Isosceles Not a Triangle Not a Triangle Not a Triangle Not a Triangle Not a Triangle Isosceles Not a Triangle Isosceles Not a Triangle Not a Triangle Not a Triangle

Case 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

a 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 100 100 100 100 100 100 100 100 100 100

b 2 2 2 2 2 100 100 100 100 100 199 199 199 199 199 200 200 200 200 200 1 1 1 1 1 2 2 2 2 2

c 1 2 100 199 200 1 2 100 199 200 1 2 100 199 200 1 2 100 199 200 1 2 100 199 200 1 2 100 199 200

Expected Output Isosceles Equilateral Not a Triangle Not a Triangle Not a Triangle Not a Triangle Not a Triangle Isosceles Not a Triangle Not a Triangle Not a Triangle Not a Triangle Not a Triangle Isosceles Scalene Not a Triangle Not a Triangle Not a Triangle Scalene Isosceles Not a Triangle Not a Triangle Isosceles Not a Triangle Not a Triangle Not a Triangle Not a Triangle Isosceles Not a Triangle Not a Triangle

22

Agenda 

Boundary Value Testing 

Boundary Value Analysis  

   

Robustness Testing Worst Case Testing Special Value Testing Examples   





Generalizing Boundary Value Analysis Limitations of Boundary Value Analysis

Test Cases for the Triangle Problem Test Cases for the NextDate Problem Test Cases for the Commission Problem

Guidelines for Boundary value Testing

Equivalence Class Testing 

Equivalence Classes   

   

Weak Equivalence Class Testing Strong Equivalence Class Testing Traditional Equivalence Class Testing

Equivalence Class Test Cases for the Triangle Problem Equivalence Class Test Cases for the NextDate Function Equivalence Class Test Cases for the Commission Problem Guidelines and Observations

23

Test Cases for the NextDate Problem Boundary Value Analysis Test Cases

month min = 1 min+ = 2 nom = 6 max- = 11 max = 12

day min = 1 min+ = 2 nom = 15 max- = 30 max = 31

year min = 1812 min+ = 1813 nom = 1912 max- = 2011 max = 2012

Case month day 1 6 15 2 6 15 3 6 15 4 6 15 5 6 15 6 6 1 7 6 2 8 6 30 9 6 31 10 1 15 11 2 15 12 11 15 13 12 15

year 1812 1813 1912 2011 2012 1912 1912 1912 1912 1912 1912 1912 1912

Expected Output June 16, 1812 June 16, 1813 June 16, 1912 June 16, 2011 June 16, 2012 June 2, 1912 June 3, 1912 July 1, 1912 error January 16, 1912 February 16, 1912 November 16, 1912 December 16, 1912

24

Test Cases for the NextDate Problem Worst Case Test Cases (60 of 125) Case 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

month 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2

day 1 1 1 1 1 2 2 2 2 2 15 15 15 15 15 30 30 30 30 30 31 31 31 31 31 1 1 1 1 1

year 1812 1813 1912 2011 2012 1812 1813 1912 2011 2012 1812 1813 1912 2011 2012 1812 1813 1912 2011 2012 1812 1813 1912 2011 2012 1812 1813 1912 2011 2012

Expected Output January 2, 1812 January 2, 1813 January 2, 1912 January 2, 2011 January 2, 2012 January 3, 1812 January 3, 1813 January 3, 1912 January 3, 2011 January 3, 2012 January 16, 1812 January 16, 1813 January 16, 1912 January 16, 2011 January 16, 2012 January 31, 1812 January 31, 1813 January 31, 1912 January 31, 2011 January 31, 2012 February 1, 1812 February 1, 1813 February 1, 1912 February 1, 2011 February 1, 2012 February 2, 1812 February 2, 1813 February 2, 1912 February 2, 2011 February 2, 2012

Case 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

month 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 6 6 6 6 6 6 6 6 6 6

day 2 2 2 2 2 15 15 15 15 15 30 30 30 30 30 31 31 31 31 31 1 1 1 1 1 2 2 2 2 2

year 1812 1813 1912 2011 2012 1812 1813 1912 2011 2012 1812 1813 1912 2011 2012 1812 1813 1912 2011 2012 1812 1813 1912 2011 2012 1812 1813 1912 2011 2012

Expected Output February 3, 1812 February 3, 1813 February 3, 1912 February 3, 2011 February 3, 2012 February 16, 1812 February 16, 1813 February 16, 1912 February 16, 2011 February 16, 2012 error error error error error error error error error error June 2, 1812 June 2, 1813 June 2, 1912 June 2, 2011 June 2, 2012 June 3, 1812 June 3, 1813 June 3, 1912 June 3, 2011 June 3, 2012

25

Agenda 

Boundary Value Testing 

Boundary Value Analysis  

   

Robustness Testing Worst Case Testing Special Value Testing Examples   





Generalizing Boundary Value Analysis Limitations of Boundary Value Analysis

Test Cases for the Triangle Problem Test Cases for the NextDate Problem Test Cases for the Commission Problem

Guidelines for Boundary value Testing

Equivalence Class Testing 

Equivalence Classes   

   

Weak Equivalence Class Testing Strong Equivalence Class Testing Traditional Equivalence Class Testing

Equivalence Class Test Cases for the Triangle Problem Equivalence Class Test Cases for the NextDate Function Equivalence Class Test Cases for the Commission Problem Guidelines and Observations

26

Test Cases for the Commission Problem    



13 Boundary Value Analysis Test Cases 125 Worst Case Test Cases Boundary values for the output range, especially near the threshold points of $1000 and $1800 Part of the reason for using the output range to determine test cases is that test cases from the input range are almost all in the 20% zone We want to find input variable combinations that stress the boundary values: $100, $1000, $1800, and $7800 27

Test Cases for the Commission Problem Output Boundary Value Analysis Test Cases Case 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

locks 1 1 1 2 5 10 10 9 10 10 10 11 14 18 18 17 18 18 18 19 48 70 70 69 70

stocks 1 1 2 1 5 10 9 10 10 10 11 10 14 18 17 18 18 18 19 18 48 80 79 80 80

barrels 1 2 1 1 5 9 10 10 10 11 10 10 14 17 18 18 18 19 18 18 48 89 90 90 90

sales 100 125 130 145 500 975 970 955 1000 1025 1030 1045 1400 1775 1770 1755 1800 1825 1830 1845 4800 7775 7770 7755 7800

commission 10 12.5 13 14.5 50 97.5 97 95.5 100 103.75 104.5 106.75 160 216.25 215.5 213.25 220 225 226 229 820 1415 1414 1411 1420

comment output minimum output minimum + output minimum + output minimum + midpoint border point border point border point border point border point + border point + border point + midpoint border point border point border point border point border point + border point + border point + midpoint output maximum output maximum output maximum output maximum

28

Test Cases for the Commission Problem  



Test case 9 is the $1000 border point If we tweak the input variables we get values just below and just above the border Form of special value testing based on mathematical insight Output Special Value Test Cases Case 1 2 3

locks 10 18 18

stocks 11 17 19

barrels 9 19 17

sales 1005 1795 1805

commission 100.75 219.25 221

comment border point + border point border point +

29

Agenda 

Boundary Value Testing 

Boundary Value Analysis  

   

Robustness Testing Worst Case Testing Special Value Testing Examples   





Generalizing Boundary Value Analysis Limitations of Boundary Value Analysis

Test Cases for the Triangle Problem Test Cases for the NextDate Problem Test Cases for the Commission Problem

Guidelines for Boundary value Testing

Equivalence Class Testing 

Equivalence Classes   

   

Weak Equivalence Class Testing Strong Equivalence Class Testing Traditional Equivalence Class Testing

Equivalence Class Test Cases for the Triangle Problem Equivalence Class Test Cases for the NextDate Function Equivalence Class Test Cases for the Commission Problem Guidelines and Observations

30

Equivalence Classes 

Motivations  



  

Have a sense of complete testing Avoid redundancy

Equivalence classes form a partition of a set, where partition refers to a collection of mutually disjoint subsets whose union is the entire set (completeness, non-redundancy) The idea is to identify test cases by using one element from each equivalence class “treated the same” → “traversing the same execution path” The key is the choice of the equivalence relation that determines the classes 

“second guess” the likely implementation , and think about the functional manipulations that must somehow be present in the implementation

31

Equivalence Classes 

Program under consideration: Function of variables a, b, c  Input domain consists of sets A, B, and C  A = A1 A2 A3  B = B1 B2 B3 B4  C = C1 C2  a1 ∈ A1, b3 ∈ B3, c2 ∈ C2 

32

Agenda 

Boundary Value Testing 

Boundary Value Analysis  

   

Robustness Testing Worst Case Testing Special Value Testing Examples   





Generalizing Boundary Value Analysis Limitations of Boundary Value Analysis

Test Cases for the Triangle Problem Test Cases for the NextDate Problem Test Cases for the Commission Problem

Guidelines for Boundary value Testing

Equivalence Class Testing 

Equivalence Classes   

   

Weak Equivalence Class Testing Strong Equivalence Class Testing Traditional Equivalence Class Testing

Equivalence Class Test Cases for the Triangle Problem Equivalence Class Test Cases for the NextDate Function Equivalence Class Test Cases for the Commission Problem Guidelines and Observations

33

Weak Equivalence Class Testing 

Accomplished by using one variable from each equivalence class in a test case

A = A1 A2 A3 B = B1 B2 B3 B4 C = C1 C2 a1 ∈ A1, b3 ∈ B3, c2 ∈ C2



Test Case WE1 WE2 WE3 WE4

a a1 a2 a3 a1

b b1 b2 b3 b4

c c1 c2 c1 c2

Number of weak equivalence class test cases = number of classes in the partition with the largest number of subsets 34

Agenda 

Boundary Value Testing 

Boundary Value Analysis  

   

Robustness Testing Worst Case Testing Special Value Testing Examples   





Generalizing Boundary Value Analysis Limitations of Boundary Value Analysis

Test Cases for the Triangle Problem Test Cases for the NextDate Problem Test Cases for the Commission Problem

Guidelines for Boundary value Testing

Equivalence Class Testing 

Equivalence Classes   

   

Weak Equivalence Class Testing Strong Equivalence Class Testing Traditional Equivalence Class Testing

Equivalence Class Test Cases for the Triangle Problem Equivalence Class Test Cases for the NextDate Function Equivalence Class Test Cases for the Commission Problem Guidelines and Observations

35

Strong Equivalence Class Testing 

 

Based on the Cartesian Product of the partition subsets A x B x C = 3 x 4 x 2 = 24 elements Equivalence relations can be defined on the output range of the program function being tested

Test Case SE1 SE2 SE3 SE4 SE5 SE6 SE7 SE8 SE9 SE10 SE11 SE12 SE13 SE14 SE15 SE16 SE17 SE18 SE19 SE20 SE21 SE22 SE23 SE24

a a1 a1 a1 a1 a1 a1 a1 a1 a2 a2 a2 a2 a2 a2 a2 a2 a3 a3 a3 a3 a3 a3 a3 a3

b b1 b1 b2 b2 b3 b3 b4 b4 b1 b1 b2 b2 b3 b3 b4 b4 b1 b1 b2 b2 b3 b3 b4 b4

c c1 c2 c1 c2 c1 c2 c1 c2 c1 c2 c1 c2 c1 c2 c1 c2 c1 c2 c1 c2 c1 c2 c1 c2 36

Agenda 

Boundary Value Testing 

Boundary Value Analysis  

   

Robustness Testing Worst Case Testing Special Value Testing Examples   





Generalizing Boundary Value Analysis Limitations of Boundary Value Analysis

Test Cases for the Triangle Problem Test Cases for the NextDate Problem Test Cases for the Commission Problem

Guidelines for Boundary value Testing

Equivalence Class Testing 

Equivalence Classes   

   

Weak Equivalence Class Testing Strong Equivalence Class Testing Traditional Equivalence Class Testing

Equivalence Class Test Cases for the Triangle Problem Equivalence Class Test Cases for the NextDate Function Equivalence Class Test Cases for the Commission Problem Guidelines and Observations

37

Traditional Equivalence Class Testing  

Defines equivalence classes in terms of validity Commission problem  

 

Valid inputs: 1 ≤ lock ≤ 70, 1 ≤ stock ≤ 80, 1 ≤ barrel ≤ 90 Invalid inputs: lock < 1, lock > 70, stock < 1, stock > 80, barrel < 1, barrel > 90

For valid inputs, use one value from each valid class (like weak equivalence testing) For invalid inputs, a test case will have one invalid value and the remaining values will all be valid (single failure) 38

Traditional Equivalence Class Testing 

Problems: Very often, the specification does not define what the expected output for an invalid test case should be → a lot of time spent in defining expected outputs  Strongly typed languages eliminate the need for the consideration of invalid inputs 

39

Agenda 

Boundary Value Testing 

Boundary Value Analysis  

   

Robustness Testing Worst Case Testing Special Value Testing Examples   





Generalizing Boundary Value Analysis Limitations of Boundary Value Analysis

Test Cases for the Triangle Problem Test Cases for the NextDate Problem Test Cases for the Commission Problem

Guidelines for Boundary value Testing

Equivalence Class Testing 

Equivalence Classes   

   

Weak Equivalence Class Testing Strong Equivalence Class Testing Traditional Equivalence Class Testing

Equivalence Class Test Cases for the Triangle Problem Equivalence Class Test Cases for the NextDate Function Equivalence Class Test Cases for the Commission Problem Guidelines and Observations

40

Equivalence Class Test Cases for the Triangle Problem  

Outputs: Not a Triangle, Scalene, Isosceles, Equilateral Easier to identify output (range) equivalence classes    

R1 = { : the triangle with sides a, b, and c is equilateral} R2 = { : the triangle with sides a, b, and c is isosceles} R3 = { : the triangle with sides a, b, and c is scalene} R4 = { : sides a, b, and c do not form a triangle} Test Case OE1 OE2 OE3 OE4

a 5 2 3 4

b 5 2 4 1

c 5 3 5 2

Expected Output Equilateral Isosceles Scalene Not a Triangle

41

Equivalence Class Test Cases for the Triangle Problem 

Input (domain) equivalence classes D1 = { : a = b = c} D2 = { : a = b, a ≠ c} D3 = { : a = c, a ≠ b} D4 = { : b = c, a ≠ b} D5 = { : a ≠ b, a ≠ c, b ≠ c} // D6 = { : a ≥ b + c} D6’ = { : a = b + c} D6’’ = { : a > b + c} // D7 = { : b ≥ a + c} D7’ = { : b = a + c} D7’’ = { : b > a + c} // D8 = { : c ≥ a + b} D8’ = { : c = a + b} D8’’ = { : c > a + b} 42

Agenda 

Boundary Value Testing 

Boundary Value Analysis  

   

Robustness Testing Worst Case Testing Special Value Testing Examples   





Generalizing Boundary Value Analysis Limitations of Boundary Value Analysis

Test Cases for the Triangle Problem Test Cases for the NextDate Problem Test Cases for the Commission Problem

Guidelines for Boundary value Testing

Equivalence Class Testing 

Equivalence Classes   

   

Weak Equivalence Class Testing Strong Equivalence Class Testing Traditional Equivalence Class Testing

Equivalence Class Test Cases for the Triangle Problem Equivalence Class Test Cases for the NextDate Function Equivalence Class Test Cases for the Commission Problem Guidelines and Observations

43

Equivalence Class Test Cases for the NextDate Function 

Input variables   



1 ≤ month ≤ 12 1 ≤ day ≤ 31 1812 ≤ year ≤ 2012

Traditional approach 

Valid equivalence classes   



M1 = { month : 1 ≤ month ≤ 12 } D1 = { day : 1 ≤ day ≤ 31 } Y1 = { year : 1812 ≤ year ≤ 2012 }

Invalid equivalence classes      

M2 = { month : month < 1 } M3 = { month : month > 12 } D2 = { day : day < 1 } D3 = { day : day >31 } Y2 = { year : year < 1812 } Y3 = {year : year > 2012 }

Case ID TE1 TE2 TE3 TE4 TE5 TE6 TE7

Month 6 -1 13 6 6 6 6

Day 15 15 15 -1 32 15 15

Year 1912 1912 1912 1912 1912 1811 2013

Expected Output 6/16/1912 invalid input invalid input invalid input invalid input invalid input invalid input

44

Equivalence Class Test Cases for the NextDate Function  

Traditional approach is deficient because it “treats” the elements of a class at the valid/invalid level Different approach: What must be done to an input date? M1 = { month: month has 30 days } M2 = { month: month has 31 days } M3 = { month: month is February } D1 = { day: 1 ≤ day ≤ 28 } D2 = { day: day = 29 } D3 = { day: day = 30 } D4 = { day: day = 31 } Y1 = { year: year = 1900 } Y2 = { year: 1812 ≤ year ≤ 2012 AND (year ≠ 1900) AND (year mod 4 = 0) } Y3 = { year: 1812 ≤ year ≤ 2012 AND (year mod 4 ≠ 0) } Not a perfect set of equivalence classes!!! 45

Equivalence Class Test Cases for the NextDate Function Weak equivalence class test cases Case ID WE1 WE2 WE3 WE4

Month 6 7 2 6

Day 14 29 30 31

Year 1900 1912 1913 1900

Expected Output 6/15/1900 7/30/1912 invalid input invalid input

Strong equivalence class test cases Case ID SE1 SE2 SE3 SE4 SE5 SE6 SE7 SE8 SE9 SE10 SE11 SE12 SE13 SE14 SE15 SE16 SE17 SE18

Month 6 6 6 6 6 6 6 6 6 6 6 6 7 7 7 7 7 7

Day 14 14 14 29 29 29 30 30 30 31 31 31 14 14 14 29 29 29

Year 1900 1912 1913 1900 1912 1913 1900 1912 1913 1900 1912 1913 1900 1912 1913 1900 1912 1913

Expected Output 6/15/1900 6/15/1912 6/15/1913 6/30/1900 6/30/1912 6/30/1913 07/01/1900 07/01/1912 07/01/1913 ERROR ERROR ERROR 7/15/1900 7/15/1912 7/15/1913 7/30/1900 7/30/1912 7/30/1913

Case ID SE19 SE20 SE21 SE22 SE23 SE24 SE25 SE26 SE27 SE28 SE29 SE30 SE31 SE32 SE33 SE34 SE35 SE36

Month 7 7 7 7 7 7 2 2 2 2 2 2 2 2 2 2 2 2

Day 30 30 30 31 31 31 14 14 14 29 29 29 30 30 30 31 31 31

Year 1900 1912 1913 1900 1912 1913 1900 1912 1913 1900 1912 1913 1900 1912 1913 1900 1912 1913

Expected Output 7/31/1900 7/31/1912 7/31/1913 08/01/1900 08/01/1912 08/01/1913 2/15/1900 2/15/1912 2/15/1913 ERROR 03/01/1912 ERROR ERROR ERROR ERROR ERROR ERROR ERROR

46

Agenda 

Boundary Value Testing 

Boundary Value Analysis  

   

Robustness Testing Worst Case Testing Special Value Testing Examples   





Generalizing Boundary Value Analysis Limitations of Boundary Value Analysis

Test Cases for the Triangle Problem Test Cases for the NextDate Problem Test Cases for the Commission Problem

Guidelines for Boundary value Testing

Equivalence Class Testing 

Equivalence Classes   

   

Weak Equivalence Class Testing Strong Equivalence Class Testing Traditional Equivalence Class Testing

Equivalence Class Test Cases for the Triangle Problem Equivalence Class Test Cases for the NextDate Function Equivalence Class Test Cases for the Commission Problem Guidelines and Observations

47

Equivalence Class Test Cases for the Commission Problem 

Input Domain Equivalence Classes 

Lock   



Stock   



L1 = { lock: 1 ≤ lock ≤ 70 } L2 = { lock: lock < 1 } L3 = { lock: lock > 70 } S1 = { stock: 1 ≤ stock ≤ 80 } S2 = { stock: stock < 1 } S3 = { stock: stock > 80 }

Barrel   

B1 = { barrel: 1 ≤ barrel ≤ 90 } B2 = { barrel: barrel < 1 } B3 = { barrel: barrel > 90 }

48

Equivalence Class Test Cases for the Commission Problem Strong Input Domain Equivalence Class Test Cases

Weak Input Domain Equivalence Class Test Cases Test Case WE1 WE2 WE3

locks 35 0 71

stocks 40 0 81

barrels sales commission 45 3900 640 0 ERROR ERROR 91 ERROR ERROR

Test Case SE1 SE2 SE3 SE4 SE5 SE6 SE7 SE8 SE9 SE10 SE11 SE12 SE13 SE14 SE15 SE16 SE17 SE18 SE19 SE20 SE21 SE22 SE23 SE24 SE25 SE26 SE27

locks 35 35 35 35 35 35 35 35 35 0 0 0 0 0 0 0 0 0 71 71 71 71 71 71 71 71 71

stocks 40 40 40 0 0 0 81 81 81 40 40 40 0 0 0 81 81 81 40 40 40 0 0 0 81 81 81

barrels 45 0 91 45 0 91 45 0 91 45 0 91 45 0 91 45 0 91 45 0 91 45 0 91 45 0 91

sales commission 3900 640 ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR

49

Equivalence Class Test Cases for the Commission Problem   



sales = 45 x locks + 30 x stocks + 25 x barrels L1 = { : sales ≤ 1000 } L2 = { : 1000 < sales ≤ 1800 } L3 = { : sales > 1800 } Output Range Equivalence Class Test Cases

Test Case OR1 OR2 OR3

locks 5 15 25

stocks 5 15 25

barrels 5 15 25

sales 500 1500 2500

commission 50 175 360

50

Agenda 

Boundary Value Testing 

Boundary Value Analysis  

   

Robustness Testing Worst Case Testing Special Value Testing Examples   





Generalizing Boundary Value Analysis Limitations of Boundary Value Analysis

Test Cases for the Triangle Problem Test Cases for the NextDate Problem Test Cases for the Commission Problem

Guidelines for Boundary value Testing

Equivalence Class Testing 

Equivalence Classes   

   

Weak Equivalence Class Testing Strong Equivalence Class Testing Traditional Equivalence Class Testing

Equivalence Class Test Cases for the Triangle Problem Equivalence Class Test Cases for the NextDate Function Equivalence Class Test Cases for the Commission Problem Guidelines and Observations

51

Guidelines and Observations 1.

2.

3.

4.

The traditional form of equivalence class testing is generally not as thorough as weak equivalence class testing, which in turn, is not as thorough as the strong form of equivalence class testing The only time it makes sense to use the traditional approach is when the implementation language is not strongly typed If error conditions are a high priority, we could extend strong equivalence class testing to include invalid classes (e.g. commission problem) Equivalence class testing is appropriate when input data is defined in terms of ranges and sets of discrete values. This is certainly the case when system malfunctions can occur for out-of-limit variable values 52

Guidelines and Observations 1.

2.

3.

Equivalence class testing is strengthened by a hybrid approach with boundary value testing. (We can “reuse” the effort made in defining the equivalence classes) (e.g. NextDate function) Equivalence class testing is indicated when the program function is complex. In such cases, the complexity of the function can help identify useful equivalence classes, as in the NextDate function Strong equivalence class testing makes a presumption that the variables are independent when the Cartesian Product is taken. If there are any dependencies, these will often generate “error” test cases, as they did in the NextDate function 53

Guidelines and Observations 1.

Several tries may be needed before “the right” equivalence relation is discovered, as we saw in the NextDate example. In other cases, there is an “obvious” or “natural” equivalence partition. When in doubt, the best bet is to try to second guess aspects of any reasonable implementation 54

Agenda 

Boundary Value Testing 

Boundary Value Analysis  

   

Robustness Testing Worst Case Testing Special Value Testing Examples   





Generalizing Boundary Value Analysis Limitations of Boundary Value Analysis

Test Cases for the Triangle Problem Test Cases for the NextDate Problem Test Cases for the Commission Problem

Guidelines for Boundary value Testing

Equivalence Class Testing 

Equivalence Classes   

   

Weak Equivalence Class Testing Strong Equivalence Class Testing Traditional Equivalence Class Testing

Equivalence Class Test Cases for the Triangle Problem Equivalence Class Test Cases for the NextDate Function Equivalence Class Test Cases for the Commission Problem Guidelines and Observations

55

References 

Software Testing A Craftsman's Approach 2nd edition, Paul C. Jorgensen, CRC Press (Chapters 5 and 6)

56