Functional Testing Part 1 Boundary Value Testing – Equivalence Class Testing
Agenda
Boundary Value Testing
Boundary Value Analysis
Robustness Testing Worst Case Testing Special Value Testing Examples
Generalizing Boundary Value Analysis Limitations of Boundary Value Analysis
Test Cases for the Triangle Problem Test Cases for the NextDate Problem Test Cases for the Commission Problem
Guidelines for Boundary value Testing
Equivalence Class Testing
Equivalence Classes
Weak Equivalence Class Testing Strong Equivalence Class Testing Traditional Equivalence Class Testing
Equivalence Class Test Cases for the Triangle Problem Equivalence Class Test Cases for the NextDate Function Equivalence Class Test Cases for the Commission Problem Guidelines and Observations
2
Agenda
Boundary Value Testing
Boundary Value Analysis
Robustness Testing Worst Case Testing Special Value Testing Examples
Generalizing Boundary Value Analysis Limitations of Boundary Value Analysis
Test Cases for the Triangle Problem Test Cases for the NextDate Problem Test Cases for the Commission Problem
Guidelines for Boundary value Testing
Equivalence Class Testing
Equivalence Classes
Weak Equivalence Class Testing Strong Equivalence Class Testing Traditional Equivalence Class Testing
Equivalence Class Test Cases for the Triangle Problem Equivalence Class Test Cases for the NextDate Function Equivalence Class Test Cases for the Commission Problem Guidelines and Observations
3
Boundary Value Analysis
F(x1, x2), a ≤ x1 ≤ b, c ≤ x2 ≤ d
Boundary value analysis focuses on the boundary of the input space to identify test cases The rationale behind boundary value testing is that errors tend to occur near the extreme values of an input variable
e.g. loop conditions (< instead of ≤), counters 4
Boundary Value Analysis
Basic idea: use input variable values at their minimum (min), just above the minimum (min+), a nominal value (nom), just below their maximum (max-), and at their maximum (max) “Single fault” assumption in reliability theory: failures are only rarely the result of the simultaneous occurrence of two (or more) faults. The boundary value analysis test cases are obtained by holding the values of all but one variable at their nominal values, and letting that variable assume its extreme values 5
Boundary Value Analysis F(x1, x2) x2
Input Space (domain)
d
d
c
c a
Test Cases (function of two variables)
x2
b
x1
a
<x1nom , x2min> <x1nom , x2min+> <x 1nom, x2nom > <x1nom , x2max-> <x1nom, x2max> <x1min, x2nom > <x1min+, x2nom > <x 1nom, x2nom > <x1max-, x2nom > <x1max, x2nom>
b
x1
6
Agenda
Boundary Value Testing
Boundary Value Analysis
Robustness Testing Worst Case Testing Special Value Testing Examples
Generalizing Boundary Value Analysis Limitations of Boundary Value Analysis
Test Cases for the Triangle Problem Test Cases for the NextDate Problem Test Cases for the Commission Problem
Guidelines for Boundary value Testing
Equivalence Class Testing
Equivalence Classes
Weak Equivalence Class Testing Strong Equivalence Class Testing Traditional Equivalence Class Testing
Equivalence Class Test Cases for the Triangle Problem Equivalence Class Test Cases for the NextDate Function Equivalence Class Test Cases for the Commission Problem Guidelines and Observations
7
Generalizing Boundary Value Analysis
By the number of variables
For n variables → 4n + 1 test cases
By the kinds of ranges, depends on the type (nature) of the variables
Variables have discrete, bounded values
Variables have no explicit bounds
Create “artificial” bounds e.g. triangle problem
Boolean variables
e.g. NextDate function, commission problem
Decision table-based testing
Logical variables (bound to a value or another logic variable)
e.g. PIN and transaction type in SATM System
8
Agenda
Boundary Value Testing
Boundary Value Analysis
Robustness Testing Worst Case Testing Special Value Testing Examples
Generalizing Boundary Value Analysis Limitations of Boundary Value Analysis
Test Cases for the Triangle Problem Test Cases for the NextDate Problem Test Cases for the Commission Problem
Guidelines for Boundary value Testing
Equivalence Class Testing
Equivalence Classes
Weak Equivalence Class Testing Strong Equivalence Class Testing Traditional Equivalence Class Testing
Equivalence Class Test Cases for the Triangle Problem Equivalence Class Test Cases for the NextDate Function Equivalence Class Test Cases for the Commission Problem Guidelines and Observations
9
Limitations of Boundary Value Analysis
Boundary value analysis works well when the program to be tested is a function of several independent variables that represent bounded physical quantities e.g. NextDate test cases are inadequate (little stress on February, dependencies among month, day, and year) e.g. variables refer to physical quantities, such as temperature, air speed, load etc.
10
Agenda
Boundary Value Testing
Boundary Value Analysis
Robustness Testing Worst Case Testing Special Value Testing Examples
Generalizing Boundary Value Analysis Limitations of Boundary Value Analysis
Test Cases for the Triangle Problem Test Cases for the NextDate Problem Test Cases for the Commission Problem
Guidelines for Boundary value Testing
Equivalence Class Testing
Equivalence Classes
Weak Equivalence Class Testing Strong Equivalence Class Testing Traditional Equivalence Class Testing
Equivalence Class Test Cases for the Triangle Problem Equivalence Class Test Cases for the NextDate Function Equivalence Class Test Cases for the Commission Problem Guidelines and Observations
11
Robustness Testing
Simple extension of boundary value analysis In addition to the five boundary value analysis values of a variable, see what happens when the extrema are exceeded with a value slightly greater than the maximum (max+) and a value slightly less than the minimum (min-) Focuses on the expected outputs
e.g. exceeding load capacity of a public elevator
Forces attention on exception handling 12
Robustness Testing Robustness Test Cases (function of two variables)
x2
<x1nom , x2min> <x1nom , x2min+> <x 1nom, x2nom > <x1nom , x2max-> <x1nom, x2max> <x1min, x2nom > <x1min+, x2nom > <x 1nom, x2nom > <x1max-, x2nom > <x1max, x2nom> <x 1min-, x2nom > <x1nom , x2max+> <x1max+, x2nom> <x 1nom, x2min- >
d
c a
b
x1
13
Agenda
Boundary Value Testing
Boundary Value Analysis
Robustness Testing Worst Case Testing Special Value Testing Examples
Generalizing Boundary Value Analysis Limitations of Boundary Value Analysis
Test Cases for the Triangle Problem Test Cases for the NextDate Problem Test Cases for the Commission Problem
Guidelines for Boundary value Testing
Equivalence Class Testing
Equivalence Classes
Weak Equivalence Class Testing Strong Equivalence Class Testing Traditional Equivalence Class Testing
Equivalence Class Test Cases for the Triangle Problem Equivalence Class Test Cases for the NextDate Function Equivalence Class Test Cases for the Commission Problem Guidelines and Observations
14
Worst Case Testing
Worst case analysis: more than one variable has an extreme value Procedure:
More thorough than boundary value analysis Represents more effort
For each variable create the set <min, min+, nom, max-, max> Take the Cartesian product of these sets to generate test cases
For n variables → 5n test cases (as opposed to 4n+1 test cases for boundary value analysis)
Follows the generalization pattern Same limitations Robust worst case testing can be applied 15
Worst Case Testing Worst Case Test Cases (function of two variables)
x2
d
c a
b
x1
<x 1min, x2min> <x1min, x2min+> <x1min, x2nom > <x1min, x2max-> <x1min, x2max> <x1min+, x2min> <x1min+, x2min+> <x1min+, x2nom > <x1min+, x2max-> <x 1min+, x2max> <x1nom , x2min> <x1nom , x2min+> <x 1nom, x2nom > <x1nom , x2max-> <x1nom, x2max>
<x1max-, x2min> <x1max-, x2min+> <x1max-, x2nom > <x1max-, x2max-> <x 1max-, x2max> <x1max, x2min> <x 1max, x2min+> <x1max, x2nom> <x 1max, x2max-> <x1max, x2max>
16
Worst Case Testing Robust Worst Case Test Cases (function of two variables)
x2
d
c a
b
x1
<x 1min-, x2min- > <x1min- , x2min> <x1min- , x2min+> <x 1min-, x2nom > <x1min- , x2max-> <x1min-, x2max> <x1min- , x2max+> <x1min, x2min-> <x 1min, x2min> <x1min, x2min+> <x1min, x2nom > <x1min, x2max-> <x1min, x2max> <x 1min, x2max+>
<x1min+, x2min- > <x1min+, x2min> <x1min+, x2min+> <x1min+, x2nom > <x1min+, x2max-> <x 1min+, x2max> <x1min+, x2max+> <x 1nom, x2min- > <x1nom , x2min> <x1nom , x2min+> <x 1nom, x2nom > <x1nom , x2max-> <x1nom, x2max> <x1nom , x2max+>
<x1max-, x2min- > <x1max-, x2min> <x1max-, x2min+> <x1max-, x2nom > <x1max-, x2max-> <x 1max-, x2max> <x1max-, x2max+> <x1max, x2min-> <x1max, x2min> <x 1max, x2min+> <x1max, x2nom> <x 1max, x2max-> <x1max, x2max> <x1max, x2max+>
<x1max+, x2min-> <x 1max+, x2min > <x1max+, x2min+> <x1max+, x2nom> <x1max+, x2max-> <x1max+, x2max> <x 1max+, x2max+>
17
Agenda
Boundary Value Testing
Boundary Value Analysis
Robustness Testing Worst Case Testing Special Value Testing Examples
Generalizing Boundary Value Analysis Limitations of Boundary Value Analysis
Test Cases for the Triangle Problem Test Cases for the NextDate Problem Test Cases for the Commission Problem
Guidelines for Boundary value Testing
Equivalence Class Testing
Equivalence Classes
Weak Equivalence Class Testing Strong Equivalence Class Testing Traditional Equivalence Class Testing
Equivalence Class Test Cases for the Triangle Problem Equivalence Class Test Cases for the NextDate Function Equivalence Class Test Cases for the Commission Problem Guidelines and Observations
18
Special Value Testing
The most widely practiced form of functional testing Most intuitive, least uniform, no guidelines The tester uses his/her domain knowledge, experience with similar programs, “ad hoc testing” It is dependent on the abilities of the tester Even though it is highly subjective, it often results in a set of test cases which is more effective in revealing faults than the test sets generated by the other methods 19
Agenda
Boundary Value Testing
Boundary Value Analysis
Robustness Testing Worst Case Testing Special Value Testing Examples
Generalizing Boundary Value Analysis Limitations of Boundary Value Analysis
Test Cases for the Triangle Problem Test Cases for the NextDate Problem Test Cases for the Commission Problem
Guidelines for Boundary value Testing
Equivalence Class Testing
Equivalence Classes
Weak Equivalence Class Testing Strong Equivalence Class Testing Traditional Equivalence Class Testing
Equivalence Class Test Cases for the Triangle Problem Equivalence Class Test Cases for the NextDate Function Equivalence Class Test Cases for the Commission Problem Guidelines and Observations
20
Test Cases for the Triangle Problem Boundary Value Analysis Test Cases
min = 1 min+ = 2 nom = 100 max- = 199 max = 200
Case 1 2 3 4 5 6 7 8 9 10 11 12 13
a 100 100 100 100 100 100 100 100 100 1 2 199 200
b 100 100 100 100 100 1 2 199 200 100 100 100 100
c 1 2 100 199 200 100 100 100 100 100 100 100 100
Expected Output Isosceles Isosceles Equilateral Isosceles Not a Triangle Isosceles Isosceles Isosceles Not a Triangle Isosceles Isosceles Isosceles Not a Triangle
21
Test Cases for the Triangle Problem Worst Case Test Cases (60 of 125) Case 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
a 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2
b 1 1 1 1 1 2 2 2 2 2 100 100 100 100 100 199 199 199 199 199 200 200 200 200 200 1 1 1 1 1
c 1 2 100 199 200 1 2 100 199 200 1 2 100 199 200 1 2 100 199 200 1 2 100 199 200 1 2 100 199 200
Expected Output Equilateral Not a Triangle Not a Triangle Not a Triangle Not a Triangle Not a Triangle Isosceles Not a Triangle Not a Triangle Not a Triangle Not a Triangle Not a Triangle Isosceles Not a Triangle Not a Triangle Not a Triangle Not a Triangle Not a Triangle Isosceles Not a Triangle Not a Triangle Not a Triangle Not a Triangle Not a Triangle Isosceles Not a Triangle Isosceles Not a Triangle Not a Triangle Not a Triangle
Case 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60
a 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 100 100 100 100 100 100 100 100 100 100
b 2 2 2 2 2 100 100 100 100 100 199 199 199 199 199 200 200 200 200 200 1 1 1 1 1 2 2 2 2 2
c 1 2 100 199 200 1 2 100 199 200 1 2 100 199 200 1 2 100 199 200 1 2 100 199 200 1 2 100 199 200
Expected Output Isosceles Equilateral Not a Triangle Not a Triangle Not a Triangle Not a Triangle Not a Triangle Isosceles Not a Triangle Not a Triangle Not a Triangle Not a Triangle Not a Triangle Isosceles Scalene Not a Triangle Not a Triangle Not a Triangle Scalene Isosceles Not a Triangle Not a Triangle Isosceles Not a Triangle Not a Triangle Not a Triangle Not a Triangle Isosceles Not a Triangle Not a Triangle
22
Agenda
Boundary Value Testing
Boundary Value Analysis
Robustness Testing Worst Case Testing Special Value Testing Examples
Generalizing Boundary Value Analysis Limitations of Boundary Value Analysis
Test Cases for the Triangle Problem Test Cases for the NextDate Problem Test Cases for the Commission Problem
Guidelines for Boundary value Testing
Equivalence Class Testing
Equivalence Classes
Weak Equivalence Class Testing Strong Equivalence Class Testing Traditional Equivalence Class Testing
Equivalence Class Test Cases for the Triangle Problem Equivalence Class Test Cases for the NextDate Function Equivalence Class Test Cases for the Commission Problem Guidelines and Observations
23
Test Cases for the NextDate Problem Boundary Value Analysis Test Cases
month min = 1 min+ = 2 nom = 6 max- = 11 max = 12
day min = 1 min+ = 2 nom = 15 max- = 30 max = 31
year min = 1812 min+ = 1813 nom = 1912 max- = 2011 max = 2012
Case month day 1 6 15 2 6 15 3 6 15 4 6 15 5 6 15 6 6 1 7 6 2 8 6 30 9 6 31 10 1 15 11 2 15 12 11 15 13 12 15
year 1812 1813 1912 2011 2012 1912 1912 1912 1912 1912 1912 1912 1912
Expected Output June 16, 1812 June 16, 1813 June 16, 1912 June 16, 2011 June 16, 2012 June 2, 1912 June 3, 1912 July 1, 1912 error January 16, 1912 February 16, 1912 November 16, 1912 December 16, 1912
24
Test Cases for the NextDate Problem Worst Case Test Cases (60 of 125) Case 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
month 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2
day 1 1 1 1 1 2 2 2 2 2 15 15 15 15 15 30 30 30 30 30 31 31 31 31 31 1 1 1 1 1
year 1812 1813 1912 2011 2012 1812 1813 1912 2011 2012 1812 1813 1912 2011 2012 1812 1813 1912 2011 2012 1812 1813 1912 2011 2012 1812 1813 1912 2011 2012
Expected Output January 2, 1812 January 2, 1813 January 2, 1912 January 2, 2011 January 2, 2012 January 3, 1812 January 3, 1813 January 3, 1912 January 3, 2011 January 3, 2012 January 16, 1812 January 16, 1813 January 16, 1912 January 16, 2011 January 16, 2012 January 31, 1812 January 31, 1813 January 31, 1912 January 31, 2011 January 31, 2012 February 1, 1812 February 1, 1813 February 1, 1912 February 1, 2011 February 1, 2012 February 2, 1812 February 2, 1813 February 2, 1912 February 2, 2011 February 2, 2012
Case 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60
month 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 6 6 6 6 6 6 6 6 6 6
day 2 2 2 2 2 15 15 15 15 15 30 30 30 30 30 31 31 31 31 31 1 1 1 1 1 2 2 2 2 2
year 1812 1813 1912 2011 2012 1812 1813 1912 2011 2012 1812 1813 1912 2011 2012 1812 1813 1912 2011 2012 1812 1813 1912 2011 2012 1812 1813 1912 2011 2012
Expected Output February 3, 1812 February 3, 1813 February 3, 1912 February 3, 2011 February 3, 2012 February 16, 1812 February 16, 1813 February 16, 1912 February 16, 2011 February 16, 2012 error error error error error error error error error error June 2, 1812 June 2, 1813 June 2, 1912 June 2, 2011 June 2, 2012 June 3, 1812 June 3, 1813 June 3, 1912 June 3, 2011 June 3, 2012
25
Agenda
Boundary Value Testing
Boundary Value Analysis
Robustness Testing Worst Case Testing Special Value Testing Examples
Generalizing Boundary Value Analysis Limitations of Boundary Value Analysis
Test Cases for the Triangle Problem Test Cases for the NextDate Problem Test Cases for the Commission Problem
Guidelines for Boundary value Testing
Equivalence Class Testing
Equivalence Classes
Weak Equivalence Class Testing Strong Equivalence Class Testing Traditional Equivalence Class Testing
Equivalence Class Test Cases for the Triangle Problem Equivalence Class Test Cases for the NextDate Function Equivalence Class Test Cases for the Commission Problem Guidelines and Observations
26
Test Cases for the Commission Problem
13 Boundary Value Analysis Test Cases 125 Worst Case Test Cases Boundary values for the output range, especially near the threshold points of $1000 and $1800 Part of the reason for using the output range to determine test cases is that test cases from the input range are almost all in the 20% zone We want to find input variable combinations that stress the boundary values: $100, $1000, $1800, and $7800 27
Test Cases for the Commission Problem Output Boundary Value Analysis Test Cases Case 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
locks 1 1 1 2 5 10 10 9 10 10 10 11 14 18 18 17 18 18 18 19 48 70 70 69 70
stocks 1 1 2 1 5 10 9 10 10 10 11 10 14 18 17 18 18 18 19 18 48 80 79 80 80
barrels 1 2 1 1 5 9 10 10 10 11 10 10 14 17 18 18 18 19 18 18 48 89 90 90 90
sales 100 125 130 145 500 975 970 955 1000 1025 1030 1045 1400 1775 1770 1755 1800 1825 1830 1845 4800 7775 7770 7755 7800
commission 10 12.5 13 14.5 50 97.5 97 95.5 100 103.75 104.5 106.75 160 216.25 215.5 213.25 220 225 226 229 820 1415 1414 1411 1420
comment output minimum output minimum + output minimum + output minimum + midpoint border point border point border point border point border point + border point + border point + midpoint border point border point border point border point border point + border point + border point + midpoint output maximum output maximum output maximum output maximum
28
Test Cases for the Commission Problem
Test case 9 is the $1000 border point If we tweak the input variables we get values just below and just above the border Form of special value testing based on mathematical insight Output Special Value Test Cases Case 1 2 3
locks 10 18 18
stocks 11 17 19
barrels 9 19 17
sales 1005 1795 1805
commission 100.75 219.25 221
comment border point + border point border point +
29
Agenda
Boundary Value Testing
Boundary Value Analysis
Robustness Testing Worst Case Testing Special Value Testing Examples
Generalizing Boundary Value Analysis Limitations of Boundary Value Analysis
Test Cases for the Triangle Problem Test Cases for the NextDate Problem Test Cases for the Commission Problem
Guidelines for Boundary value Testing
Equivalence Class Testing
Equivalence Classes
Weak Equivalence Class Testing Strong Equivalence Class Testing Traditional Equivalence Class Testing
Equivalence Class Test Cases for the Triangle Problem Equivalence Class Test Cases for the NextDate Function Equivalence Class Test Cases for the Commission Problem Guidelines and Observations
30
Equivalence Classes
Motivations
Have a sense of complete testing Avoid redundancy
Equivalence classes form a partition of a set, where partition refers to a collection of mutually disjoint subsets whose union is the entire set (completeness, non-redundancy) The idea is to identify test cases by using one element from each equivalence class “treated the same” → “traversing the same execution path” The key is the choice of the equivalence relation that determines the classes
“second guess” the likely implementation , and think about the functional manipulations that must somehow be present in the implementation
31
Equivalence Classes
Program under consideration: Function of variables a, b, c Input domain consists of sets A, B, and C A = A1 A2 A3 B = B1 B2 B3 B4 C = C1 C2 a1 ∈ A1, b3 ∈ B3, c2 ∈ C2
32
Agenda
Boundary Value Testing
Boundary Value Analysis
Robustness Testing Worst Case Testing Special Value Testing Examples
Generalizing Boundary Value Analysis Limitations of Boundary Value Analysis
Test Cases for the Triangle Problem Test Cases for the NextDate Problem Test Cases for the Commission Problem
Guidelines for Boundary value Testing
Equivalence Class Testing
Equivalence Classes
Weak Equivalence Class Testing Strong Equivalence Class Testing Traditional Equivalence Class Testing
Equivalence Class Test Cases for the Triangle Problem Equivalence Class Test Cases for the NextDate Function Equivalence Class Test Cases for the Commission Problem Guidelines and Observations
33
Weak Equivalence Class Testing
Accomplished by using one variable from each equivalence class in a test case
A = A1 A2 A3 B = B1 B2 B3 B4 C = C1 C2 a1 ∈ A1, b3 ∈ B3, c2 ∈ C2
Test Case WE1 WE2 WE3 WE4
a a1 a2 a3 a1
b b1 b2 b3 b4
c c1 c2 c1 c2
Number of weak equivalence class test cases = number of classes in the partition with the largest number of subsets 34
Agenda
Boundary Value Testing
Boundary Value Analysis
Robustness Testing Worst Case Testing Special Value Testing Examples
Generalizing Boundary Value Analysis Limitations of Boundary Value Analysis
Test Cases for the Triangle Problem Test Cases for the NextDate Problem Test Cases for the Commission Problem
Guidelines for Boundary value Testing
Equivalence Class Testing
Equivalence Classes
Weak Equivalence Class Testing Strong Equivalence Class Testing Traditional Equivalence Class Testing
Equivalence Class Test Cases for the Triangle Problem Equivalence Class Test Cases for the NextDate Function Equivalence Class Test Cases for the Commission Problem Guidelines and Observations
35
Strong Equivalence Class Testing
Based on the Cartesian Product of the partition subsets A x B x C = 3 x 4 x 2 = 24 elements Equivalence relations can be defined on the output range of the program function being tested
Test Case SE1 SE2 SE3 SE4 SE5 SE6 SE7 SE8 SE9 SE10 SE11 SE12 SE13 SE14 SE15 SE16 SE17 SE18 SE19 SE20 SE21 SE22 SE23 SE24
a a1 a1 a1 a1 a1 a1 a1 a1 a2 a2 a2 a2 a2 a2 a2 a2 a3 a3 a3 a3 a3 a3 a3 a3
b b1 b1 b2 b2 b3 b3 b4 b4 b1 b1 b2 b2 b3 b3 b4 b4 b1 b1 b2 b2 b3 b3 b4 b4
c c1 c2 c1 c2 c1 c2 c1 c2 c1 c2 c1 c2 c1 c2 c1 c2 c1 c2 c1 c2 c1 c2 c1 c2 36
Agenda
Boundary Value Testing
Boundary Value Analysis
Robustness Testing Worst Case Testing Special Value Testing Examples
Generalizing Boundary Value Analysis Limitations of Boundary Value Analysis
Test Cases for the Triangle Problem Test Cases for the NextDate Problem Test Cases for the Commission Problem
Guidelines for Boundary value Testing
Equivalence Class Testing
Equivalence Classes
Weak Equivalence Class Testing Strong Equivalence Class Testing Traditional Equivalence Class Testing
Equivalence Class Test Cases for the Triangle Problem Equivalence Class Test Cases for the NextDate Function Equivalence Class Test Cases for the Commission Problem Guidelines and Observations
37
Traditional Equivalence Class Testing
Defines equivalence classes in terms of validity Commission problem
Valid inputs: 1 ≤ lock ≤ 70, 1 ≤ stock ≤ 80, 1 ≤ barrel ≤ 90 Invalid inputs: lock < 1, lock > 70, stock < 1, stock > 80, barrel < 1, barrel > 90
For valid inputs, use one value from each valid class (like weak equivalence testing) For invalid inputs, a test case will have one invalid value and the remaining values will all be valid (single failure) 38
Traditional Equivalence Class Testing
Problems: Very often, the specification does not define what the expected output for an invalid test case should be → a lot of time spent in defining expected outputs Strongly typed languages eliminate the need for the consideration of invalid inputs
39
Agenda
Boundary Value Testing
Boundary Value Analysis
Robustness Testing Worst Case Testing Special Value Testing Examples
Generalizing Boundary Value Analysis Limitations of Boundary Value Analysis
Test Cases for the Triangle Problem Test Cases for the NextDate Problem Test Cases for the Commission Problem
Guidelines for Boundary value Testing
Equivalence Class Testing
Equivalence Classes
Weak Equivalence Class Testing Strong Equivalence Class Testing Traditional Equivalence Class Testing
Equivalence Class Test Cases for the Triangle Problem Equivalence Class Test Cases for the NextDate Function Equivalence Class Test Cases for the Commission Problem Guidelines and Observations
40
Equivalence Class Test Cases for the Triangle Problem
Outputs: Not a Triangle, Scalene, Isosceles, Equilateral Easier to identify output (range) equivalence classes
R1 = {
: the triangle with sides a, b, and c is equilateral} R2 = { : the triangle with sides a, b, and c is isosceles} R3 = { : the triangle with sides a, b, and c is scalene} R4 = { : sides a, b, and c do not form a triangle} Test Case OE1 OE2 OE3 OE4
a 5 2 3 4
b 5 2 4 1
c 5 3 5 2
Expected Output Equilateral Isosceles Scalene Not a Triangle
41
Equivalence Class Test Cases for the Triangle Problem
Input (domain) equivalence classes D1 = { : a = b = c} D2 = { : a = b, a ≠ c} D3 = { : a = c, a ≠ b} D4 = { : b = c, a ≠ b} D5 = { : a ≠ b, a ≠ c, b ≠ c} // D6 = { : a ≥ b + c} D6’ = { : a = b + c} D6’’ = { : a > b + c} // D7 = { : b ≥ a + c} D7’ = { : b = a + c} D7’’ = { : b > a + c} // D8 = { : c ≥ a + b} D8’ = { : c = a + b} D8’’ = { : c > a + b} 42
Agenda
Boundary Value Testing
Boundary Value Analysis
Robustness Testing Worst Case Testing Special Value Testing Examples
Generalizing Boundary Value Analysis Limitations of Boundary Value Analysis
Test Cases for the Triangle Problem Test Cases for the NextDate Problem Test Cases for the Commission Problem
Guidelines for Boundary value Testing
Equivalence Class Testing
Equivalence Classes
Weak Equivalence Class Testing Strong Equivalence Class Testing Traditional Equivalence Class Testing
Equivalence Class Test Cases for the Triangle Problem Equivalence Class Test Cases for the NextDate Function Equivalence Class Test Cases for the Commission Problem Guidelines and Observations
43
Equivalence Class Test Cases for the NextDate Function
Input variables
1 ≤ month ≤ 12 1 ≤ day ≤ 31 1812 ≤ year ≤ 2012
Traditional approach
Valid equivalence classes
M1 = { month : 1 ≤ month ≤ 12 } D1 = { day : 1 ≤ day ≤ 31 } Y1 = { year : 1812 ≤ year ≤ 2012 }
Invalid equivalence classes
M2 = { month : month < 1 } M3 = { month : month > 12 } D2 = { day : day < 1 } D3 = { day : day >31 } Y2 = { year : year < 1812 } Y3 = {year : year > 2012 }
Case ID TE1 TE2 TE3 TE4 TE5 TE6 TE7
Month 6 -1 13 6 6 6 6
Day 15 15 15 -1 32 15 15
Year 1912 1912 1912 1912 1912 1811 2013
Expected Output 6/16/1912 invalid input invalid input invalid input invalid input invalid input invalid input
44
Equivalence Class Test Cases for the NextDate Function
Traditional approach is deficient because it “treats” the elements of a class at the valid/invalid level Different approach: What must be done to an input date? M1 = { month: month has 30 days } M2 = { month: month has 31 days } M3 = { month: month is February } D1 = { day: 1 ≤ day ≤ 28 } D2 = { day: day = 29 } D3 = { day: day = 30 } D4 = { day: day = 31 } Y1 = { year: year = 1900 } Y2 = { year: 1812 ≤ year ≤ 2012 AND (year ≠ 1900) AND (year mod 4 = 0) } Y3 = { year: 1812 ≤ year ≤ 2012 AND (year mod 4 ≠ 0) } Not a perfect set of equivalence classes!!! 45
Equivalence Class Test Cases for the NextDate Function Weak equivalence class test cases Case ID WE1 WE2 WE3 WE4
Month 6 7 2 6
Day 14 29 30 31
Year 1900 1912 1913 1900
Expected Output 6/15/1900 7/30/1912 invalid input invalid input
Strong equivalence class test cases Case ID SE1 SE2 SE3 SE4 SE5 SE6 SE7 SE8 SE9 SE10 SE11 SE12 SE13 SE14 SE15 SE16 SE17 SE18
Month 6 6 6 6 6 6 6 6 6 6 6 6 7 7 7 7 7 7
Day 14 14 14 29 29 29 30 30 30 31 31 31 14 14 14 29 29 29
Year 1900 1912 1913 1900 1912 1913 1900 1912 1913 1900 1912 1913 1900 1912 1913 1900 1912 1913
Expected Output 6/15/1900 6/15/1912 6/15/1913 6/30/1900 6/30/1912 6/30/1913 07/01/1900 07/01/1912 07/01/1913 ERROR ERROR ERROR 7/15/1900 7/15/1912 7/15/1913 7/30/1900 7/30/1912 7/30/1913
Case ID SE19 SE20 SE21 SE22 SE23 SE24 SE25 SE26 SE27 SE28 SE29 SE30 SE31 SE32 SE33 SE34 SE35 SE36
Month 7 7 7 7 7 7 2 2 2 2 2 2 2 2 2 2 2 2
Day 30 30 30 31 31 31 14 14 14 29 29 29 30 30 30 31 31 31
Year 1900 1912 1913 1900 1912 1913 1900 1912 1913 1900 1912 1913 1900 1912 1913 1900 1912 1913
Expected Output 7/31/1900 7/31/1912 7/31/1913 08/01/1900 08/01/1912 08/01/1913 2/15/1900 2/15/1912 2/15/1913 ERROR 03/01/1912 ERROR ERROR ERROR ERROR ERROR ERROR ERROR
46
Agenda
Boundary Value Testing
Boundary Value Analysis
Robustness Testing Worst Case Testing Special Value Testing Examples
Generalizing Boundary Value Analysis Limitations of Boundary Value Analysis
Test Cases for the Triangle Problem Test Cases for the NextDate Problem Test Cases for the Commission Problem
Guidelines for Boundary value Testing
Equivalence Class Testing
Equivalence Classes
Weak Equivalence Class Testing Strong Equivalence Class Testing Traditional Equivalence Class Testing
Equivalence Class Test Cases for the Triangle Problem Equivalence Class Test Cases for the NextDate Function Equivalence Class Test Cases for the Commission Problem Guidelines and Observations
47
Equivalence Class Test Cases for the Commission Problem
Input Domain Equivalence Classes
Lock
Stock
L1 = { lock: 1 ≤ lock ≤ 70 } L2 = { lock: lock < 1 } L3 = { lock: lock > 70 } S1 = { stock: 1 ≤ stock ≤ 80 } S2 = { stock: stock < 1 } S3 = { stock: stock > 80 }
Barrel
B1 = { barrel: 1 ≤ barrel ≤ 90 } B2 = { barrel: barrel < 1 } B3 = { barrel: barrel > 90 }
48
Equivalence Class Test Cases for the Commission Problem Strong Input Domain Equivalence Class Test Cases
Weak Input Domain Equivalence Class Test Cases Test Case WE1 WE2 WE3
locks 35 0 71
stocks 40 0 81
barrels sales commission 45 3900 640 0 ERROR ERROR 91 ERROR ERROR
Test Case SE1 SE2 SE3 SE4 SE5 SE6 SE7 SE8 SE9 SE10 SE11 SE12 SE13 SE14 SE15 SE16 SE17 SE18 SE19 SE20 SE21 SE22 SE23 SE24 SE25 SE26 SE27
locks 35 35 35 35 35 35 35 35 35 0 0 0 0 0 0 0 0 0 71 71 71 71 71 71 71 71 71
stocks 40 40 40 0 0 0 81 81 81 40 40 40 0 0 0 81 81 81 40 40 40 0 0 0 81 81 81
barrels 45 0 91 45 0 91 45 0 91 45 0 91 45 0 91 45 0 91 45 0 91 45 0 91 45 0 91
sales commission 3900 640 ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR ERROR
49
Equivalence Class Test Cases for the Commission Problem
sales = 45 x locks + 30 x stocks + 25 x barrels L1 = { : sales ≤ 1000 } L2 = { : 1000 < sales ≤ 1800 } L3 = { : sales > 1800 } Output Range Equivalence Class Test Cases
Test Case OR1 OR2 OR3
locks 5 15 25
stocks 5 15 25
barrels 5 15 25
sales 500 1500 2500
commission 50 175 360
50
Agenda
Boundary Value Testing
Boundary Value Analysis
Robustness Testing Worst Case Testing Special Value Testing Examples
Generalizing Boundary Value Analysis Limitations of Boundary Value Analysis
Test Cases for the Triangle Problem Test Cases for the NextDate Problem Test Cases for the Commission Problem
Guidelines for Boundary value Testing
Equivalence Class Testing
Equivalence Classes
Weak Equivalence Class Testing Strong Equivalence Class Testing Traditional Equivalence Class Testing
Equivalence Class Test Cases for the Triangle Problem Equivalence Class Test Cases for the NextDate Function Equivalence Class Test Cases for the Commission Problem Guidelines and Observations
51
Guidelines and Observations 1.
2.
3.
4.
The traditional form of equivalence class testing is generally not as thorough as weak equivalence class testing, which in turn, is not as thorough as the strong form of equivalence class testing The only time it makes sense to use the traditional approach is when the implementation language is not strongly typed If error conditions are a high priority, we could extend strong equivalence class testing to include invalid classes (e.g. commission problem) Equivalence class testing is appropriate when input data is defined in terms of ranges and sets of discrete values. This is certainly the case when system malfunctions can occur for out-of-limit variable values 52
Guidelines and Observations 1.
2.
3.
Equivalence class testing is strengthened by a hybrid approach with boundary value testing. (We can “reuse” the effort made in defining the equivalence classes) (e.g. NextDate function) Equivalence class testing is indicated when the program function is complex. In such cases, the complexity of the function can help identify useful equivalence classes, as in the NextDate function Strong equivalence class testing makes a presumption that the variables are independent when the Cartesian Product is taken. If there are any dependencies, these will often generate “error” test cases, as they did in the NextDate function 53
Guidelines and Observations 1.
Several tries may be needed before “the right” equivalence relation is discovered, as we saw in the NextDate example. In other cases, there is an “obvious” or “natural” equivalence partition. When in doubt, the best bet is to try to second guess aspects of any reasonable implementation 54
Agenda
Boundary Value Testing
Boundary Value Analysis
Robustness Testing Worst Case Testing Special Value Testing Examples
Generalizing Boundary Value Analysis Limitations of Boundary Value Analysis
Test Cases for the Triangle Problem Test Cases for the NextDate Problem Test Cases for the Commission Problem
Guidelines for Boundary value Testing
Equivalence Class Testing
Equivalence Classes
Weak Equivalence Class Testing Strong Equivalence Class Testing Traditional Equivalence Class Testing
Equivalence Class Test Cases for the Triangle Problem Equivalence Class Test Cases for the NextDate Function Equivalence Class Test Cases for the Commission Problem Guidelines and Observations
55
References
Software Testing A Craftsman's Approach 2nd edition, Paul C. Jorgensen, CRC Press (Chapters 5 and 6)
56