Validation And Verification

Validation and Verification Validation: •Establishing the fitness of a software product for its use. •“Are we building the right product?” •Requires interaction with customers. Verification: •Establishing the correspondence between the software and its specification. •“Are we building the product right?” •Requires interaction with software. Static V&V: •Software inspections •Static analysis of source code -Control/data flow analysis Dynamic V&V: •Defect-testing -Looks for errors in functionality •Load testing -Looks for errors in scalability, performance, and reliability Failure: •Occurs when a program misbehaves. Fault: •A problem that exists in source code. Error: •A human action that results in software containing a fault. An error leads to the inclusion of a fault that may or may not lead to a failure. Testing cannot prove that a program never fails; only that it contains faults. Software Testing: •The process of executing a program and comparing the actual behavior with the expected behavior. •Deviations between actual and expected behavior should indicate defects in the program whose removal improves the quality of the software. Defect classification: •The process of identifying the type of deviation between actual and expected (design error, specification error, testing error, etc.) Debugging: •The process of tracking down the source of the defect and removing it. Exhaustive testing: •Execute program on all possible inputs and compare actual to expected behavior. •Could “prove” program correctness. •Not practical for any non-trivial program. Practical testing: •Select a tiny % of all possible tests. •Goal: executing the tiny % of tests will uncover a large % of defects present! •A “testing method” is essentially a way to decide which tiny % to pick. White box testing (also “Clear” or “Glass”) •Assume knowledge of code •Coveragebased testing •Automated tools available to assess coverage

Black box testing •Don’t assume knowledge of code •Specification-based testing •Automated tools available (requires formal specification) other methods exist •Mutation testing, etc. Statement coverage For a test case to uncover a defect, the statement containing the defect must be executed. Therefore, a set of test cases, which guarantees all statements are executed, might uncover a large number of the defects present. Whether or not the defects are actually uncovered depends upon the program state at the time each statement is executed. Control flow coverage Control flow coverage adds conditions to statement coverage to raise the odds of discovering defects. Branch coverage: •Every conditional is evaluated as both true and false during testing. Loop coverage: •Every loop must be executed both 0 times and more than 1 time. Path coverage: •All permutations of paths through the program are taken Specification based testing Specification: •A mapping between the possible “inputs” to the program and the corresponding expected “outputs” Specification-based testing: •Design a set of test cases to see if inputs actually map to outputs. •Does not require access to source code Differences with White Box (coverage) testing: •Can catch errors of omission. •Effectiveness depends upon a high quality specification Equivalence classes Goal: Divide the possible inputs into categories such that testing one point in each category is equivalent to testing all points in the category. Provide one test case for each point. Equivalence class definition is usually an iterative process and goes on throughout development. Use heuristics to get you started designing your test cases. Additions for web app test Case design Every page is retrieved at least once •Prevent 404 errors. Every link is followed at least once. •Prevent 404 errors. All form input fields are tested with: •Normal values •Erroneous values •Maximum/minimum values Always check response for appropriateness. Yamaura, IEEE Software, Nov. 1998 •Proper test case density is one test case per 10-15 LOC. •Test case type percentages: -Basic and normal tests < 60% -Boundary and limit tests >10%, -Error

tests > 15% •Run a 48 hour continuous operation test (reiterating basic functions) for memory leaks, deadlock, time-out, etc. •Tendency is to write too many tests for well Understood functions, and too few for poorly understood. Use density to uncover this.