L10 L11- Software Testing

Software Engineering

Softwa re Testing 

Version 1.0



“ Te stin g ca n o n ly sh o w th e p re se n ce o f e rro rs , n o t th e ir a b se n ce s . ” - E d sg e r D ijk stra Rushdi Shams, Lecturer, Dept of CSE, KUET

1

The testing process Component testing  Testing of individual program components;  Usually the responsibility of the component developer

(except sometimes for critical systems);  Tests are derived from the developer’s experience.

System testing  Testing of groups of components integrated to create a

system or sub-system;  The responsibility of an independent testing team;  Tests are based on a system specification.

Rushdi Shams, Lecturer, Dept of CSE, KUET

2

Testing phases Component testing Software developer

System testing Independent testing team

Rushdi Shams, Lecturer, Dept of CSE, KUET

3

Testing process goals Validation testing  To demonstrate to the developer and the system

customer that the software meets its requirements;  A successful test shows that the system operates as intended.

Defect testing  To discover faults or defects in the software where its

behaviour is incorrect or not in conformance with its specification;  A successful test is a test that makes the system perform incorrectly and so exposes a defect in the system. Rushdi Shams, Lecturer, Dept of CSE, KUET

4

The software testing process Test cases

Design test cases

Test data

Prepare test data

Test results

Run program with test da ta

Rushdi Shams, Lecturer, Dept of CSE, KUET

Test reports

Compare results to test cases

5

Testing policies Only exhaustive testing can show a program is

free from defects. However, exhaustive testing is impossible, Testing policies define the approach to be used in selecting system tests:  All functions accessed through menus should be

tested;  Combinations of functions accessed through the same menu should be tested;  Where user input is required, all functions must be tested with correct and incorrect input.

Rushdi Shams, Lecturer, Dept of CSE, KUET

6

System testing Involves integrating components to create a

system or sub-system. May involve testing an increment to be delivered to the customer. Two phases:

Integration testing - the test team have access

to the system source code. The system is tested as components are integrated. Release testing - the test team test the complete system to be delivered as a blackbox. Rushdi Shams, Lecturer, Dept of CSE, KUET

7

Integration testing Involves building a system from its components

and testing it for problems that arise from component interactions. Top-down integration

 Develop the skeleton of the system and populate it with

components.

Bottom-up integration  Integrate infrastructure components then add

functional components.

To simplify error localisation, systems should be

incrementally integrated.

Rushdi Shams, Lecturer, Dept of CSE, KUET

8

Incremental integration testing A A

T1

T1

T2

A

T2 T2

B T3

B T3

B

C

T3

T4 C

T4 D

Test sequence 1

T1

Test sequence 2

Rushdi Shams, Lecturer, Dept of CSE, KUET

T5

Test sequence 3 9

Regression Testing When incremental integration testing is done

automatically (with software tools), the test is called regression testing. www.junit.org 

Rushdi Shams, Lecturer, Dept of CSE, KUET

10

Rushdi Shams, Lecturer, Dept of CSE, KUET

11

Testing approaches Architectural validation  Top-down integration testing is better at discovering

errors in the system architecture.

System demonstration  Top-down integration testing allows a limited

demonstration at an early stage in the development.

Test implementation  Often easier with bottom-up integration testing.

Test observation  Problems with both approaches. Extra code may be

required to observe tests.

Rushdi Shams, Lecturer, Dept of CSE, KUET

12

Release testing The process of testing a release of a system

that will be distributed to customers. Primary goal is to increase the supplier’s confidence that the system meets its requirements. Release testing is usually black-box or functional testing

Based on the system specification only; Testers do not have knowledge of the system

implementation.

Rushdi Shams, Lecturer, Dept of CSE, KUET

13

Black-box testing Input test da ta

Ie

Inputs causing anomalous behaviour

Oe

Outputs w hich reveal the presence of defects

System

Output testesults r

Rushdi Shams, Lecturer, Dept of CSE, KUET

14

Acceptance Testing When release testing is done in presence of

clients, that time, s/he may confirm whether s/he accepts the release or not. This type of release testing is called acceptance testing

Rushdi Shams, Lecturer, Dept of CSE, KUET

15

Testing guidelines Testing guidelines are hints for the testing team to

help them choose tests that will reveal defects in the system  Choose inputs that force the system to generate all

error messages;  Design inputs that cause buffers to overflow;  Repeat the same input or input series several times;  Force invalid outputs to be generated;  Force computation results to be too large or too small.

Rushdi Shams, Lecturer, Dept of CSE, KUET

16

Performance testing Part of release testing may involve testing the

emergent properties of a system, such as performance and reliability. Performance tests usually involve planning a series of tests where the load is steadily increased until the system performance becomes unacceptable.

Rushdi Shams, Lecturer, Dept of CSE, KUET

17

Stress testing Exercises the system beyond its maximum design

load. Stressing the system often causes defects to come to light. Stressing the system test failure behaviour.. Systems should not fail catastrophically. Stress testing checks for unacceptable loss of service or data. Stress testing is particularly relevant to distributed systems that can exhibit severe degradation as a network becomes overloaded. Rushdi Shams, Lecturer, Dept of CSE, KUET

18

Sanity Test vs Smoke Test Sanity test determines whether it is reasonable

to proceed with further test. Smoke test determines whether it is possible to proceed with further test. A software smoke test determines whether the program launches and whether its interfaces are accessible and responsive (for example, the responsiveness of a web page or an input button). If the smoke test fails, it is impossible to conduct a sanity test.   Rushdi Shams, Lecturer, Dept of CSE, KUET

19

Sanity Test Vs Smoke Test sanity test exercises the smallest subset of

application functions needed to determine whether the application logic is generally functional and correct (for example, an interest rate calculation for a financial application). If the sanity test fails, it is not reasonable to attempt more rigorous testing. Both are ways to avoid wasting time and effort by quickly determining whether an application is too flawed to merit any rigorous testing. Rushdi Shams, Lecturer, Dept of CSE, KUET 20

Testing scenario A student in Scotland is studying American History and has been asked to write a paper on ŌFrontier mentality in the American West from 1840 to 1880Õ.To do this, she needs to find sources from a range of libraries. She logs on to the LIBSYS system and uses the search facility to discover if she can access original documents from that time. She discovers sources in various US university libraries and downloads copies of some of these. However, for one document, she needs to have confirmation from her university that she is a genuine student and that use is for non-commercial purposes. The student then uses the facility in LIBSYS that can request such permission and registers her request. If granted, the document will be downloaded to the registered libraryÕs server and printed for her. She receives a message from LIBSYS telling her that she will receive an e-mail message when the printed document is available for collection.

Rushdi Shams, Lecturer, Dept of CSE, KUET

21

System tests 1 . T es t the login mec hanism us ing c orre ct and in co rrect logins to che c k tha t v alid users are a c cep ted and inv alid us ers are reje cted. 2 . T es t the search fa cility u sing d if fere nt queries a gains t known sources to che ck tha t the se arch me cha nism is ac tua l y finding do cu me nts. 3 . T es t the sy st em pre se ntation fa c liity to ch eck th at in forma tion about do cu me nts is displayed prop erly. 4 . T es t the me chanism ot requ es t perm ission for downlo ading. 5 . T es t the e-mail respons e indic a itng that the downlo aded doc um en t is ava ila ble .

Rushdi Shams, Lecturer, Dept of CSE, KUET

22