Testing Fundamentals

SOTWARE TESTING GUIDE

Testing Fundamentals Introduction to Software Testing and its importance Software Testing  Software testing is a process to identify the correctness, completeness and quality of developed software, it includes a set of activities conducted with the intent of finding errors in software, so that it could be corrected before the product is released to the end-users.  In simple words software testing is an activity to check that the software system is defect free. Why Software Testing is Important? Testing is important because software bugs could be expensive or even dangerous. Software bugs can potentially cause monetary and human loss, history is full of such examples. China Airlines Airbus A300 crash, Canada's Therac-25 radiation therapy machine malfunction, Nissan cars have to recall over 1 million cars from the market due to software failure in the airbag sensory detectors etc. Types of Software Testing  Functional Testing.  Unit Testing  Integration Testing  Smoke  UAT ( User Acceptance Testing)  So on  Non-Functional Testing or Performance Testing.  Performance  Endurance  Load  So on  Maintenance Testing.  Regression  Maintenance

Fundamental Principles of Software Testing 7 Software Testing Principles:  Exhaustive testing is impossible:  Exhaustive testing is not possible. Instead, we need the optimal amount of testing based on the risk assessment of the application.

 Defect Clustering:  Defect Clustering in Software Testing means that the majority of the defects are caused by a small number of modules, i.e. the distribution of defects are not across the application but rather centralized in limited sections of the application.  Defect Clustering in Software Testing is based on the Pareto principle, also known as the 80-20 rule, where it is stated that approximately 80% of the problems are caused by 20% of the modules.  Pesticide Paradox:  Pesticide paradox in software testing is the process of repeating the same test cases again and again eventually the same test cases will no longer find new bugs.  So to overcome the pesticide paradox it is necessary to review the test cases regularly and add or update them to find more defects.  Testing shows presence of defects:  Testing talks about the presence of defects and don’t talk about the absence of defects. i.e. Software Testing reduces the probability of undiscovered defects remaining in the software but even if no defects are found, it is not a proof of correctness.  Absence of errors – fallacy:  Finding and fixing defects does not help if the system build is unusable and does not fulfil the user's needs & requirements.  Early Testing:  Testing should start as early as possible in the Software Development Life Cycle. So that any defects in the requirements or design phase are captured in early stages. It is much cheaper to fix a Defect in early stages of testing.  It is recommended that you start finding the bug the moment the requirements are defined.  Testing is context dependent:  Testing is context dependent which basically means that the way you test an e-commerce site will be different from the way you test a commercial off the shelf application.  All the developed software’s are not identical.  You might use a different approach, methodologies, techniques and types of testing depending upon the application type.  For instance testing, any POS system at a retail store will be different than testing an ATM machine.

SDLC Software Development Life Cycle (SDLC) is a process used by the software industry to design, develop and test high quality software’s. The SDLC aims to produce a high-quality Software that meets or exceeds customer expectations, reaches completion within times and cost estimates.  SDLC is the acronym of Software Development Life Cycle.  It is also called as Software Development Process.  SDLC is a framework defining tasks performed at each step in the software development process.  It consists of a detailed plan describing how to develop, maintain, replace and alter or enhance specific software.  Though SDLC uses the term ‘Development’, it does not involve just coding tasks done by developers but also incorporates the tasks contributed by testers and stakeholders.  In SDLC, test cases are created.  ISO/IEC 12207 is an international standard for software life-cycle processes. It aims to be the standard that defines all the tasks required for developing and maintaining software. The following figure is a graphical representation of the various stages of a typical SDLC.

STEP:1 STEP:2

STEP:3

STEP:4

STEP:5

STEP:6

SDLC Models: Various software development life cycle models defined and designed which are followed during the software development process. Following are the most important and popular SDLC models followed in the industry:  Waterfall Model  Iterative Model  Spiral Model  V-Model  Big Bang Model Other related methodologies are Agile Model, RAD Model and Prototyping Models. Waterfall Design:  Waterfall approach was first SDLC Model to be used widely in Software Engineering to ensure success of the project and also is an example for sequential model.  In "The Waterfall" approach, the whole process of software development is divided into separate phases.  In this Waterfall model, typically, the outcome of one phase acts as the input for the next phase sequentially.  Following illustration is a representation of the different phases of the Waterfall Model.

S.No

Phase

Activities Performed

Deliverables

1

Requirement Analysis

1. Capture all the requirements. 2. Do brainstorming and walkthrough to understand the requirements. 3. Do the requirements feasibility test to ensure that the requirements are testable or not.

RUD ( Requirements Understanding Document)

2

System Design

1. As per the requirements, create the design 2. Capture the hardware / software requirements. 3. Document the designs

HLD ( High Level Design document) LLD (Low level design document)

3

Implementation

1. As per the design create the programs / code 2. Integrate the codes for the next phase. 3. Unit testing of the code

Programs Unit test cases and results

4

Testing

1. Integrate the unit tested code and test it to make sure if it works as expected. 2. Perform all the testing activities (Functional and nonfunctional) to make sure that the system meets the requirements. 3. In case of any anomaly, report it. 4. Track your progress on testing through tools like traceability metrics, ALM 5. Report your testing activities.

Test cases Test reports Defect reports Updated matrices.

5

Deployment

1. Make sure that the environment is up 2. Make sure that there are no severe 1 defects open. 3. Make sure that the test exit criteria are met. 4. Deploy the application in the respective environment. 5. Perform a sanity check in the environment after the application is deployed to ensure the application does not break.

User Manual

1. Make sure that the application is up and running in the respective environment. 2. In case user encounters and defect, make sure to note and fix the issues faced. 3. In case any issue is fixed; the updated code is deployed in the environment. 4.The application is always enhanced to incorporate more features, update the environment with the latest features

User Manual

6

Maintenance

Environment definition / specification

List of production tickets List of new features implemented.

When to use SDLC Waterfall Model? SDLC Waterfall model is used when,      

Requirements are very well documented, clear and fixed. Product definition is stable. Technology is understood and is not dynamic. There are no ambiguous requirements. Resources are well trained and available. The project is short.

Pros and Cons of waterfall model: Advantages:  Simple & easy to understand and use.  Works well for smaller projects where requirements are very well understood.  Easy to manage due to the rigidity of the model. Each phase has specific deliverables and a review process.  The entry and exit criteria are well defined, so it easy and systematic to proceed with quality.  Results are well documented at the end of every stage.

Dis-advantages:  Cannot adopt the changes in requirements.  It becomes very difficult to move back to the phase. For example, if the application has now moved to the testing stage and there is a change in requirement, It becomes difficult to go back and change it.  Delivery of the final product is late as there is no prototype which is demonstrated intermediately.  For bigger and complex projects, this model is not good as a risk factor is higher.  Not suitable for the projects where requirements are changed frequently.  Since the testing is done at a later stage, it does not allow identifying the challenges and risks in the earlier phase so the risk mitigation strategy is difficult to prepare.

Conclusion: In the waterfall model, it is very important to take the sign off of the deliverables of each phase. As of today most of the projects are moving with Agile and Prototype models, Waterfall model still holds good for smaller projects. If requirements are straightforward and testable, Waterfall model will yield the best results.

Iterative Model Design:

SPIRAL Model Design:

 Spiral model works in an iterative nature. It is a combination of both Prototype development process and Linear development process (waterfall model).  This model place more emphasis on risk analysis.  Mostly this model adopts to the large and complicated projects where risk is high. Every Iteration starts with a planning and ends with the product evaluation by client. Example: Microsoft released Windows 8 and improved it based on user feedback and released the next version (Windows 8.1).

Spiral Model undergoes 4 phases.    

Planning Phase – Requirement Gathering, Cost Estimation, Resource Allocation. Risk Analysis Phase – Strengths and weaknesses of the project. Design Phase – Coding, Internal Testing and deployment. Evaluation Phase – Client Evaluation (Client side Testing) to get the feedback.

Advantages:    

It allows requirement changes. Suitable for large and complicated projects. It allows better risk analysis. Cost effective due to good risk management. Disadvantages:  Not suitable for small projects.  Success of the project depends on risk analysis phase.  Have to hire more experienced resource especially for risk analysis.

V Model Design: To understand the V model, let’s first understand what is verification and validation in software. Verification: Verification is a static analysis technique. In this technique testing is done without executing the code. Examples include – Reviews, Inspection and walkthrough. Validation: Validation is a dynamic analysis technique where testing is done by executing the code. Examples include functional and non-functional testing techniques.  The V-model is an SDLC model where execution of processes happens in a sequential manner in a V-shape.  It is also known as Verification and Validation model.  The V-Model is an extension of the waterfall model and is based on the association of a testing phase for each corresponding development stage. This means that for every single phase in the development cycle, there is a directly associated testing phase.  This is a highly-disciplined model and the next phase starts only after completion of the previous phase. In V model, the development and QA activities are done simultaneously. There is no discrete phase called Testing, rather testing starts right from the requirement phase. The verification and validation activities go hand in hand. To understand the V model, let’s look at the figure below:

 In a typical development process, the left hand side shows the development activities and right hand side shows the testing activities.  I should not be wrong if I say that in the development phase both verification and validation are performed along with the actual development activities. Now let’s understand the figure:

Left Hand side: As said earlier, left hand side activities are the development activities. Normally we feel, what testing can we do in development phase, but this is the beauty of this model which demonstrates that testing can be done in all phase of development activities as well. Requirement analysis: In this phase the requirements are collected, analysed and studied. Here how the system is implemented, is not important but, what the system is supposed to do, is important. Brain storming sessions/walkthrough, interviews are done to have the objectives clear. Verification activities: Requirements reviews. Validation activities: Creation of UAT (User acceptance test) test cases Artifacts produced: Requirements understanding document, UAT test cases. System requirements / High level design: In this phase a high level design of the software is build. The team studies and investigates on how the requirements could be implemented. The technical feasibility of the requirements is also studied. The team also comes up with the modules that would be created/ dependencies, Hardware / software needs Verification activities: Design reviews Validation activities: Creation of System test plan and cases, Creation of traceability metrics Artifacts produced: System test cases, Feasibility reports, System test plan, Hardware software requirements, and modules to be created etc.

Architectural design: In this phase, based on the high level design, software architecture is created. The modules, their relationships and dependencies, architectural diagrams, database tables, technology details are all finalized in this phase. Verification activities: Design reviews Validation activities: Integration test plan and test cases. Artifacts produced: Design documents, Integration test plan and test cases, Database table designs etc. Module design/ Low level Design: In this phase each and every module or the software components are designed individually. Methods, classes, interfaces, data types etc are all finalized in this phase. Verification activities: Design reviews. Validation activities: Creation and review of unit test cases. Artifacts produced: Unit test cases. Implementation / Code: In this phase, actual coding is done. Verification activities: Code review, test cases review. Validation activities: Creation of functional test cases. Artifacts produced: test cases, review checklist.

Right Hand Side: Right hand side demonstrates the testing activities or the Validation Phase. We will start from bottom. Unit Testing: In this phase all the unit test case, created in the Low level design phase are executed. *Unit testing is a white box testing technique, where a piece of code is written which invokes a method (or any other piece of code) to test whether the code snippet is giving the expected output or not. This testing is basically performed by the development team. In case of any anomaly, defects are logged and tracked. Artifacts produced: Unit test execution results Integration Testing: In this phase the integration test cases are executed which were created in the Architectural design phase. In case of any anomalies, defects are logged and tracked. *Integration Testing: Integration testing is a technique where the unit tested modules are integrated and tested whether the integrated modules are rendering the expected results. In simpler words, it validates whether the components of the application work together as expected. Artifacts produced: Integration test results.

Systems testing: In this phase all the system test cases, functional test cases and non-functional test cases are executed. In other words, the actual and full fledge testing of the application takes place here. Defects are logged and tracked for its closure. Progress reporting is also a major part in this phase. The traceability metrics are updated to check the coverage and risk mitigated. Artifacts produced: Test results, Test logs, defect report, test summary report and updated traceability matrices. User acceptance Testing: Acceptance testing is basically related to the business requirements testing. Here testing is done to validate that the business requirements are met in the user environment. Compatibility testing and sometimes non-functional testing (Load, stress and volume) testing are also done in this phase. Artifacts produced: UAT results, Updated Business coverage matrices.

When to use V Model? / V model is applicable when:    

Requirement is well defined and not ambiguous Acceptance criteria are well defined. Project is short to medium in size. Technology and tools used are not dynamic.

Pros and Cons of using V model:

STLC  Software Testing Life Cycle (STLC) is defined as a sequence of activities conducted to perform Software Testing.  It consists of series of activities carried out methodologically to help certify your software product. Diagram - Different stages in Software Test Life Cycle

Each of these stages have a definite Entry and Exit criteria. Activities & Deliverables associated with it. Entry Criteria: Entry Criteria gives the prerequisite items that must be completed before testing can begin. Exit Criteria: Exit Criteria defines the items that must be completed before testing can be concluded You have Entry and Exit Criteria for all levels in the Software Testing Life Cycle (STLC).

STAGE:1

STAGE-2

STAGE-3

STAGE-4

STAGE-5

STAGE-6

SDLC VS STLC

********************************************************************************