Testing
This document was uploaded by user and they confirmed that they have the permission to share it. If you are author or own the copyright of this book, please report to us by using this DMCA report form. Report DMCA
Overview
Download & View Testing as PDF for free.
More details
- Words: 2,438
- Pages: 14
CONTENTS CONTENTS.......................................................................................................................1 1. SOFTWARE TESTING................................................................................................2 1.1 Steps of Software testing:..........................................................................................2 1.2 Two types of testing:..................................................................................................2 1.3 Two Types of testing methods: ..........................................................................................................................................3 1.4 Software Test Models:...........................................................................................3 1.4.1: Waterfall Model.................................................................................................3 1.4.2 Iterative Model:...................................................................................................5 1.4.3 V-Model:.............................................................................................................6 2. BUGS..............................................................................................................................8 2.1 Bug Impacts:..............................................................................................................8 2.2 Bug severity:..............................................................................................................8 2.3 Bug Priority:...............................................................................................................8 2.4 Bug Report components:............................................................................................9 2.5 Bug Tracking System...............................................................................................10 2.5.1 Tracker fields....................................................................................................10 Project - Required field. Tracker is a collection of multiple projects........................10 2.6 BUG LIFE CYCLE:.................................................................................................11 3. TEST PLAN:................................................................................................................12 4. TEST CASE: ...............................................................................................................13 4.1 Test Case Design Techniques:..................................................................................13
1. SOFTWARE TESTING Testing is the process of identifying defects, where a defect is any variance between actual and expected results. The job of the tester is to find defects; the goal of testing project is to reduce the risk not produce perfect software. Software testing, depending on the testing method employed, can be implemented at any time in the development process.
1.1 Steps of Software testing: •
•
•
•
•
•
Understanding Requirements o Business level requirements through project documentation and client interaction o System level requirement through CRS and SRS Test Planning o Test strategy o Test Plan and methodologies o Test environment readiness review Test Definition / Development o Test Procedures o Test scenarios o Test Case o Test Scripts Test Execution (Automated / Manual) o Functional Testing / Module Testing o Regression Testing o System Testing o Performance Testing Test Reporting o Test Summary o Test Reports o Test Metrics Project Sign Off o Client sign off o Project closure report
1.2 Two types of testing: * Static Testing: Reviews, walkthroughs, or inspections are considered as static testing, can be omitted.
2
* Dynamic Testing: Actually executing programmed code with a given set of test cases is referred to as Dynamic Testing.
1.3 Two Types of testing methods: 1. Black Box Testing: It treats the software as a “Black Box” without any knowledge of internal implementation. One of the methods of Black box testing is Traceability Matrix. Traceability Matrix is a document usually in form of table that correlates any two baseline documents that require a many to many relationship to determine 2. White Box Testing: It deals with the internal logic and structure of the code.
1.4 Software Test Models: 1.4.1: Waterfall Model. The model consists of six distinct stages, namely:
3
1.
in the requirements analysis phase
.
The problem is specified along with the desired service objectives (goals)
.
The constraints are identified
2.
In the specification phase the system specification is produced from the detailed definitions of above. This document should clearly define the product function.
3.
In the system and software design phase, the system specifications are translated into a software representation. The software engineer at this stage is concerned with:
4
·
Data structure
·
Software architecture
·
Algorithmic detail and
·
Interface representations The hardware requirements are also determined at this stage along with a picture of the overall system architecture. By the end of this stage the software engineer should be able to identify the relationship between the hardware, software and the associated interfaces. Any faults in the specification should ideally not be passed ‘down stream’
4. In the implementation and testing phase stage the designs are translated into the software domain ·
Detailed documentation from the design phase can significantly reduce the coding effort.
·
Testing at this stage focuses on making sure that any errors are identified and that the software meets its required specification.
5. In the integration and system testing phase all the program units are integrated and tested to ensure that the complete system meets the software requirements. After this stage the software is delivered to the customer. 6. The maintenance phase the usually the longest stage of the software. In this phase the software is updated to: ·
Meet the changing customer needs
·
Adapted to accommodate changes in the external environment
·
Correct errors and oversights previously undetected in the testing phases
1.4.2 Iterative Model:
5
Iterative development is a rework scheduling strategy in which time is set aside to revise and improve parts of the system. It does not presuppose incremental development, but works very well with it. A typical difference is that the output from an increment is not necessarily subject to further refinement, and it’s testing or user feedback is not used as input for revising the plans or specifications of the successive increments.
START
Requirements
Review
Design
Implementation & test
Complete
1.4.3 V-Model:
6
The V-model is a software development process which can be presumed to be the extension of the waterfall model. Instead of moving down in a linear way, the process steps are bent upwards after the coding phase, to form the typical V shape. The VModel demonstrates the relationships between each phase of the development life cycle and its associated phase of testing. The V-model deploys a well-structured method in which each phase can be implemented by the detailed documentation of the previous phase. Testing activities like test designing start at the beginning of the project well before coding and therefore saves a huge amount of the project time.
7
2. BUGS A fault in a program which causes the program to perform in an unintended or unanticipated manner. What is the difference between Error, Bug, Defect and failure? Error: programmatically mistake leads to error. Bug: Deviation from the expected result. Defect: Problem in algorithm leads to failure. Failure: Result of any of the above.
How to write a fully effective Bug report? To write a fully effective report you must: - Explain how to reproduce the problem - Analyze the error so you can describe it in a minimum number of steps. - Write a report that is complete and easy to understand.
2.1 Bug Impacts: • • • •
Low- This is a minor problem. Medium- This is a problem that effects more isolated piece of functionality. High- used for only serious problems, effecting many sites, with no workaround Urgent- complete inability to use the product at almost any site.
2.2 Bug severity: • • • •
Crash- Causes the software to crash Major Bug- Major defect with no work around. Work around- Set when there is a defect but has a work around. Trivial- Not a Major bug * Bug Severity is set by Tester initially and then by the Lead at the verification Level and further by the Developer at fixing time.
2.3 Bug Priority: • • •
Fix ASAP- Fix as soon as possible. Fix Soon- Fix after priority 1 is finished. Fix if time- Fix if time is there otherwise it can be deferred.
8
* Bug Priority is set by the Tester initially and then by the Lead at the verification Level and further by the Developer at fixing time.
2.4 Bug Report components: Report number: Unique number given to a bug. Program / module being tested: The name of a program or module that being tested Version & release number: The version of the product that you are testing. Problem Summary: (data entry field that's one line) precise to what the problem is. Report Type: Describes the type of problem found, for example it could be software or hardware bug. Severity: Normally, how you view the bug. Various levels of severity: Low - Medium - High - Urgent Environment: Environment in which the bug is found. Detailed Description: Detailed description of the bug that is found How to reproduce: Detailed description of how to reproduce the bug. Reported by: The name of person who writes the report. Assigned to developer: The name of developer who assigned to fix the bug. Status: Priority: Assigned by the project manager who asks the programmers to fix bugs in priority order? Resolution: Defines the current status of the problem. There are four types of resolution such as deferred, not a problem, will not fix, and as designed.
9
2.5 Bug Tracking System A bug tracking system helps manage software development projects by tracking software bugs, action items, and change requests with problem reports. It is a software application that is designed to help quality assurance and programmers keep track of reported software bugs in their work. E.g.: BugAware, BugBox etc.
2.5.1 Tracker fields Project - Required field. Tracker is a collection of multiple projects. Issue Type - Required field. Bugs are classified as one of the following: Bug, Improvement, New Feature, Task, Sub task. Summary - Required field. A brief, concise description of the problem. Security Level - The higher the security level, the fewer people who can view the issue. Priority - Bugs are prioritized using one of the following: Blocker, critical, major, minor, trivial.
Component/s - Required field. Select the area in Moodle which is affected by this bug. Select 'Unknown' if you are unsure. Affects Version/s - Required field. This is the Moodle version in which the bug has been found. It is entered by the person logging the bug, and typically only 1 version is specified.
Assigned To - This is the person who will fix (code) the issue. Tracker automatically assigns issues to Component Leads. Developers or QA Testers can reassign issues. Reporter - The person who logs the bug. This field is automatically filled by Tracker. Environment - Specify the operating system, software platform and/or hardware specifications if applicable to this bug. Description - A full and complete yet concise description of the problem or improvement. Database - Optional field. If applicable to the bug, identify the database type. URL - Optional field. If possible, provide a URL address that demonstrates an example of this bug. QA Assignee - Used to designate the person who will test this issue.
10
Fixed Version/s - This is the Moodle version the bug was or will be fixed in. Attachment - Optional. Attach a file that will help developers and testers better understand the bug. Maximum attachment size is 512Kb. Comment - The comment field is a detailed register of all changes that relate to this bug. Resolution - This field is only displayed when resolving or closing a bug. Specify a code that best describes how this bug was resolved.
How to report a bug, improvement, or new feature request Login to Tracker Select "Create New Issue" from the menu under the Moodle Tracker logo From the dropdown menu select the "Issue type": Bug, New Feature, Task or Improvement You will see a series of dropdown and free text fields. Complete as many as you can. Some fields are required while others are optional. Press the 'Create' button at the bottom of the page to create the bug.
2.6 BUG LIFE CYCLE: NEW
OPEN
REJECTED
ASSIGN
REOPENED
TEST
DEFERRED
VERIFIED CLOSED 1. NEW: When the Bug is posted for the first time, its state will be “NEW”. This means that the Bug is not yet approved.
11
2. OPEN: After a Tester has posted a Bug, the lead of the tester approves that the bug is genuine & he changes the state as “OPEN”. 3. ASSIGN: Once the lead changes the state as “OPEN”, he assigns the Bug to the corresponding developer or developer team. The state of the bug is now changed to “ASSIGN”. 4. TEST: Once the developer fixes the bug he has to assign the bug to the testing team for next round of testing. Now the state of the bug is changed to “TEST”. It specifies that the bug has been fixed & is released to the testing team. 5. DEFERRED: It means that the bug is expected to be fixed in next releases. The state of the Bug is changed to “DEFERRED”. 6. REJECTED: If the developer feels hat the Bug is not genuine, he rejects the Bug. Then the state of the bug is changed to “REJECTED”. 7. DUPLICATE: If the bug is repeated twice or the two bugs mention the same concept of the bug, then one bug status is changed to “DUPLICATE”. 8. VERIFIED: Once the bug is fixed & the status is changed to “TEST”, the tester tests the bug. If the bug is not present in the Software, he approves that the bug is fixed& changes the status to “VERIFIED”. 9. REOPENED: If the Bug still exists even after the bug is fixed by the developer, the tester changes the status to “REOPENED”. The bug traverses the life cycle again. 10. CLOSED: Once the bug is fixed, it is tested by the tester. If the tester feels that the Bug no longer exists in the Software, he changes the status of the bug to “CLOSED”. This means that the Bug is fixed, tested and approved.
3. TEST PLAN: A test plan is a systematic approach to testing a systemtest plan consists of • Introduction • Objective • Scope • Features tested and not be tested
12
• • • • •
Acceptance criteria Suspension criteria Testing approach Tool to be used Risks and contingencies
4. TEST CASE: A test case in software engineering is a set of conditions or variables under which a tester will determine whether an application or software system is working correctly or not.
A test case is usually a single step, or occasionally a sequence of steps, to test the correct behavior/functionalities, features of an application. An expected result or expected outcome is usually given. Additional information that may be included: • • • • • • • •
test case ID test case description test step or order of execution number related requirement(s) depth test category author Check boxes for whether the test is automatable and has been automated.
Additional fields that may be included and completed when the tests are executed: • •
pass/fail remarks
4.1 Test Case Design Techniques: •
Requirements Analysis
13
The most common method for creating test cases is to analyze the functional specification or requirements and create test cases that specifically prove or attempt to disprove that the application meets those requirements.
•
Equivalence Partitioning
An equivalence partition (or equivalence class) is series of values where the expected outcome is the same, and therefore using 1 value will test the expected outcome as well as 10 values would from that particular group.
•
Error Guessing
Error Guessing is where the experience of the test analyst is used to anticipate where errors may occur and create test cases to test this area of the application.
•
Ad Hoc
Ad Hoc tests are tests performed where no test case design technique is used. A number of Ad Hoc tests are performed during the testing cycle and an effort should be made to record these tests as they provide vital information.
14
1. SOFTWARE TESTING Testing is the process of identifying defects, where a defect is any variance between actual and expected results. The job of the tester is to find defects; the goal of testing project is to reduce the risk not produce perfect software. Software testing, depending on the testing method employed, can be implemented at any time in the development process.
1.1 Steps of Software testing: •
•
•
•
•
•
Understanding Requirements o Business level requirements through project documentation and client interaction o System level requirement through CRS and SRS Test Planning o Test strategy o Test Plan and methodologies o Test environment readiness review Test Definition / Development o Test Procedures o Test scenarios o Test Case o Test Scripts Test Execution (Automated / Manual) o Functional Testing / Module Testing o Regression Testing o System Testing o Performance Testing Test Reporting o Test Summary o Test Reports o Test Metrics Project Sign Off o Client sign off o Project closure report
1.2 Two types of testing: * Static Testing: Reviews, walkthroughs, or inspections are considered as static testing, can be omitted.
2
* Dynamic Testing: Actually executing programmed code with a given set of test cases is referred to as Dynamic Testing.
1.3 Two Types of testing methods: 1. Black Box Testing: It treats the software as a “Black Box” without any knowledge of internal implementation. One of the methods of Black box testing is Traceability Matrix. Traceability Matrix is a document usually in form of table that correlates any two baseline documents that require a many to many relationship to determine 2. White Box Testing: It deals with the internal logic and structure of the code.
1.4 Software Test Models: 1.4.1: Waterfall Model. The model consists of six distinct stages, namely:
3
1.
in the requirements analysis phase
.
The problem is specified along with the desired service objectives (goals)
.
The constraints are identified
2.
In the specification phase the system specification is produced from the detailed definitions of above. This document should clearly define the product function.
3.
In the system and software design phase, the system specifications are translated into a software representation. The software engineer at this stage is concerned with:
4
·
Data structure
·
Software architecture
·
Algorithmic detail and
·
Interface representations The hardware requirements are also determined at this stage along with a picture of the overall system architecture. By the end of this stage the software engineer should be able to identify the relationship between the hardware, software and the associated interfaces. Any faults in the specification should ideally not be passed ‘down stream’
4. In the implementation and testing phase stage the designs are translated into the software domain ·
Detailed documentation from the design phase can significantly reduce the coding effort.
·
Testing at this stage focuses on making sure that any errors are identified and that the software meets its required specification.
5. In the integration and system testing phase all the program units are integrated and tested to ensure that the complete system meets the software requirements. After this stage the software is delivered to the customer. 6. The maintenance phase the usually the longest stage of the software. In this phase the software is updated to: ·
Meet the changing customer needs
·
Adapted to accommodate changes in the external environment
·
Correct errors and oversights previously undetected in the testing phases
1.4.2 Iterative Model:
5
Iterative development is a rework scheduling strategy in which time is set aside to revise and improve parts of the system. It does not presuppose incremental development, but works very well with it. A typical difference is that the output from an increment is not necessarily subject to further refinement, and it’s testing or user feedback is not used as input for revising the plans or specifications of the successive increments.
START
Requirements
Review
Design
Implementation & test
Complete
1.4.3 V-Model:
6
The V-model is a software development process which can be presumed to be the extension of the waterfall model. Instead of moving down in a linear way, the process steps are bent upwards after the coding phase, to form the typical V shape. The VModel demonstrates the relationships between each phase of the development life cycle and its associated phase of testing. The V-model deploys a well-structured method in which each phase can be implemented by the detailed documentation of the previous phase. Testing activities like test designing start at the beginning of the project well before coding and therefore saves a huge amount of the project time.
7
2. BUGS A fault in a program which causes the program to perform in an unintended or unanticipated manner. What is the difference between Error, Bug, Defect and failure? Error: programmatically mistake leads to error. Bug: Deviation from the expected result. Defect: Problem in algorithm leads to failure. Failure: Result of any of the above.
How to write a fully effective Bug report? To write a fully effective report you must: - Explain how to reproduce the problem - Analyze the error so you can describe it in a minimum number of steps. - Write a report that is complete and easy to understand.
2.1 Bug Impacts: • • • •
Low- This is a minor problem. Medium- This is a problem that effects more isolated piece of functionality. High- used for only serious problems, effecting many sites, with no workaround Urgent- complete inability to use the product at almost any site.
2.2 Bug severity: • • • •
Crash- Causes the software to crash Major Bug- Major defect with no work around. Work around- Set when there is a defect but has a work around. Trivial- Not a Major bug * Bug Severity is set by Tester initially and then by the Lead at the verification Level and further by the Developer at fixing time.
2.3 Bug Priority: • • •
Fix ASAP- Fix as soon as possible. Fix Soon- Fix after priority 1 is finished. Fix if time- Fix if time is there otherwise it can be deferred.
8
* Bug Priority is set by the Tester initially and then by the Lead at the verification Level and further by the Developer at fixing time.
2.4 Bug Report components: Report number: Unique number given to a bug. Program / module being tested: The name of a program or module that being tested Version & release number: The version of the product that you are testing. Problem Summary: (data entry field that's one line) precise to what the problem is. Report Type: Describes the type of problem found, for example it could be software or hardware bug. Severity: Normally, how you view the bug. Various levels of severity: Low - Medium - High - Urgent Environment: Environment in which the bug is found. Detailed Description: Detailed description of the bug that is found How to reproduce: Detailed description of how to reproduce the bug. Reported by: The name of person who writes the report. Assigned to developer: The name of developer who assigned to fix the bug. Status: Priority: Assigned by the project manager who asks the programmers to fix bugs in priority order? Resolution: Defines the current status of the problem. There are four types of resolution such as deferred, not a problem, will not fix, and as designed.
9
2.5 Bug Tracking System A bug tracking system helps manage software development projects by tracking software bugs, action items, and change requests with problem reports. It is a software application that is designed to help quality assurance and programmers keep track of reported software bugs in their work. E.g.: BugAware, BugBox etc.
2.5.1 Tracker fields Project - Required field. Tracker is a collection of multiple projects. Issue Type - Required field. Bugs are classified as one of the following: Bug, Improvement, New Feature, Task, Sub task. Summary - Required field. A brief, concise description of the problem. Security Level - The higher the security level, the fewer people who can view the issue. Priority - Bugs are prioritized using one of the following: Blocker, critical, major, minor, trivial.
Component/s - Required field. Select the area in Moodle which is affected by this bug. Select 'Unknown' if you are unsure. Affects Version/s - Required field. This is the Moodle version in which the bug has been found. It is entered by the person logging the bug, and typically only 1 version is specified.
Assigned To - This is the person who will fix (code) the issue. Tracker automatically assigns issues to Component Leads. Developers or QA Testers can reassign issues. Reporter - The person who logs the bug. This field is automatically filled by Tracker. Environment - Specify the operating system, software platform and/or hardware specifications if applicable to this bug. Description - A full and complete yet concise description of the problem or improvement. Database - Optional field. If applicable to the bug, identify the database type. URL - Optional field. If possible, provide a URL address that demonstrates an example of this bug. QA Assignee - Used to designate the person who will test this issue.
10
Fixed Version/s - This is the Moodle version the bug was or will be fixed in. Attachment - Optional. Attach a file that will help developers and testers better understand the bug. Maximum attachment size is 512Kb. Comment - The comment field is a detailed register of all changes that relate to this bug. Resolution - This field is only displayed when resolving or closing a bug. Specify a code that best describes how this bug was resolved.
How to report a bug, improvement, or new feature request Login to Tracker Select "Create New Issue" from the menu under the Moodle Tracker logo From the dropdown menu select the "Issue type": Bug, New Feature, Task or Improvement You will see a series of dropdown and free text fields. Complete as many as you can. Some fields are required while others are optional. Press the 'Create' button at the bottom of the page to create the bug.
2.6 BUG LIFE CYCLE: NEW
OPEN
REJECTED
ASSIGN
REOPENED
TEST
DEFERRED
VERIFIED CLOSED 1. NEW: When the Bug is posted for the first time, its state will be “NEW”. This means that the Bug is not yet approved.
11
2. OPEN: After a Tester has posted a Bug, the lead of the tester approves that the bug is genuine & he changes the state as “OPEN”. 3. ASSIGN: Once the lead changes the state as “OPEN”, he assigns the Bug to the corresponding developer or developer team. The state of the bug is now changed to “ASSIGN”. 4. TEST: Once the developer fixes the bug he has to assign the bug to the testing team for next round of testing. Now the state of the bug is changed to “TEST”. It specifies that the bug has been fixed & is released to the testing team. 5. DEFERRED: It means that the bug is expected to be fixed in next releases. The state of the Bug is changed to “DEFERRED”. 6. REJECTED: If the developer feels hat the Bug is not genuine, he rejects the Bug. Then the state of the bug is changed to “REJECTED”. 7. DUPLICATE: If the bug is repeated twice or the two bugs mention the same concept of the bug, then one bug status is changed to “DUPLICATE”. 8. VERIFIED: Once the bug is fixed & the status is changed to “TEST”, the tester tests the bug. If the bug is not present in the Software, he approves that the bug is fixed& changes the status to “VERIFIED”. 9. REOPENED: If the Bug still exists even after the bug is fixed by the developer, the tester changes the status to “REOPENED”. The bug traverses the life cycle again. 10. CLOSED: Once the bug is fixed, it is tested by the tester. If the tester feels that the Bug no longer exists in the Software, he changes the status of the bug to “CLOSED”. This means that the Bug is fixed, tested and approved.
3. TEST PLAN: A test plan is a systematic approach to testing a systemtest plan consists of • Introduction • Objective • Scope • Features tested and not be tested
12
• • • • •
Acceptance criteria Suspension criteria Testing approach Tool to be used Risks and contingencies
4. TEST CASE: A test case in software engineering is a set of conditions or variables under which a tester will determine whether an application or software system is working correctly or not.
A test case is usually a single step, or occasionally a sequence of steps, to test the correct behavior/functionalities, features of an application. An expected result or expected outcome is usually given. Additional information that may be included: • • • • • • • •
test case ID test case description test step or order of execution number related requirement(s) depth test category author Check boxes for whether the test is automatable and has been automated.
Additional fields that may be included and completed when the tests are executed: • •
pass/fail remarks
4.1 Test Case Design Techniques: •
Requirements Analysis
13
The most common method for creating test cases is to analyze the functional specification or requirements and create test cases that specifically prove or attempt to disprove that the application meets those requirements.
•
Equivalence Partitioning
An equivalence partition (or equivalence class) is series of values where the expected outcome is the same, and therefore using 1 value will test the expected outcome as well as 10 values would from that particular group.
•
Error Guessing
Error Guessing is where the experience of the test analyst is used to anticipate where errors may occur and create test cases to test this area of the application.
•
Ad Hoc
Ad Hoc tests are tests performed where no test case design technique is used. A number of Ad Hoc tests are performed during the testing cycle and an effort should be made to record these tests as they provide vital information.
14
