Testing Processes & Methodology-old

Testing Processes & Methodology

Fundamentals

1

Testing Concepts Introduction is necessary to understand any concept better. Study on Testing is necessary before we use or practice the process in an Organization or Daily use. For this, we need conceptual level of knowledge of Testing like…. History of Testing Why testing is necessary? Introduction of Testing Where / when we use Testing? How we do testing or define Testing Process? Lets go in detail……..

2

History  The separation of debugging from testing was initially introduced by Glenford J. Myers in 1979. Drs. Dave Gelperin and William C. Hetzel classified in 1988 the phases and goals in software testing  There was no clear difference between testing and debugging until 1956. From 1957-1978 there was the demonstration oriented period where debugging and testing was distinguished now - in this period it was shown, that software satisfies the requirements.  The time between 1979-1982 is announced as the destruction oriented period, where the goal was to find errors.  From 1988 on it was seen as prevention oriented period where tests were to demonstrate that software satisfies its specification, to detect faults and to prevent faults.  Dr. Gelperin chaired the IEEE 829-1989 (Test Documentation Standard). Dr. Gelperin and Jerry E. Durant also went on to develop High Impact Inspection Technology that builds upon traditional Inspections but utilizes a test driven additive. 3

Why testing is Important??  The Software Crisis dates back to the 1960’s when the primary reasons for this situation were less than acceptable software engineering practices.  In the early stages of software, a lot of code written but no established standards. Then in early 70’s a lot of computer programs started failing and people lost confidence and thus an industry crisis was declared.  This crisis though identified in the early years, exists to date and we have examples of software failures around the world.  The primary reason for these failures mentioned above is due to bad software engineering practices adopted.  Some of the worst software practices include  No historical software-measurement data  Rejection of accurate cost estimates.  Failure to use automated estimating and planning tools.  Excessive, irrational schedule pressure and creep in user requirements.  Failure to monitor progress and to perform risk management.  Failure to use design reviews and code inspections. 4

Why testing is Important??  To avoid these failures and thus improve the record, what is needed is a better understanding of the process, better estimation techniques for cost time and quality measures. But the question is, what is a process? Process transform inputs to outputs i.e. a product. A software process is a set of activities, methods and practices involving transformation that people use to develop and maintain software.  To deliver successful software projects, a focus on the process is essential, and improvements in the existing system. This focus would help in the predictability of outcomes, project trends, and project characteristics.  Process management is concerned with the knowledge and management of the software process, its technical aspects and also ensures that the processes are being followed as expected and improvements are shown.  Correct identification would then make it easier to identify the best practices that can be applied because one process that might be suitable for one organization may not be most suitable for another.  Therefore to make a successful product a combination of Process and Technicalities will be required under the umbrella of a well-defined process.

5

Why testing is Important??  It is important to identify and relate the role software testing plays not only in producing quality software but also maneuvering the overall process.  The computer society defines testing as follows: “Testing -- A verification method that applies a controlled set of conditions and stimuli for the purpose of finding errors. This is the most desirable method of verifying the functional and performance requirements. Test results are documented proof that requirements were met and can be repeated. The resulting data can be reviewed by all concerned for confirmation of capabilities.”

6

Why do we Test?? Here below listed are some of the reasons for why testing is necessary :         

To build confidence in the software under test prove that the software is correct demonstrate conformance to requirements find faults reduce costs show system meets user needs assess the software quality Testing and Risk How much testing is enough

However, prove that the software is correct is not a valid point, because it is not possible to prove that system has no faults. It is only possible to prove that the system has faults – by finding some of them.

7

Why do we Test?? Some Questions which may arise are: Can we Prove software is Correct? What find faults exactly mean? Do we can say that Build Confidence means no Faults? One more controversy we come across is the fewer faults we find, the better the quality of the software? finding few or no faults can mean one of three things: good software, poor testing or both poor testing and poor software. Without knowing independently about the quality of the testing, no justified conclusions can be drawn about the quality of the software.

8

Introduction What is Software Testing? Software testing is more than just error detection; Testing software is operating the software under controlled conditions, to (1) verify that it behaves “as specified”; (2) to detect errors, and (3) to validate that what has been specified is what the user actually wanted. 

Verification : is the checking or testing of items, including software, for conformance and consistency by evaluating the results against prespecified requirements.  [Verification: Are we building the system right?]



Error Detection: Testing should intentionally attempt to make things go wrong to determine if things happen when they shouldn’t or things don’t happen when they should.



Validation : looks at the system correctness – i.e. is the process of checking that what has been specified is what the user actually wanted.  [Validation: Are we building the right system?]

Contd… 9

Introduction In other words, validation checks to see if we are building what the customer wants/needs, and verification checks to see if we are building that system correctly. Both verification and validation are necessary, but different components of any testing activity. The definition of testing according to the ANSI/IEEE 1059 standard is that testing is the process of analyzing a software item to detect the differences between existing and required conditions (that is defects/errors/bugs) and to evaluate the features of the software item. Remember: The purpose of testing is verification, validation and error detection in order to find problems – and the purpose of finding those problems is to get them fixed.

10

When Testing should occur??  Testing should be performed at every development stage of the product.  Test data sets must be derived and their correctness and consistency should be monitored throughout the development process.  If we divide the lifecycle of software development into “Requirements Analysis”, “Design”, “Programming/Construction” and “Operation and Maintenance”, then testing should accompany each of the above phases.  If testing is isolated as a single phase late in the cycle, errors in the problem statement or design may incur exorbitant costs. Not only must the original error be corrected, but the entire structure built upon it must also be changed.  Therefore, testing should not be isolated as an inspection activity. Rather testing should be involved throughout the SDLC in order to bring out a quality product.

11

Testing Activities in diff. Phases The Fundamental Test Process comprises five activities: Planning, Specification, Execution, Recording, and Checking for Test Completion. Planning : All good testing is based upon good test planning. There should already be an overall test strategy. This Test Planning activity produces a test plan specific to a level of testing. When producing a test plan, clearly define the scope of the testing and state all the assumptions being made. Identify any other software required before testing can commence (e.g. stubs & drivers, word processor, spreadsheet package or other 3rd party software) and state the completion criteria to be used to determine when this level of testing is complete. Specification : The fundamental test process describes this activity as designing the test cases using the techniques selected during planning. For each test case, specify its objective, the initial state of the software, the input sequence and the expected outcome. Specification can be considered as three separate tasks: Identify test conditions — determines ‘what’ is to be tested; Design test cases - determine ‘how’ the ‘what’s’(test conditions) are going to be exercised; Build test cases - implementation of the test cases (scripts, data, etc.).

12

Testing Activities in diff. Phases Execution : The purpose of this activity is to execute all of the test cases.This can be done either manually or with the use of a test execution automation tool. The order in which the test cases are executed is significant. The most important test cases should be executed first. In general, the most important test cases are the ones that are most likely to find the most serious faults but may also be those that concentrate on the most important parts of the system. If too many faults are found by the first few tests we may decide that it is not worth executing the rest of them. In practice time pressures may mean that there is time to execute only a subset of the specified test cases. In this case it is particularly important to have prioritised the test cases to ensure that at least the most important ones are executed. Recording : In practice the Test Recording activity is done in parallel with Test Execution. To start with we need to record the versions of the software under test and the test specification being used. Then for each test case we should record the actual outcome and the test coverage levels achieved for those measures specified as test completion criteria in the test plan.  

13

Testing Activities in diff. Phases The actual outcome should be compared against the expected outcome and any discrepancy found logged and analysed in order to establish where the fault lies. It may be that the test case was not executed correctly in which case it should be repeated. The fault may lie in the environment set-up or be the result of using the wrong version of software under test. In these cases the fault should be fixed and the test case executed again.  The records made should be detailed enough to provide an unambiguous account of the testing carried out. They may be used to establish that the testing was carried out according to the plan. Check completion : This activity has the purpose of checking the records against the completion criteria specified in the test plan. If these criteria are not met, it will be necessary to go back to the specification stage to specify more test cases to meet the completion criteria. There are many different types of coverage measure and different coverage measures apply to different levels of testing.

14

Testing Approach A traditional testing approach Show that the system: -does what it should -doesn't do what it shouldn't But a better testing approach Show that the system: -does what it shouldn’t -doesn't do what it should

15

What type of testing to be done???? Mainly Testing is

16