Defining A Testing Strategy For A Practical Priorities In System Testing

http://www.kualitatem.com Thinking ahead: Defining a testing strategy for a Practical Priorities in System Testing Software Engineering I Research Paper Abstract . This paper is about the methods of System Testing, which will involves the design and engineering of the system to provide a functional, performance, security, and usability testing services. Also, this paper discusses how these tools are in the software environment.

Table 1. 1.1 1.2 2. 3. 3.1 3.2 4. 4.1 4.2

of Contents Introduction 2 Information Goal 2 Purpose 3 Method of Approach 5 Results and Conclusions 5 Results 5 Conclusions 5 References and Bibliography References 9 Bibliography 10

9

Introduction System testing accounts for 30 to 50 percent of software development costs and the percentage is increasing. Organizations are seeing the costs explode due to large testing staffs, long schedules, and most damaging of all, missing opportunities. The costs of not testing are also exploding. Users will not tolerate down time, processing errors, bad performance, and difficult to use software. In spite of these escalating costs, software is still routinely fielded with defects that range in severity from mere annoyances to the user to total disasters. The results often show up headlines and can cause embarrassment and ruin to companies who have produced the faulty software. It seems that companies are spending a lot on software testing without a clear return on their investment. This contradiction is rooted in two clear facts about software testing. • First, testing software enormously difficult and increasing complexity for software designs will get more so. • Seconding, testing is typically done without a clear methodology and without the required automation support. In addition many test organizations that have been using methodology for mainframe development now find that it is obsolete for new software environment. [IDOL02] o Information Goal System testing faces new challenges with the changing technology used for developing and delivering software. System help solve complex testing issue related to interaction, concurrency, and time/fault tolerance by addressing the functional, robustness, load, performance and regression testing phases from small, single threads or tasks up to very large, distributed system [HELM02]. In other words, system testing verifies that all elements mesh properly and that overall system functions / performance is achieved. Project management veteran Tom Mochal is director of internal development at a software company in Atlanta and author of “System test: Can your program take a lickin’ and keep on tickin”? says that there is no specific “system test”. Mr

Mocal says but there is a variety of test fall under the general system test umbrella. Some of these tests can be difficult to set up and execute because they require you to simulate the production environment. If a company has the hardware and software that required simulating the live environment, it can be difficult to stage the proper set of people, transaction, and circumstance you see in production [MOCH01]. For instance, although a web application works fine and meet the business requirements can it resist the attack of a hacker who wants to read your database? That event will be hard to test for. Companies that have made the press because of major system failures within the system, typically are not brought on by code errors or logic problems. The problems are caused by poor response times, weak security, lost data, screens people cannot understand, and so forth. These are all aspects that the system may address [PRESS97]. There are several types of system test and test strategy. This paper will address all aspect of system testing and the use of test strategy on a project to avoid processing errors, bad performance, and difficult to use software. o Purpose The purpose is to addressed the aspects of system testing and to define testing strategy (overall context for the entire test process) too avoid a major errors, bad performance and difficult to use software application. System testing is a collection of tests designed to verify that a program or system of programs is ready for production. There is a need for system testing to determine whether the application is at the proper level of industrial strength and ready for the primetime production environment. There are several types of system tests: • Performance: Test the application with transaction volume that is within and at top of the range expected from the business requirement. Make sure that the resulting system behavior is within expectations or any formal service level agreement. • Stress: Expose the system to transaction volume substantially higher than what would normally be expected and over a concentrated time period. • Security: Ensure that people have the access level that is required and deny access to the system to unauthorized people. • Requirements: Track each business requirement through the development process and make sure that it is include in the final system. • Usability: Determine that people can use the system easily and without frustration. • Documentation: Check that hard copy and online documentation are understandable and accurate. • Training: Ensure that online or in person training is effective and meets the training requirements. • Interface: Test your application interfaces with external databases or third-party companies. • Disaster recovery: See whether you can recover the system from a simulated disaster. • Multiple locations: Verify your system can function between multiple locations, if necessary. Not all of types of system tests are necessary or related to an IT system. It is important to know that all of the system tests start with assumption that the system is complete and stable. It is difficult to do a system testing if the person who is running into a large number of program bugs as well. [MOCH01]. Method of Approach Of all the many aspects of System testing: performance testing, stress testing, recovery testing, and security testing are four (4) areas that most developers think about when they plan the system test. [MOCT02] 1. Performance testing. The development team must understand the level of

transaction volume the system needs to handle. For real-time and embedded systems, functional requirements may be satisfied but performance problems make the unacceptable. Performance testing checks the run-time performance in the context of the integrated system. [SOBE97]. The areas that you look for include: • Online response time for screen navigation. • Online response time for processing data against databases. • The time needed to run batch processes. • Delivery time for hard-copy reports (e.g., are they ready by 8:00 A.M.?). • Printer capacity to handle requests. • Downtime for batch processing or periodic maintenance, • File sizes and storage capacity. 2. Stress testing. Stress testing is designed to test the software with abnormal situations. Stress testing attempts to find the limits at which the system will fail through abnormal quantity or frequency of inputs. [SOBE97] For example: • High rate of interrupts • Data rates an order of magnitude above ‘normal’ • Test cases that require maximum memory or other resources. • Test case that cause ‘thrashing’ in a virtual operating system. • Test case that cause excessive ‘hunting’ for data on the disk systems. Can also attempt sensitivity testing to determine if particular combination of otherwise normal inputs can cause improper processing. 3. Recovery Testing. Many computer-based systems must recover from faults and resume processing within a prespecified time. In some cases, a system must be fault tolerant; that is, processing faults must not cause overall system function to cease. In other case, a system failure must be corrected with a specific period of time or severe economic damage will occur. [PRES97] 4. Security Testing. Systems with sensitive information or which have potential to harm individuals can be a target for improper or illegal use. This include: • Attempted penetration of the system by ‘outside’ individuals for fun or personal gain. • Disgruntled or dishonest employee. During security testing the tester plays a role of the individual trying to penetrate the system. [SOBE97] Large range of methods: • Attempt to acquire passwords through external clerical means • Use custom software to attack the system • Overwhelm the system with respects • Cause the system errors and attempt to penetrate the system during recovery • Browse through insecure data. Results and Conclusion o Result Most developer think about the four (4) important areas that developers of System testing, which include performance testing, stress testing, recovery testing, and security testing for they plan the system test. [MOCT02] Most developer has been on a development project where the testing process was not considered until the programming was already in progress. With this in mind, the developer end up trying to plan the test process while they are in a testing process. The test environments are probably not set up properly and it’s unclear whom responsible. This situation can be avoided by thinking ahead and determining your testing strategy before the coding process begins. When focusing on a test process and its life cycle, it focused mainly on the basic

principles that needed to plan the test process early in the development life cycle. The testing process starts in the analysis phase, with the formulation of the Testing Strategy, which is later translated into the lower-level Test Plan for a larger project. During the analysis phase, you gather and validate the business requirements for the solution. It makes sense that the Test Strategy is completed during this phase as well. In a sense, you are defining the overall testing requirements. The purpose of the Testing Strategy is define the overall context for the entire test process. The process is different depending on the specific characteristic of your solution. The characteristic is the most important part of the test process, since all future testing decisions will be made within the context for the strategy. [MOCH01]. Here are the basic parts of the testing strategy: • Project Overview: You can copy this from the Project Definition. • Business Risks: These are high-level risks of the project that will affect the overall testing. The risk can be classified as high, medium, and low, depending on the nature and impact of the problem. For instance, the risk of doing business on the Internet may drive the need for rigorous system tests of firewalls, technical architecture, and security. • Test Milestones: This section gives the reader a preliminary overview of the testing timelines. • Test Approach: This describes the test process at a high level, including how you will conduct uniting test, integration testing, system testing, and acceptance testing. (If the project is large enough, each of these might be its own section.) This is where you make fundamental decisions regarding the type of testing that makes sense for your project: 1. If you are implementing a packaged solution, the approach may start in system test, with vendor providing close support. 2. If you are doing iterative development cycles, the testing approach will reflect thus overall development life cycle. 3. For system testing, define the major testing events, such as stress testing, security testing, disaster recovery testing, usability testing, and response time testing. • Testing Environment: Think through the technologies and facilities needed for the testing process. If the overall testing environment needs are understood up front, it will be easier to break out the specific activities required putting the environment in place. In addition, plan for and acquire some parts of the environment well advance. There may be other high-level section, depending on the project, such as testing objectives, test assumptions, testing organization, and testing tools, along with effort and cost estimates. [MOCH01] o

Conclusion

When thinking ahead by preparing a Testing Strategy remembers that it is a foundation of successfully testing. When preparing a Testing Strategy it is important to keep these three points in mind: • Formulate an overall Testing Strategy during the analysis phase. An overall approach to testing and describes how the test process will ensure that the solution has the appropriate level of quality and reliability. • Provides the overall guidelines from which all future testing decisions are made. It allows the rest of the testing process to be defined more effectively. • The Testing Strategy needs to be understood and approved by the sponsor. If the strategy is accepted, there is a much better likelihood that the final solution will meet the customer’s expectations. Companies that have made the press because of major system failures within the system typically are not brought on by code errors or logic problems. If a test strategy were conducted properly problems that caused by poor response times, weak security, lost data, screens people cannot understand, and so forth would have

been prevented. The results can be successful and not cause embarrassment and ruin companies who produced the fault software application. References and Bibliography o

References

[HELM02] Helm, James C., System Testing with Rational Test RealTime Comprehensive Test Automation of Message-Based Systems. November 2002. Internet URL: http://www.rational.com/products/testrt/sys_test.jsp? [IDOL02] Idol, Chuck, Automated Testing Technology. November 2002. Internet URL: http://www.longislandbuilders.com/ATT.htm [PRES97] Pressman, Roger S., Software Engineering: A Practitioner Approach. 4th ed., New York: McGraw-Hill Companies, Inc., 1997. [MOCH02] Mochal, Tom, System testing: Can your program take a lickin’ and keep on tickin’? September 18, 2001. Internet URL: http://www.builder.com [MOCT02] Mochal, Tom, System testing to check security and validate system requirements. September 25, 2001. Internet URL: http://www.builder.com [SOBE97] Sobey, A. J., Software Engineering 1A/1B/1M. June 8, 1997. Internet URL: http://www.louisa.level.unisa.edu.au/se1/testing-notes/test02_5.htm [MOCH01] Mochal, Tom, Defining your testing strategy Aug. 25, 2001. Internet URL: http://www.builder.com o

Bibliography

James C. Helm, System Testing with Rational Test RealTime Comprehensive Test Automation of Message-Based Systems. November 2002. Internet URL: http://www.rational.com/products/testrt/sys_test.jsp? Chuck Idol Automated Testing Technology. November 2002. Internet URL: http://www.longislandbuilders.com/ATT.htm Roger S. Pressman, Ph.D., Software Engineering: A Practitioner Approach. 4th ed., New York: McGraw-Hill Companies, Inc., 1997. Tom Mochal, System testing: Can your program take a lickin’ and keep on tickin’? September 18, 2001. Internet URL: http://www.builder.com Tom Mochal, System testing to check security and validate system requirements. September 25, 2001. Internet URL: http://www.builder.com Dr. A. J. Sobey, Software Engineering 1A/1B/1M. June 8, 1997. Internet URL: http://www.louisa.level.unisa.edu.au/se1/testing-notes/test02_5.htm Tom Mochal, Defining your testing strategy Aug. 25, 2001. Internet URL: http://www.builder.com