Testcase

FEATURE At a Glance: • Shortening time-to-market consistently, release-afterrelease, will only be achieved by embracing effective quality practices. Here is how one company succeeded!

A Case Study in Extreme Acceptance Testing by James A. Cantor and Liz Derr

Introduction Spurred by business' motivation to decrease time-to-market in order to maximize savings or revenue opportunity, Extreme Programming (XP) approaches have gained significant attention recently. In this quest, without quality practices that keep pace, new releases deliver defects to customers just as quickly! Egreetings was not engaged in XP. I've referred to these practices as "extreme" because they afforded "continuous" acceptance testing, and provided a structure that spurred continuous dialog among customer, developer, and tester. Consistently, developers had both requirements and tests available to them before they began to code. Here's how advanced quality practices evolved and succeeded at Egreetings.com, a top 50 website.

Evolution Began in a Chaotic Environment against Ambitious Business Plans Egreetings' application was based in CGIBIN technology. This legacy application was nearing the limits of its capacity to handle high traffic volume. Its ability to accept modifications and enhancements was almost negligible. To compete, Egreetings had to operate on a more nimble, easily modified application with a more compelling presentation layer. First, the site needed a facelift. Next, gift order processing needed to be installed to enhance revenue. Then an entirely new, rearchitected application duplicating legacy functionality had to be specified, designed, engineered, and launched. The definition of CMM1 level 1 describes organizations that have not yet embarked on process improvement. Their software effort can be characterized as ad-hoc, and sometimes chaotic. In such organizations, few processes are defined, and successful implementations can be attributed to individual effort as well as heroics. Egreetings'

development climate was chaotic… most of the time. To embark on process improvement, a Quality Assurance department was formed. Assumptions implicit in this action were that better testing would yield better quality and impart predictability to launches. There was no specific meaning associated with "better testing". At Egreetings, it did take a QA department to push for better requirements authorship, application deployment, and quality verification practices. The department was formed two weeks before the presentation layer redesign project was to be launched. The project had been under development for the prior two months. After launching the redesigned presentation layer, the company needed to enhance their revenue stream by offering gift-purchasing capabilities. The rearchitecture project commenced at about the same time the gift order-processing project was at full pace.

1 CMM is a product of the Software Engineering Institute (SEI) established by Carnegie Mellon University. It provides a maturity model that describes an organization’s ability to achieve software launch and quality goals predictably. 6

Journal of Software Testing Professionals

http://www.testinginstitute.com

June 2001

Rearchitecture's goal was to provide a site that could sustain daily content and weekly feature releases. So the legacy application was to be enhanced with additional gift functionality while the site was being rearchitected for a later launch. All of the new gift order functionality was to be folded into the rearchitected application. Here is a timeline of the major projects and events (Figure 1). Referring to Figure 1, for major projects in months 1 through 4, Egreeting's capability in getting software defined, designed, developed, tested, and launched could be characterized as entirely "chaotic" until the beginning of the gift-order processing project. The legacy site Redesign Project launched with over 80 undiscovered defects. Two weeks after the Redesign site was launched, feature releases recommenced, and were accomplished every 1-3 weeks. One permanent and three temporary testers performed ad-hoc testing of releases until midway through the 4th month. Subsequent to the Rearchitecture Site launch, feature releases occurred semiweekly (twice per week) or weekly. Content releases occurred daily.

Figure 1 QA during months 1-4 could have been characterized as: • Our engineers did some testing • We used domain experts to run through our software • The whole company checked out the new releases • We used a test check sheet to make sure tests achieved basic functional coverage Acceptance tests were designed intuitively, on the fly, relying on the testers' domain expertise. Company wide testing depended upon the initiative of individual testers during a specifically scheduled test window. And there were no requirements from which to derive test cases.

Acceptance test cases were developed through "Intuitive Decomposition", where test analysts ask the question, "How can I break this?" Good test analysts take pride in their ability to sense where defects will occur. Low-level business functions had not been specifically identified, except in the form of web pages (e.g. login page, profile page). Acceptance test cases were documented based on the author's ability to intuitively perceive the paths through the business function. Each path identified became a test case.

Practices Applied to Single Projects First Figure 3 represents the process flow during the first four months. Classically, we followed waterfall development and testing process. Each successive step was dependent upon the preceding step. At the beginning of the "Gift Order Processing" project in month 5, the large project process matured to a semi-waterfall model, since test cases were authored before coding began, then continuously revised. See figure 4. QA was beginning to exit from the critical path. But "adding-in" the test authorship task was not simple! Both development and test effectiveness depend upon and support good requirements, presenting a

Figure 2 June 2001

http://www.testinginstitute.com

Journal of Software Testing Professionals

7

Software

Testing Web and eBusiness Applications

Testing &

Managing the Software Testing Process Testing Object-Oriented Software

Quality Courses

Software Test Automation and Scripting Techniques Software Inspection & Reviews: A Process-Oriented Approach

by

Practical Techniques for Software Quality Assurance

Leading

Software Requirements Exploration and Definition Software Test Planning and Design

Industry Experts

Building the Software Quality Assurance Funtion Step by Step Requirement-Based Testing

The International Institute for Software Testing offers all its courses at your site to maximize learning and minimize cost to your organization. All courses count towards the Certification of Software Test Professionals (CSTP) For additional information visit: www.testinginstitute.com or call 651-306-1387