Test Case And Use Cases

Written By Nishikant Banchhor Software Engineer Introduction Use cases are a popular way to express software requirements. They are popular because they are practical. A use case bridges the gap between user needs and system functionality by directly stating the user intention and system response for each step in a particular interaction. Use cases are simple enough that almost anyone can read them. Even customers or users can read use cases without any special training. However, writing use cases takes some practice. It is not difficult to start writing use cases, but really mastering them takes training, practice, and insight. No single use case specifies the entire requirements of the system. Each use case merely explains one particular interaction. An organized suite of use cases and other specification techniques are needed to fully specify the software requirements. The figure below illustrates where a use case document fits into the overall software requirements specification (SRS) and how it relates to other documents. This white paper focuses on the yellow "Use Cases" box. Ideally, your use case document is just one part of an overall set of project documents. But, don't worry if you just want to jump straight into writing use cases: this white paper focuses on them.

Project Proposal Business Need Market Opportunity

Target & Benefits Target Market Segment Claimed Customer Benefits

→ Software Requirements Spec Use Cases Use Case Suite → Use Cases Feature Specifications Feature Set Feature Specifications

User Needs Classes of Users User Stories

→

Interview Notes Requests from Customers

→

Non-Functional Requirements Environmental Requirements

Design Structural Design Behavioral Design → User Interface Build System Architecture Persistence Security

Quality Assurance QA Plan → Test Suite Test Cases Test Runs

One goal of writing use cases is to specify the system to be built, so that the resulting specification can be handed off to someone else for implementation. Another important goal is to allow potential customers to read the use cases and validate them. Long before you reach these goals, you will find that the process of writing use cases is itself a very useful way to clarify your own thinking about the system requirements: by writing step-by-step descriptions, you force yourself to think through the details of the system's features. Use cases and feature specifications complement each other. Use cases concretely explain how a user accomplishes a goal using one or more features of the system. Feature specifications describe the same system from a different perspective: a feature spec abstractly describes everything about one software feature and all it's possible uses. It is a good idea to write the use cases and the feature specs in parallel. For example, when working through a use case, you might realize that a particular feature needs an additional option, so you would note that in the feature spec. Likewise, when making a pass over the feature specifications, you might realize that a feature needs a particular input value to work properly, so you might need to add a step to all use cases for that feature. Together, use cases and feature specs provide checks and balances that help you write requirements that are more complete, correct, and consistent. The rest of this white paper works through the six steps shown in yellow in the diagram below. Beside those steps, we show the corresponding steps for writing feature specifications. (You may notice that this is very similar to the test case writing steps.) Use Case Writing Steps

Feature Spec Writing Steps

1: Identify Classes of Users

1: Identify Functional Areas

↓

↓

2: Outline the Use Case Suite

2: Outline the Feature Set

↓

↓

3: List Use Case Names

3: List Feature Names

↓

↓

4: Write Some Use Case Descriptions

4: Write Some Feature Descriptions

↓

↓

5: Write Steps for Selected Use Cases

5: Write Specs for Selected Features

↓

↓

6: Evaluate Use Cases

6: Evaluate Features

Note that in steps 4 and 5, we recommend that you only specify the most important use cases in detail. In any complex system, there will be a large number of potential use cases. It is usually best to take a breadth-first approach: map out your use case suite first, then fill in details incrementally as needed. This concept is key to getting the most value out of the limited time that you have to write specifications. After the SRS is written, each part is used in later work on the system. Both use cases and feature specs affect both the design and quality assurance of the system. However, feature specifications can affect the design more directly, and the use case suite can provide a stronger starting point for the system test suite. Step One: Identify Classes of Users

The first step in writing use cases is understanding the users, their goals, and their key needs. Not all users are alike. Some users will expect to walk up to the system and accomplish one goal as quickly as possible. For example, some banking ATM customers just want "fast cash". Others may be power users who will master every option and shortcut over time. It is important to identify and list classes of users so that none of them are forgotten. Too often, an entire class of users are initially overlooked, e.g., administrators. This leads to a frustrating series of requirements changes when their requirements must be added later. Next, make sure that you know the key needs of each class of user. For example, one class of user may simply need a speedy transaction, whereas another class of user may need more guidance in making choices, and a third class of user may expect to reuse their knowledge of competing products. How ReadySET Pro Helps ReadySET Pro helps you identify classes of users and take advantage of that information while planning your use case suite. The ReadySET Pro user needs template and use case suite template come prepopulated with reusable sample text describing classes of users that are found in many applications. The sample text will not fit your application exactly, but it gives you a big head start on this step. When you specify additional classes of users, ReadySET Pro guides you in describing their key needs. For example, which business objects are of primary concern to these users? And, which other types of users do they interact with to accomplish specific business goals? The ReadySET Pro templates are simply templates: they do not force you to follow a predefined process and they are not a magic bullet that does your work for you. For this first step, the reusable sample text gives you a head start can be expected to save you up to an hour. As you will see below, these time savings add up in each step. In addition to saving time, ReadySET Pro can improve the quality of your project documents by helping with both the breadth and details of your work. ReadySET Pro provides a broad and remarkably complete set of project document templates that ask the right questions. So, you are less likely to overlook important areas of development. At the detailed level, the reusable sample text is well written and to-the-point. So, you can adapt it to fit your project, and still end up with a well written result. Step Two: Outline the Use Case Suite It is tempting to skip this step and jump directly into writing a use case. After writing one use case, you would write another, and so on. That process would be like coding your application without outlining the design first. You would never really know how much further you need to go before you are done, if you were

spending too much time in one area, or if you had forgotten other important areas altogether. The second step in our breadth-first approach to writing use cases is to outline the use case suite. A use case suite is an organized table of contents for your use cases: it simply lists the names of all use cases that you intend to write. The suite can be organized several different ways. For example, you can list all the classes of users, and then list use cases under each. One particularly good use case suite organization is to use a grid where the rows are classes of users and the columns are business objects. Each cell in the grid will list use case names that are done by that class of user on that type of object. For example, in an e-commerce system, shoppers would have use cases for adding and removing products from their shopping carts. In contrast, administrators might have some very different use cases, for example, calculating the percentage of abandoned shopping carts. The advantage of using an organized list or grid is that it gives you the big picture, and helps you put your finger on any area that needs more work. For example, in the e-commerce grid, there might be a business object "Coupon". It is obvious that shoppers use coupons, but it is easy to overlook the use cases for administrators who must create coupons. If it is overlooked, there will be a clearly visible blank space in the use case suite. These clear indications of missing requirements allow you to improve the requirements sooner rather than get bogged down in too many frustrating requirements changes later. How ReadySET Pro Helps The ReadySET Pro use case suite template is pre-populated with reusable sample lists and grids that help you organize your use cases around classes of users, priority, product releases, business objects, and other meaningful criteria. These lists and grids help you, for example, make sure that you cover every class of user that your system must support. As stated above, starting with the ReadySET Pro templates can definitely save some time. In this particular step, you can expect to save some time on the actual writing. But more importantly, the templates guide your thinking toward a more complete specification that will lead to a better quality product in the end. Try our ROI Calculator to see for yourself how incremental improvements in quality can save substantial time and money. Step Three: List Use Case Names If you did step two, this step will be much easier to do well. Having an organized use case suite makes it easier to name use cases because the task is broken down into much smaller subtasks, each of which is more specific and concrete. Put your finger, or cursor, on each list item or grid cell in your use case suite. Then, for each one, ask yourself about the relevant goals of users. Most of the time, you will think of two to five use cases fairly easily. Sometimes, there will be a list item or grid cell that really should be empty. For example, the administrator

of an e-commerce site might not have any use cases for editing product descriptions, if that is done by a "store manager" class of users instead. In that situation, write "N/A" for "Not Applicable". Other times, you might know that there should be some use cases listed, but you cannot think of them at the moment. In that situation, write "TODO" as a reminder to come back to it later. The name of each use case should be an active verb phrase describing a goal. For example: "Select product for purchase". The name should be written in terms that a user might use themselves. So, "Put product in shopping cart" is also fine. Use case names should not be written in implementation terms: "Insert SKU into checkout phase one hash-table" is definitely wrong. Keep in mind that one of the goals of writing use cases is to allow potential customers and users to read and validate them. It is important to keep in mind that use cases focus on users' goals. For example, a banking ATM user's goal might be to "Get cash quickly." It is never the user's goal to "Choose preferred language", that is simply a step imposed by the system on the user who is trying to get cash quickly. As you work through this step, you may also think of a feature that should be specified in the feature set. If that happens, just take a moment to note down the name of the feature. Before moving on to the next step, it is worth pointing out that the result of this step is itself very valuable. Having a fairly complete suite of use case names, by itself, is major progress on the system specification. It already is enough for you to start doing some things that can help your overall project succeed. At this point, you can already get a better feeling for the scope of the release. You can already roughly prioritize use cases. You are already validating your definition of the classes of users. And, you can already recognize some needed features that might have been overlooked if you had jumped down into detailed steps too soon. How ReadySET Pro Helps The use case suite template is pre-populated with the names of several common use cases. This reusable sample text is centered around the process of user registration and login. That gives you a big head start if you are building an application that requires login. The sample text can also be helpful for application without login, because the samples demonstrate the correct tone, phrase structure, and level of detail. Step Four: Write Some Use Case Descriptions In step three, you may have generated ten to fifty use case names on your first pass. That number will grow as you continue to formalize the software requirements specification. That level completeness of the specification is very desirable because it gives more guidance in design and implementation planning, it can lead to more realistic schedules and release scoping decisions, and it can reduce requirements changes later.

The downside to mapping out the big picture is simply that it is too big. It could take a long time to fully specify every use case that you have mapped out. And, the resulting document could become too large, making it harder to validate and maintain. The solution is to be selective before moving to the next level of detail. For example, if there are clearly too many use cases for one release, reschedule some of them for later releases. Also, it's a good idea to just write descriptions rather than get into detailed steps for each use case. Going deep into the details of just a few use cases is enough to shake out uncertainties and identify areas for improvement. The bulk of the use cases can be done later, it needed at all. Write one to three sentence descriptions of each use case that you plan to implement in this release. The description should provide a little more information on the user's goal and briefly outline the strategy that the user will follow. Sometimes there will be two or more ways to accomplish the same goal. If there are significant differences in strategy, it is a good idea to split the use case into two distinct use cases. How ReadySET Pro Helps The use cases template is pre-populated with descriptions of several use cases focused on user registration and login. As mentioned in the previous steps, that reusable text can give you a big head start, and demonstrate the correct tone, phrase structure, and level of detail. Step Five: Write Steps for Selected Use Cases Now it is time for the main event: actually writing the use case steps. This is a task that you can expect to take ten to forty-five minutes for each use case. That might average out to only about ten use cases in a typical work day. Again, you must be selective to get the most value in return for your limited available time. Focus on use cases that seem most likely to affect the success of the project. For example, select use cases that: • • • • •

Enable users to achieve the key benefits claimed for your product Determine a user's first impression of the product Challenge the user's knowledge or abilities Affect workflows that involve multiple users Explain the usage of novel or difficult-to-use features Each use case should show a straightforward example of the user succeeding at a goal. The steps in a use case are almost always a linear sequence. You should not use programming constructs such as if-statements or loops, if at all possible. Rather than use conditional statements, use an extension point to describe any type of failure or error condition. Each use case step has two parts: a user intention and system response: User Intention

The user intention is a phrase describing what the user intends to do in that step. Typical steps involve accessing information, providing input, or initiating commands. Usually the user intent clearly implies a UI action. For example, if I intend to save a file, then I could probably press Control-S. However, "press Control-S" is not written in use cases. In general, you should try not to mention specific UI details: they are too low-level and may change later. System Response The system response is a phrase describing the user-visible part of the system's reaction to the user's action. As above, it is best not to mention specific details that may change later. For example, the system's response to the user saving a file might be "Report filename that was saved". The system response should not describe an internal action. For example, it may be true that the system will "Update database record", but unless that is something that the user can immediately see, it is not relevant to the use case. Use case steps can be written in either two-column or one-column format. The two-column format forces every step to include an explicit user intention and system response. The one-column format gives you more flexibility to skip system responses that are obvious, and to handle multiple actors interacting with the system in one use case. Recall that one of the advantages of writing use cases is that it forces you to clearly think through the details of the system. Capture your insights by writing notes and questions as you go. If a use case step reminds you of a specific requirement in a system feature, make a note of it in the corresponding feature specification. How ReadySET Pro Helps ReadySET Pro contains several high-quality, reusable sample use cases. These use cases demonstrate the proper use of both formats, and the proper handling of exceptions. You can preview the Use Case Template or any other template through the Document Map. A separate "use case format" document defines a notation for expressing user intentions using a small set of standard keywords. Each use case template in ReadySET Pro also indicates its priority, frequency, preconditions, and direct actors. Possible values for these project management fields are also defined in the "use case format" document. Step Six: Evaluate Use Cases An important goal of any requirements specification is to support validation of the requirements. There are two main ways to evaluate use cases: • •

Potential customers and users can read the use cases and provide feedback. Software designers can review the use cases to find potential problems long before the system is implemented.

To make customer or user validation more effective, it is important to keep the steps simple, concrete, and at the right level of detail. As recommended above, you should avoid using programming constructs, in part, because users may not be familiar with them. Also, users have a bad habit of providing feedback on anything that is part of the use case, even if that is not the type of feedback you need. For example, if you had included unneeded UI details such as "Scroll and control-click the name of each desired country in the list", then you are likely to get feedback on the way that standard list widgets work rather than insights into the actual task at hand. Phrasing the user intention as "Select desired counties" forces everyone to focus on the relevant issues, e.g., will the user even know which countries are desired at this point in the interaction? When reviewing use cases, you should start by checking for too many steps or especially difficult steps. A step may be difficult for a user if it requires knowledge that the user does not have in mind at that time, e.g., what are the zip codes of your last three residences? User often have difficulty remembering to perform "post-completion" steps that do not seem relevant to their immediate goal, e.g., children often do not remember to put away toys because their goal was simply to play with them. Another very simple way to evaluate use cases is to try performing them yourself. A rough mockup of the system can simply list the contents of each screen, without getting into details of screen layout or particular UI widget selections. Pretending to use the mockup to work through the use case can point out some use case problems. For example, you may have specified a particular system response for a step, but then find that there is no good way for the system to present that response on the current screen. You can perform a more careful evaluation of your use cases and UI mockups with cognitive walk-throughs. In the cognitive walk-through method, you ask yourself these questions for each step: • • • •

Will the user realize that he/she should have the intention listed in this step? Will the user notice the relevant UI affordance? Will the user associate the intention with the affordance? Does the system response clearly indicate that progress is being made toward the use case goal?

How ReadySET Pro Helps ReadySET Pro's use case templates contain high-quality, reusable sample text that makes it easier to write use cases at the right level of detail. Keeping the right level of detail makes it easier to evaluate and improve your use cases without getting bogged down in UI details too soon. The light-weight, web-based nature of ReadySET Pro makes it easy to share individual documents with a potential customer or user so that they can provide feedback. And, the comprehensive set of templates provided by ReadySET Pro helps you evaluate the use cases for consistency with project goals, claimed product benefits, user needs, the system design, and the QA plan.

The ReadySET Pro UI design template encourages designers to start with abstract descriptions of each screen that can be used to evaluate use cases. Without this step in design, far too many UI designers immediately jump into UI screen layout, widget selection, and careful wording of particular labels. While those design decisions need to be made eventually, it is usually better to start by validating the overall interaction flow for the highest priority use cases. Conclusion This white paper presented six simple steps to writing effective use cases. The keys to using use cases to effectively specify software requirements are to: • • •

Take a breadth-first approach that maps out a fairly complete set of possible user interactions and provides full details on those use cases that are likely to give the best return on your efforts. Write use cases at the right level of abstraction, focusing on user intent and avoiding unneeded UI details. Evaluate use cases to help validate the system requirements before moving on to design and implementation. ReadySET Pro provides valuable help for writing effective use cases by giving you templates that include reusable content and set good examples for you to follow. Both of these advantages give you a big head start on your own software requirements specification. ReadySET Pro users typically save at least three hours by using the use case suite and use case templates alone. To estimate how these savings, and the savings from other templates, add up to days or weeks of project time, see the ROI Calculator. Write Your Software Testing Plan Today

Introduction All software needs to be tested. In fact, software testing is a major part of the overall software development process that involves many people and countless hours of detailed work. Unfortunately, most testing efforts are under-planned: some software testing professionals work in the field for years without ever seeing a really comprehensive QA plan and test suite. Part of the problem is that QA efforts often begin too late in the release cycle when there is too much pressure to take shortcuts. This white paper shows how ReadySET Pro can be used to quickly create a comprehensive system test suite with test cases. ReadySET Pro's unique contentrich templates help you write better test suites that can improve your product quality. These templates make the writing process much faster than starting from scratch, which means that you are more likely to be able to complete your plan, even under time pressure.

The figure below illustrates where your QA plan and test suite fit with other project documents. This white paper focuses on the yellow "Quality Assurance" box. Ideally, your testing documents are just part of an overall set of project documents. But even if you do not have the others, you will be able to follow the discussion of test planning documents. Project Proposal Business Need Market Opportunity

Target & Benefits Target Market Segment Claimed Customer Benefits

→ Software Requirements Spec Use Cases Use Case Suite → Use Cases Feature Specifications Feature Set Feature Specifications

User Needs Classes of Users User Stories

→

Interview Notes Requests from Customers

→

Non-Functional Requirements Environmental Requirements

Design Structural Design Behavioral Design → User Interface Build System Architecture Persistence Security

Quality Assurance QA Plan → Test Suite Test Cases Test Runs

Software development projects that don't have enough test planning tend to bog down with defects that can put the entire project's success at risk. Test planning helps in the following specific areas: Requirements Validation Designing a system test suite forces you to deeply understand the requirements. As you understand the requirements more, you will notice incompleteness, ambiguity, and inconsistency. Correcting these problems early can speed up development and reduce the number of late requirements changes. Testing Coordination Testing involves many people working together over time. For the team to be effective, their efforts must be coordinated with a written plan. Test Coverage Testing only half of a large system is sure to allow thousands of defects into the shipping product. A QA plan is needed to set coverage criteria and evaluate coverage. A test suite must be carefully designed with the coverage criteria in mind. Test Automation Too often, QA teams hope to use automated testing, but end up stuck with ad-hoc manual testing. This happens because they never really formalize the requirements, so they must always rely on human judgment to evaluate test outputs. Creating automated test scripts without outlining the test suite is like writing code without a design document. The following diagram illustrates the

gap between ad-hoc testing and automated testing, and how systematic testing with a test suite bridges that gap. Ad-Hoc Testing Just see if you can break it Make up test cases "on the fly" Human interpretation of requirements

Systematic Testing Driven by explicit QA goals → Test suite designed → for coverage → Tests specify expected output

Automated Testing Driven by explicit QA goals → Test suite designed for → coverage → Scripts need no human judgment

The rest of this white paper works through the steps shown in the diagram below. (You may notice that this is very similar to the use case writing steps.) Use Case Writing Steps 1: Overall QA Planning

↓

2: Outline the Test Suite

↓

3: List Test Case Names

↓

4: Write Some Test Case Descriptions

↓

5: Write Selected Test Cases

↓

6: Evaluate Test Cases

Note that in steps 4 and 5, we recommend that you only specify the most important test cases in detail. In any complex system, there will be a large number of potential test cases. We encourage you to take a breadth-first approach: map out your test case suite first, then fill in details incrementally as needed. This concept is key to getting the most value out of the limited time that you have for test planning. Step One: Overall QA Planning Software quality is not one-size-fits-all: different software products need different types of testing because they have different QA goals. For example, a real-time system may place much more priority of performance than would a typical desktop business application. The task of QA planning is discussed in detail in the "Quality Throughout the LifeCycle" white paper. The main parts of the overall QA plan are: • • • •

Select and prioritize quality goals for this release Select QA activities to achieve those goals Evaluate how well the activities support the goals Plan the actions needed to carry out the activities The overall QA plan addresses all quality activities. Quality can be achieved by building in better quality from the start, and by testing to find and remove defects. Specific QA activities include: coding preconditions, reviewing design and code, unit testing, integration testing, system testing, beta testing, using analysis tools, and field failure reports, among others. The rest of this paper will focus in on just the system testing activity. How ReadySET Pro Helps ReadySET Pro helps with every part of the overall QA plan. The ReadySET Pro QA plan template is pre-populated with detailed goals, activities, evaluation, a plan of action, and a built-in checklist. Adapting the QA plan to your project is largely a matter of "chipping away." That means, selecting the sample answers that fit best rather than writing new text. To see how fast and easy chipping away can be, see the Step-By-Step Demo. ReadySET Pro users have reported that the QA plan template alone saved them an average of three hours. More importantly, a good QA plan can save many more hours later by reducing the number of defects. Use the ROI Calculator to see how the savings for the QA plan, test cases, and other documents really add up to save days or weeks of your project schedule. Step Two: Outline the Test Suite Once you have prioritized your QA goals, it is time to outline the system test suite. A test suite document is an organized table of contents for your test cases: it simply lists the names of all test cases that you intend to write. The suite can be organized in several ways. For example, you can list all the system

components, and then list test cases under each. Or, you could list major product features, and then list test cases for each of those. One of the best test suite organizations is to use a grid where the rows are types of business objects and the columns are types of operations. Each cell in the grid lists test cases that test one type of operation on one type of object. For example, in an e-commerce system, a Product business object would have test cases for each of the following operations: adding a product to the system, listing or browsing products, editing products, deleting products, searching products, and calculating values related to the product such as shipping cost or days-untilshipment. The next row an e-commerce test suite grid might focus on the Customer Order business object and have test cases for almost all the same operations. The advantage of using an organized list or grid is that it gives you the big picture, and it helps you put your finger on any area that needs more work. For example, in the e-commerce grid, there might be a business object "Coupon." It is obvious that shoppers use coupons, but it is easy to forget to test the ability for administrators to create coupons. If it is overlooked, there will be a clearly visible blank space in the test suite document. These clear indications of missing test cases allow you to improve the test suite sooner, make more realistic estimates of testing time needed, and find more defects. These advantages allow the found defects to be fixed sooner and help keep management expectations in sync with reality, which helps keep the project out of crisis-management-mode. How ReadySET Pro Helps The ReadySET Pro test suite template is pre-populated with reusable sample lists and grids that help you organize your test cases around business objects, operations, or requirement priority. These lists and grids help you achieve your desired specification-based coverage criteria. For example, it is very easy to see whether you cover every type of business object that your system uses. As stated above, starting with the ReadySET Pro templates can definitely save some time. In this particular step, you can expect to save some time on the actual writing. But more importantly, the templates guide your thinking toward a more complete test suite that will lead to a better quality product in the end. Since so much tine and effort in software development goes into tracking, diagnosing, repairing, and retesting defects, improving quality saves a significant amount of time. Step Three: List Test Case Names After you have outlined your test suite, this step becomes much easier to do well. Having an organized system test suite makes it easier to list test cases because the task is broken down into many small, specific subtasks. Put your finger, or cursor, on each list item or grid cell in your test suite. Then, for each one, ask yourself about the relevant system requirements. If you have a written use case document, you will often be able to turn each use case into one or more test cases. There may be some list items or grid cells that really should be empty. For example, an e-commerce application might not have any delete

operation for the Customer Order business object. Explicitly mark with "N/A" any cells that logically should not have test cases. If you cannot think of any test cases for a part of the suite that logically should have some test cases, explicitly mark it as "TODO". The name of each test case should be a short phrase describing a general test situation. Append a unique number to each test for the given test situation. For example: login-1, login-2, login-3 for three alternative ways to test logging in. And, sales-tax-in-state-1 and sales-tax-out-of-state-1 for two different situations where collected sales taxes are reported to the government according to two different procedures. Use distinct test cases when different steps will be needed to test each situation. One test case can be used when the steps are the same and different input values are needed. As you gradually fill in the test suite outline, you may think of features or use cases that should be in the software requirements specification (SRS), but are not there yet. Quickly note any missing requirements in the SRS document as you go along. Before moving on to the next step, it is worth highlighting the value of having a fairly complete test suite outline. The test suite outline is a useful asset that can help your project succeed. At this point, you can already get a better feeling for the scope of the testing effort. You can already roughly prioritize test cases. You are already starting to look at your requirements critically and you may have identified missing or unclear requirements. And, you can already estimate the level of specification-based test coverage that you will achieve. How ReadySET Pro Helps The test suite template is pre-populated with the names of sample test cases for an e-commerce application. This reusable sample text is centered around the process of user registration and login. That gives you a big head start if you are building an application that requires login. The sample text can also be helpful for applications without login, because the samples demonstrate the correct tone, phrase structure, and level of detail. Step Four: Write Some Test Case Descriptions In step three, you may have generated between ten and fifty test case names on your first pass. That number will go up as you continue to make your testing more systematic. The advantage of having a large number of tests is that it usually increases the coverage. The disadvantage to creating a big test suite is simply that it is too big. It could take a long time to fully specify every test case that you have mapped out. And, the resulting document could become too large, making it harder to maintain. A good strategy is to be selective before drilling down to the next level of detail. For example, you might prioritize the test cases based on the priorities of the features or use cases that they test. Also, it's a good idea to first write descriptions rather than get into detailed steps for each test case. Going deep

into the details of just a few test cases may be enough to shake out ambiguity or incompleteness in the requirements. The remaining cases should all be specified eventually, however you might choose to rely on ad-hoc testing for lower priority features in early releases. For each test case, write one to three sentences describing its purpose. The description should provide enough information so that you could come back to it after several weeks and recall the same ad-hoc testing steps that you have in mind now. Later, when you actually write detailed steps in the test case, you will be able to expect any team member to carry out the test the same way that you intended. The act of writing the descriptions forces you to think a bit more about each test case. When describing a test case, you may realize that it should actually be split into two test cases, or merged with another test case. And again, make sure to note any requirements problems or questions that you uncover. How ReadySET Pro Helps The test cases template is pre-populated with descriptions of several test cases focused on user registration and login. As mentioned in the previous steps, that reusable text can give you a big head start, and demonstrate the correct tone, phrase structure, and level of detail. Step Five: Write Selected Test Cases Now it is time for the main event: actually writing the test case steps and specifying test data. This is a task that you can expect to take ten to forty-five minutes for each test case. That might work out to approximately ten test cases in a typical work day. So, you must be selective to get the most value in return for your limited available time. Focus on the test cases that seem most in need of additional detail. For example, select system test cases that cover: • • • • •

High priority use cases or features Software components that are currently available for testing (rather than specifying tests on components that cannot actually be tested yet) Features that must work properly before other features can be exercised (e.g., if login does not work, you cannot test anything that requires a logged in user) Features that are needed for product demos or screenshots Requirements that need to be made more clear Each test case should be simple enough to clearly succeed or fail, with little or no gray area in between. Ideally, the steps of a test case are a simple sequence: set up the test situation, exercise the system with specific test inputs, verify the correctness of the system outputs. You may use programming constructs such as if-statements or loops, if needed. Systems that are highly testable tend to have a large number of simple test cases that follow the set-up-exercise-verify pattern. For those test cases, a one-column

format can clearly express the needed steps. However, not all test cases are so simple. Sometimes it is impractical to test one requirement at a time. Instead, some system test cases may be longer scenarios that exercise several requirements and verify correctness at each step. For those test cases, a twocolumn format can prove useful. In the one-column format, each step is a brief verb phrase that describes the action that the tester should take. For example, "enter username," "enter password," "click 'Login'," "see Welcome page," and "verify that greeting has correct username" are all steps. Verification of expected outputs are written using the verbs "see" and "verify." If multiple inputs are needed, or multiple outputs must be verified, one-column test cases will simply have more steps. In the two-column format, each test case step has two parts: a test input, and an expected output: Test Input The Test Input is a verb phrase describing what the tester should do in that step. Expected Output The Expected Output is a noun phrase describing all the output that the tester should observe at that step. You may notice that the two formats for test cases mirror the two formats for use cases. The difference is that use cases are a form of requirements, whereas test cases deal with more details of the implemented system. Use cases focus mainly on the user's tasks and how the system supports those tasks, while specifying as few implementation details as possible. A major advantage of use cases is that they are simple enough to be read by actual users who can help validate requirements. In contrast, test cases should more technical documents with enough implementation detail to allow any member of the development team to carry out a test exactly the same way. If you have written use cases, they can be copied and pasted as a good starting point for test cases. When leveraging use cases in this way, make sure to add enough detail to make the test reliably repeatable. If you only have one test input value for a given test case, then you could write that test data value directly into the step where it is used. However, many test cases will have a set of test data values which must all be used to adequately cover all possible inputs. We encourage you to define and use test input variables. Each variable is defined with a set of its selected values, and then it is used in test case steps just as you would use a variable in a programming language. When carrying out the tests, the tester should repeat each test case with each possible combination of test variable values, or as many as practical. Carefully selecting test data is as important as defining the steps of the test case. The concepts of boundary conditions and equivalence partitions are key to good test data selection. Try these steps to select test data: •

Determine the set of all input values that can possibly be entered for a given input parameter. For example, the age of a person might be entered as any integer.

•

• •

•

•

Define the boundary between valid and invalid input values. For example, negative ages are nonsense. You might also check for clearly unreasonable inputs. For example, an age entered as 200 is much more likely to be a typo than a user who is actually two-hundred years old. Review the requirements and find boundaries in the valid range that should cause the system to behave in different ways. For example, the system might treat minors differently than adults, so the boundary would be age 18. Now you have a set of equivalence partitions: sets of values that the system should treat uniformly. For example, all minors are treated one way, and all adults are treated another way. Double check the requirements to make sure that you have not missed a partition division, e.g., not all adults are old enough to drink alcohol in the U.S. Choose one input value somewhere in the middle of each equivalence partition (e.g., -5, 12, and 44), one directly on each boundary (e.g., 0 and 18), and one on each side of each boundary (e.g., 1, 17, and 19). Test data vales that are expected to cause errors (e.g., -5) should be tested in separate robustness test cases. In functional correctness test cases, make sure that you have inputs that will force the system to generate each possible type of response to valid input. And, in robustness test cases, make sure to force the system to generate each relevant error message. Recall that one of the advantages of writing test cases is that it forces you to clearly think through the requirements. Capture your insights by writing notes and questions as you go. If a test case step exposes an unclear requirement, make a note of it in the appropriate part of the system requirements specification. How ReadySET Pro Helps ReadySET Pro contains several high-quality, reusable sample test cases. These test cases demonstrate the proper use of both formats, for both functional and robustness testing. You can preview the Test Case Template or any other template through the Document Map. A separate "test case format" document defines a notation for expressing test case steps using a small set of standard keywords. Each test case template in ReadySET Pro also indicates its preconditions. The preconditions of a test case describe conditions that must be true before the test can be carried out. Writing preconditions helps to avoid writing redundant steps for setting up the needed testing situation. Step Six: Evaluate Test Cases A suite of system test cases can find many defects, but still leave many other critical defects undetected. One clear way to guard against undetected defects is to increase the coverage of your test suite. While a suite of unit tests might be evaluated in terms of its implementation coverage, a suite of system test cases should instead be evaluated in terms of specification coverage. Implementation coverage measures the percentage of

lines of code that are executed by the unit test cases. If there is a line of code that is never executed, then there could be an undetected defect on that line. Specification coverage measures the percentage of written requirements that the system test suite covers. If there is a requirement that is not tested by any system test case, then you are not assured that the requirement has been satisfied. You can evaluate the coverage of your system tests on two levels. First, the test suite itself is an organized table of contents for the test cases that can make it easy to notice parts of the system that are not being tested. Second, within an individual test case, the set of possible input values should cover all input value equivalence partitions for each parameter. How ReadySET Pro Helps ReadySET Pro's system test suite template and test case templates contain highquality, reusable sample text that helps you plan and write test cases that cover all requirements. The templates encourage you to use a format that makes it easier to evaluate your software testing plans. The light-weight, web-based nature of ReadySET Pro makes it easy for the entire development team to use and update the latest revisions of the system test suite. And, the comprehensive set of templates provided by ReadySET Pro helps you keep the system test documents in sync with the overall QA plan, the system requirements specification, and design documents. Conclusion This white paper laid out the steps needed to quickly create a system test suite and test cases using the ReadySET Pro templates. The keys to effectively system testing are to: • • • •

Set explicit quality goals that are appropriate for the current release and understand where your test plan fits in your overall QA plan. Take a breadth-first approach by first mapping out a test suite with good coverage, and then prioritizing your work on specifying the test cases themselves. Write system test cases in enough detail that any member of the development team could carry out the testing to see the same results, and choose test input values that cover all equivalence partitions. Evaluate the system test suite and test cases to improve coverage of the system requirements. ReadySET Pro provides valuable help for planning your system testing by giving you templates that include reusable content and set good examples for you to follow. Both of these advantages give you a big head start on your own test plans. ReadySET Pro users typically save at least three hours by using the test case suite and test case templates alone. See how these savings, and the savings from other templates, add up to days or weeks of project time by trying the ROI Calculator.