Test Methodologies

Test Methodologies Aztecsoft has evolved a three-step approach to testing software, based on best practices observed throughout a wide variety of successful testing projects. Test Strategy

Test Execution

Test Planning

Test Strategy A product's test strategy ties the product's release and sign-off criteria to its business objectives. The overall testing strategy is defined in collaboration with the customer. In order to take key dependencies into account, test planning, testcase design, test automation and test execution are aligned with the development schedule. Meaningful test scheduling requires a clear understanding of ETA's and sequencing for : • Completion of low- and high-level specifications; • Code-complete (coding for everything but bug-fixes stops); • Completion of component unit-testing (when QA can begin interaction testing); • UI-freeze (after which QA can be confident that (a) UI level automation will not break repeatedly due to fluctuating UI and screen layouts, and (b) that API-level automation will not be undermined by changes in interfaces/API's). Identifying key trade-offs is essential, for it is impossible to test all scenarios, cover the full configuration matrix, and automate all test cases, while remaining within the practical limits of time and budget. Trade-offs on the development side need to be in concert with trade-off development trade-offs; otherwise, development and test will have conflicting priorities. This is the stage in which project focus is established. We identify the features, components, sub-components, and items to be tested and the range of tests to be carried out. In addition to available automation we also estimate other required and possible automation. We catalog the tools used by the customer, potentially useful off-the-shelf tools, and internal Aztecsoft tools that may be used for the project. We identify which features/components will be tested manually, which will be tested via automation, and what kind of automation tool is required (script-based, GUI-based, proprietary, off-the-shelf, etc.). To summarize, the key activities are as follows: • Define project scope & commitments • Define terms of reference • Set customer expectations • Tie together the business objectives of the STQE project with the release/sign off criterion and associated testing activity • Integrate the STQE processes with development lifecycle • Partition the problem into manageable test plans • Identify key dependencies & trade-offs • Scope resource requirements TOP

Test Planning The next step is Test Planning, which defines the approach for testing. The first task is to establish,

and seek confirmation from the customer, a clear understanding of the project and its deliverables. Exhaustive analysis ensures that there is no mismatch between our understanding and the customer's requirements. For example we determine whether we should construct a clean test environment for each test run, or we can use system imaging to shorten test-bed setup. All relevant products interface, component, and other external dependencies are identified and the timeframe for delivering the results is computed. The resulting plan is presented in industrystandard format to the customer; further steps are not taken without customer acceptance. Here are the key steps for Test Planning: • Define release criteria (with the release manager) • Outline and prioritize the testing effort. • Chart test automation requirements • Identify resource requirements at various stages of testing • Set up calendar-based activity plan • State reporting mechanism & establish communication model • Configure team including number, type, and seniority of resources and length of time required, mapped each resource onto the activity plan. TOP

Test Execution Execution of the test plan begins with fulfillment of test-start criteria, and ends with the fulfillment of test-complete criteria. Although each project has different requirements, the intermediary steps will resemble the following: • Prepare comprehensive test plan specifications and test cases for each level of testing. • Review all test plans and test cases • Prepare test data and test logs. • Set up the test environment so that all operations can be completed promptly, accurately, and efficiently. • Execute Error/Trap tests to ensure testers accuracy. • Execute tests as described, noting where test cases require revision and updating. • Report all bugs in the manner agreed upon with the customer, following all defect management protocols, informing customer of current status, monitoring and driving to resolution all red-flag conditions, and ensuring that communication between all parties is complete and accurate. • Run spot checks to ensure quality. • Update weekly the Project Health Status document for Internal Audit & Tracking. • When the project has been completed, review all aspects of the project and submit a Project Retrospective report to the customer that objectively evaluates the project's execution