Qa & Engineering Lrc Presentation

The Company…. ♦ The Market Leader in Globalization

Technology

– Pioneers in visual translation environments – Solutions for major platforms & programming languages – 24.9% owned by Corel Corporation – Sales growing 40+% annually – Currently 9,000+ customers worldwide – Offices in Ireland and USA • Japanese office to open Q3 2003

Challenges

Simultaneous Translation, Engineering & Testing

♦ We live in a Parallel Universe – Translation • • • •

Seldom starts or finishes on time Difficulty getting source materials from development teams Diverse goals of translation/development teams Goal : Translate as much as possible as quickly as possible

- Quality Assurance

• To monitor quality levels of Engineering • To monitor quality levels of Translation • Goal : To locate, verify and report as many quality performance issues during the schedule

– Engineering

• Struggle to control inputs to drive definite outcomes • Create continuous streaming of localization materials to Translators and Quality Assurance team • Goal : To locate, debug and fix as many quality performance issues during the schedule

Challenges

Simultaneous Translation, Engineering & Testing

♦ We live in a Parallel Universe – Translation • • • •

Seldom starts or finishes on time Difficulty getting source materials from development teams Diverse goals of translation/development teams Goal : Translate as much as possible as quickly as possible

- Quality Assurance

• To monitor quality levels of Engineering • To monitor quality levels of Translation • Goal : To locate, verify and report as many quality performance issues during the schedule

– Engineering

• Struggle to control inputs to drive definite outcomes • Create continuous streaming of localization materials to Translators and Quality Assurance team • Goal : To locate, debug and fix as many quality performance issues during the schedule

QA Engineering

Overview of Test Phases

– Evaluation & Planning Phase

– Goal : Defines the scope of the Test Plan – Includes Resource Plan & Scehdule – Product Modularization Report & Baseline Defined

- Exploratory Phase

- Goal : Gain familiarity with application - Pretesting of application

- Development Phase

- Goal : Create Test Scenarios to support Test Plan

- Execution Phase

- Modular testing of application areas - Internationalization & Language Verification

- Pruning Phase

- Interlaced throughout your testing cycles - Provides audit for improving process and test scenarios

- Regression & Verification Phase - Retest and verify Engineering activities

- Sign-off Phase

- Additional testing requirements for release and mastering

QA Engineering

Designing a Test Case

Clearly Defined Goal * What is the function of the test ?

Clear Test Activity * What steps do I follow to create the test scenario ?

Deterministic Outcome * How do I determine success/failure Remember! of test ?

Tests can only prove that errors exist, not that a product is free of errors!

QA Engineering – Common Types of Tests – MenuWalker – Functional - Scenario - Random - Automated - Interoperability - Language Quality - Data Path - Boundary Incremental Testing Strategy - I18N

Types of Testing

For optimal efficiency, your L10N teams should not test baseline functionality! It should focus on incrementally testing that the product integrity is maintained through the localization process.

QA Engineering – Common Types of Tests – MenuWalker – Functional - Scenario - Random - Automated - System - Acceptance - Language Quality - Data Path - Boundary - I18N - Performance

Types of Testing MenuWalker Tests Objective • Exercise all UI of a translated product • Locate all cosmetic issues Execution • Easily Automated • Manual Scripts easy to develop • Various Tools : Alchemy CATALYST RTV, ToolProof, Visual Test Deterministic Outcomes • UI devoid of all cosmetic and layout bugs • Translation Completeness

QA Engineering – Common Types of Tests – MenuWalker – Functional - Scenario - Random - Automated - System - Acceptance - Language Quality - Data Path - Boundary - I18N - Performance

Types of Testing Functional Tests Objective • Verify the functional integrity of application Execution • Modular approach is best • Focus on Unit, Module and Components • Data Comparative cases are easily automated Deterministic Outcomes •Product is functionally consistent with original base language application

QA Engineering – Common Types of Tests – MenuWalker – Functional - Scenario - Random - Automated - System - Acceptance - Language Quality - Data Path - Boundary - I18N - Performance

Types of Testing Scenario Tests Objective • Verify product functional integrity in various realworld test scenarios Execution • Previous regression tests and technical support feedback can be useful • Can be complex to automate • Focus on many System variations Deterministic Outcomes •Product is functionally consistent with original base language application

QA Engineering – Common Types of Tests – MenuWalker – Functional - Scenario - Random - Automated - System - Acceptance - Language Quality - Data Path - Boundary - I18N - Performance

Types of Testing Random Tests Objective • Verify product functional integrity using random test data inputs and scenarios Execution • Unstructured testing of application in real-world scenarios • Almost impossible to automate • Only useful with experienced QA Engineers Deterministic Outcomes • Generally none, but may prove ‘Product Robustness’

QA Engineering – Common Types of Tests – MenuWalker – Functional - Scenario - Random - Automated - System - Acceptance - Language Quality - Data Path - Boundary - I18N - Performance

Types of Testing Automated Tests Objective • Verify product functional integrity using automated test cases Execution • Structured testing of application in controlled scenarios • Very effective with comparative test cases Deterministic Outcomes • Software integrity is maintained in controlled test cases • Acceptance of software at significant project milestones

QA Engineering

Types of Testing

– Common Types of Automated Tests Tests Objective – MenuWalker • Verify product functional – Functional Automator, beware ! integrity using automated - Scenario test cases Automation is Resource Intensive - Random Execution Automated Scripts can be time-consuming • Structured testing of Automated - •Developing application in controlled Need understanding of programming techniques - •System scenarios • If not developed correctly, can be very high on maintainence - Acceptance • Very effective with No Replacement for real testing comparative test cases - Language Quality • Can compliment but not replace real ‘testers’ Outcomes Deterministic - Data Path Not everything adapts well to test automation • Software integrity is - Boundary controlled test • Good : Tests that verify an applicationmaintained works in a in particular way - I18N cases • Bad : Tests that find previously undiscovered errors • Acceptance of software at - Performance significant project milestones

QA Engineering – Common Types of Tests – MenuWalker – Functional - Scenario - Random - Automated - System - Acceptance - Language Quality - Data Path - Boundary - I18N - Performance

Types of Testing System Tests Objective • Verify product functional integrity under different system environments Execution • Structured test cases using various OS • Lends itself to automation Deterministic Outcomes • Functional completness on various OS environments • Performance of application • Verification of product license integrity •Completeness of Product International Support

QA Engineering – Common Types of Tests – MenuWalker – Functional - Scenario - Random - Automated - System - Acceptance - Language Quality - Data Path - Boundary - I18N - Performance

Types of Testing Acceptance Tests Objective • Verify minimum product quality to commence QA cycle Execution • Very Structured test cases • Improve by including previously regressed SPRs • Significant automation canidate Deterministic Outcomes • Verification of minimum functional completeness • First line of defence for detecting ‘corrupted’ builds

QA Engineering – Common Types of Tests – MenuWalker – Functional - Scenario - Random - Automated - System - Acceptance - Language Quality - Data Path - Boundary - I18N - Performance

Types of Testing Language Quality Tests Objective • Verify translation is consistent with linguistic guidelines Execution • Some fundamental tests can be easily automated • Sampling generally creates the best results due to volumes Deterministic Outcomes • Translation is consistent with Corporate Terminology guidelines across component boundaries • No cosmetic/layout issues exist (e.g. Spelling errors)

QA Engineering

Types of Testing

– Common Types of Language Quality Tests Tests Objective – MenuWalker • Verify translation is – Functional consistent with linguistic Examples of Language Quality Automation - Scenario guidelines Translation Consistency & Completeness Check - Random Execution •Relatively easy to automate • Some fundamental tests - Automated cantest be easily automated •Should always be part of an acceptance - System • Sampling generally creates Spell Checking - Acceptance the best results due to • Relatively easy to automate volumes - Language Quality • Should always be part of an acceptance test Outcomes Deterministic - Data Path Duplicate Hotkeys/Layout Issues/Placeholders • Translation is consistent - Boundary • Relatively easy to automate & locate with Corporate Terminology - I18N guidelines across component boundaries - Performance • No cosmetic/layout issues exist (e.g. Spelling errors)

QA Engineering – Common Types of Tests – MenuWalker – Functional - Scenario - Random - Automated - System - Acceptance - Language Quality - Data Path - Boundary - I18N - Performance

Types of Testing Data Path Tests Objective •Verify data integrity of application based on knowledge of internal workings of application Execution • Structured data used to test application • Focus on inputs driving deterministic outputs • Easy to automate with comparative data outcomes Deterministic Outcomes • Data input and generated outputs are consistent with base language application NOTE : Sometimes referred to as Grey box testing

QA Engineering – Common Types of Tests – MenuWalker – Functional - Scenario - Random - Automated - System - Acceptance - Language Quality - Data Path - Boundary - I18N - Performance

Types of Testing Boundary Tests Objective •Verify application functionality with extreme data inputs Execution • Structured data used to test application • Relatively easy to automate with comparative data outcomes Deterministic Outcomes • Data input and generated outputs are consistent with base language application • Applicatio robustness – illegal data values do not crash application

QA Engineering – Common Types of Tests – MenuWalker – Functional - Scenario - Random - Automated - System - Acceptance - Language Quality - Data Path - Boundary - I18N - Performance

Types of Testing L18N Tests Objective • Verify application supports all Locales Execution • Structured data that focus on locale used • Relatively easy to automate with comparative data outcomes • Break down into categories : Script & Regional Deterministic Outcomes • Application supports local script and regional conventions

QA Engineering

Types of Testing

– Common Types of L18N Tests Tests Objective – MenuWalker • Verify application supports – Functional all Locales Structure of L18N Tests - Scenario Execution Language/Script Locale - Random • Structured data that • Collation focus on locale used - Automated Language/Script • Casing • Relatively easy to - System Regional • Character Set (SBC, DBCS, BIDI) automate with comparative • Inputs – Keyboard, IME - Acceptance data outcomes • Output – FontsQuality Display/Printing - Language • Break down into categories • Character Conversions : Script & Regional - Data Path Regional Deterministic Outcomes - Boundary • Number, Currency, Time, Date, List Seperator, • Application supports local Address Format, Telephone Format - I18N script and regional conventions - Performance

QA Engineering – Common Types of Tests – MenuWalker – Functional - Scenario - Random - Automated - System - Acceptance - Language Quality - Data Path - Boundary - I18N - Performance

Types of Testing Performance Tests Objective •Determine how system operates under ‘Load’ conditions Execution • Structured environments used to test application • Relatively easy to automate Deterministic Outcomes • Application operates successfully under stressful load conditions • May identify areas of functionality for optimization by Development team

QA Engineering

Types of Testing

– Common Types of Performance Tests Tests Objective – MenuWalker •Determine how system – Functional Factors to Consider during Performance Testing operates under ‘Load’ - Scenario conditions Network Capacity - •Random Execution Concurrent users • Structured environments - •Automated Remote Sessions/Connections used to test application - System Disk, Memory and CPU Usage • Relatively easy to - •Acceptance Run system under minimum and maximum boundary conditions automate - Language Transaction LoadingQuality Deterministic Outcomes Path - •Data Continuous transaction loading over time • Application operates successfully under stressful - Boundary load conditions - I18N • May identify areas of functionality for - Performance

optimization by Development team

QA Engineering –

Tracking your QA Engineering Projects – Coverage Reports – By Platforms – By Module, Component, Unit – By Tester – By Build, Release – By Language – Bug Tracking – 3 Point Tracking – By Category, Severity – Find vrs Fix Ratio – Pending Tracking – Language Quality Reports – Terminology Consistency Report – Sampling Reports – Error Density

Managing & Tracking

450 400 350 300

Open

250

Closed

200

Pending

150 100 50 1

2

3

4

5

6

7

8

QA Engineering – Alchemy CATALYST 4.0 – RTV Component –

Locates and tracks common localization issues

– Validate Expert –

Automates detection and tracking of localization issues

– SDL ToolProof –

Identifies UI Issues

– General Automation Tools – Seque ( Tester) – – –

SilkTest International SilkPerformer SilkRadar

– – –

Wide selection of Developer Tools Code Analysis WEB Services Analysis

– –

WinRunner Test Director 7.x

– Rational Systems

– Mercury Interactive

Tools to Help you Manage

QA Engineering

When to start IQA

Traditional Product Development Cycle Base Product Development Base Product QA Cycle Localization Project Cycle IQA Cycle

Disadvantages –Localization & IQA not integral to base development –Base Product completes before any focus on International Issues –IQA bugs only addressed after release of base product –No sharing and coupling of test plans between Base QA and IQA teams –No International Product Development Planning

–IQA Starts after Localization begins –Translated product required to start IQA effort –Any IQA issue that needs to be fixed is eventually addressed post-base produuct release

QA Engineering

When to start IQA

Ideal Product Development Cycle Base Product Development Product QA and IQA Cycle Localization Project Cycle IQA Cycle

Advantages –IQA is an integral part of the PD Lifecycle –Releasing localized products is a Corporate Wide Goal –Base QA team are aware of and implementing IQA in their Base Product QA plans –Development fix IQA issues as they develop Base product –Corporate Goal : Universal English Product

–Localization Cycle is parallel to development effort –Localization – Start as late as possible to finish as early as possible –Simultaneous Release of language products now feasible!

Thank you….

After Lunch Challenge

Devise a test plan for the following code segment… integer foo(integer x, integer y, integer z ) { integer Result let Result = x * y / z }

return Result