Software Quality Assurance

Software Quality Assurance

SEN-269 : Software Engineering Tazeen Muzammil

What is Quality • The term 'quality' is often used in a vague,

blurred way. If someone talks about 'working on quality', they may simply mean activities designed to improve the organization and its services.  • Quality is about: – – – –

Knowing what you want to do and how you want to do it. Learning from what you do. Using what you learn to develop your organization and its services. Seeking to achieve continuous improvement. Satisfying your stakeholders - those different people and groups with an interest in your organization.

Why Quality Matters? Increasing complexity of software product Increasing demand for software products Increasing lifetime of software products Increasing use of "Safety Critical" or "Mission Critical" software

3

Software Quality Software Quality is defined as Conformance to: 1. explicitly stated functional and performance requirements, 2. explicitly documented development standards, 3. and implicit characteristics that are expected of all professionally developed software. In the context of software engineering, software quality measures how well software is designed (quality of design), and how well the software conforms to that design (quality of conformance). 4

What is Quality Control? Quality Control is the process of developing systems that ensures that products or services are designed and produced to meet or exceed customer requirements, and expectations. Key concept of quality control: Is to ensure that the products, services, or processes provided meet specific requirements and are dependable, satisfactory, and fiscally sound. Implementation approaches: - Fully automated - Entirely manual - Combination of automated tools and human interactions

What is Software Quality Assurance? Software Quality Assurance (SQA) is defined as a planned and systematic approach to the evaluation of the quality of the software product standards, processes, and procedures. SQA includes the process of assuring that standards and procedures are established and are followed throughout the software acquisition life cycle. Goal --> provide management with the necessary data about product quality. --> gain the insight and confidence of product quality

Software Quality Assurance The most popular tool used to determine quality assurance is the Shewhart Cycle, developed by Dr. W. Edwards Deming. This cycle for quality assurance consists of four steps: Plan, Do, Check, and Act. These steps are commonly abbreviated as PDCA. ISO 9001:2000 was developed in line with the long-used Shewhart cycle of process or project management; Plan, Do, Check, Act, referred to in ISO 9004 as Plan, Implement, Check, Review.

7

Plan, Do, Check, Act (PDCA) Plan; adhering to the company mission and vision, the management sets objectives based on information gathered through the quality system (customer feedback, management reports, etc.) Do; the product is produced or the service delivered in accordance with the customers requirements. Check; was it done right? Obtain information from the customer and the organization to measure and monitor performance. Act; “freeze” successful activities and improve less successful ones. This step is the key to continual improvement.

8

9

10

11

12

13

14

Quality Consulting

15

What is Software Quality Engineering? Software Quality Engineering is composed of two primary activities

Process level quality Process level quality is normally called quality assurance. 2. Process level quality establishes the techniques, procedures, and tools that help promote, encourage, facilitate, and create a software development environment in which efficient, optimized, acceptable, and as fault-free as possible software code is produced. 1.

Product level quality

Product oriented quality is normally called testing 2. Product level quality focuses on ensuring that the software delivered is as error-free as possible, functionally sound, and meets or exceeds the real user’s needs. Testing is normally done for finding errors in order to get rid of them. 1.

Cost of Quality Cost of quality includes all costs incurred in the pursuit of quality or in performing quality related activities Quality cost includes: Prevention cost: quality planning formal technical reviews testing equipment Training

Appraisal cost: Include activities to gain insight into product condition the “first time through” each process. in-process inspection equipment calibration and maintenance testing

17

Cost of Quality Failure cost: internal failure cost: Internal failure cost are incurred when we detect a defect in our product prior to shipment rework Repair failure mode analysis

external failure cost: External failure costs are is associated with defects found after the product has been shipped to customer. complaint resolution product return and replacement help line support warranty work The relative cost to find and repair a defect increases dramatically as we go from prevention to detection to internal failure to external failure cost. (See fig: Relative cost of correcting an error)

18

SQA Activities Prepare SQA plan for the project. Participate in the development of the project's software process description. The software team selects a process. The SQA group reviews Review software engineering activities to verify compliance with the description for compliance with organizational the process defined software process. policy, internal standards, externally imposed The groupsoftware identifies, documents, tracks deviations Audit SQA designated software work products to and verify compliance with those defined as part of the software process. Standards (E.g. ISO-9001), and other parts of have the software from the process and verifies that corrections been Ensure that any deviations in software or work products are project plan. made. The SQA group identifies, documents, and tracksprocedure. deviations documented and handled according to a documented Recordthe any evidence of verifies noncompliance and reportshave thembeen to from process and that corrections management.

The plan identifies: •Evaluation to be performed. •Audits and reviews to be performed. •Standards that are applicable to the project. •Procedures for error reporting and tracking. •Documents to be produced by the SQA group. •Amount of feedback provided to the software Made; and periodically reports the result of its work to the project team.

project manager. Deviation may be encountered in the project plan, process description, applicable standards, or technical work products. Non-compliance items are tracked until they are resolved. 19

Software Reviews Objective: find errors early before they become defects. Excellent ROI: empirical data show that design activities introduce 50-65 % of errors, but formal reviews can detect up-to 75 % of those. Reviews are made to Uncover errors. Ensure that software meets requirements and standards. Ensure that software and the process is manageable.

Using formal technical reviews (walkthroughs or inspections) is an effective means for improving software quality.

20

Review Roles Presenter (designer/producer). Coordinator (not person who hires/fires). Recorder records events of meeting builds paper trail

Reviewers maintenance oracle (vision) standards bearer user representative others 21

Reviews (Walkthroughs) In the review process, the producers present a product (part of the software, e.g. a code module or a design model) which is then reviewed by reviewers. Records should be kept about the reviews. This includes lists of issues discovered. A follow-up procedure must be in place to correct issues.

22

Defect Amplification Models Defect amplification models can be used to illustrate the importance of detecting defects in an early stage. They are a simple mathematical model for describing how defects are propagated through the various steps in the SD process.

23

Defect amplification, no review

Defect amplification, Reviews conducted

24

Defect Causes IES: Incomplete and erroneous specification. Example: customer "forgets" an interface to an existing system. MCC: Misinterpretation of customer communication. Example: Customer means one thing, analyst understands something else. IDS: Intentional derivation from specification. Example: Analyst or programmer things he knows better. VPS: Violation of Programming Standards. Example: Tools are used which rely on naming patterns (e.g. bean tools), but the code does not adhere to those patterns. EDR: Error in Data Representation. Example: wrong container types are used, e.g. containers which support duplicates, but this is not permitted in the business logic. ICI: Inconsistant Component Interface. Example: component interface does not support all parameters needed to customize the component.

25

Defect Causes EDL: Error in design logic. Example: wrong algorithm implemented. IET: Incomplete and erroneous testing. Example: there are errors in the test cases itself, and the code is correct w.r.t. the incorrect test cases. IID: Inaccurate and incomplete documentation. Example: the meaning of an API method is incorrectly (or not) documented, and programmers in a team using this component pass the wrong parameter to this component. PLT: Error in PL translation of design. Example: classes design to be abstract are implemented as concrete classes and instantiated. HCI: Ambiguous or inconsistent human computer interface. Example: different meaning of the same key stroke or menu function in different contexts. MIS miscellaneous. 26

Formal Technical Reviews Involves 3 to 5 people (including reviewers) Advance preparation (no more than 2 hours per person) required Duration of review meeting should be less than 2 hours Focus of review is on a discrete work product Review leader organizes the review meeting at the producer's request.

27

Formal Technical Reviews Reviewers ask questions that enable the producer to discover his or her own error (the product is under review not the producer) Producer of the work product walks the reviewers through the product Recorder writes down any significant issues raised during the review Reviewers decide to accept or reject the work product and whether to require additional reviews of product or not. 28

Why do peer reviews? To improve quality. Catches 80% of all errors if done properly. Catches both coding errors and design errors. Enforce the spirit of any organization standards. Training and insurance. 29

Review Guidelines Keep it short (< 30 minutes). Don’t schedule two in a row. Don’t review product fragments. Use standards to avoid style disagreements. Let the coordinator run the meeting and maintain order. 30

SQA Plan Management section describes the place of SQA in the structure of the organization

Documentation section describes each work product produced as part of the software process

Standards, practices, and conventions section lists all applicable standards/practices applied during the software process and any metrics to be collected as part of the software engineering work 31

SQA Plan Reviews and audits section provides an overview of the approach used in the reviews and audits to be conducted during the project

Test section references the test plan and procedure document and defines test record keeping requirements 32

SQA Plan Problem reporting and corrective action section defines procedures for reporting, tracking, and resolving errors or defects, identifies organizational responsibilities for these activities

Other tools, SQA methods, change control, record keeping, training, and risk management

33