Validation And Verification

Domain 6

Verification and Validation

Verification Concepts “Verification concepts is the understanding of principles, rationale, rules, participant roles and the psychologies of various techniques used to evaluate systems during development.”

1

CBOK Domain 6

Validation Concepts “Validation typically involves actual testing and takes place after verifications are completed.” www.Softwareqatest.com

2

Verification Techniques Audits • An independent assessment of a project • To verify whether or not the project is in compliance with appropriate policies, procedures, standards, contractual specifications • An audit may include operational aspects of the project

Verification Techniques Reviews and Inspections To determine whether or not to continue development •

• To identify defects in the product early in the life cycle.

Reviews and Inspections Types of Reviews 1. In-Process Reviews 2. Milestone Reviews (also called) Decision-point/Phase-end Reviews 4. Test Readiness Review 5. Test Completion Review 3. Post Implementation Reviews (also called) Post Mortem NOTE: CBOK has Test Readiness Review and Test Completion Review inside Decision-point/Phase-end Reviews and as separate types of reviews. – Replaced Milestone Reviews with Decisionpoint/Phase-end reviews

Types of Reviews In-Process Assess progress towards requirements •

• During a specific period of the development cycle – like design period • Limited to a segment of the product • Used to find defects in the work product and the work process • Catches defects early – where they

Types of Reviews Decision-point & Phase-end • Review of products and processes near the completion of each phase of development • Decisions for proceeding with development are based on cost, schedule, risk, progress, readiness for next phase • Also referred to as Milestone Review • Contains Requirements, Critical

Types of Reviews Decision-point & Phase-end Software Requirements Review • Requirements documented • Baseline established • Analysis areas identified • Software development plan • Test plan • Configuration management plan derived

Types of Reviews Decision-point & Phase-end Critical Design Review • Baselines the detailed design specification • Test cases are reviewed and approved • Usually, coding will begin at the close of this phase.

Types of Reviews Decision-point & Phase-end Test Readiness Reviews • Performed when the appropriate modules are near completion •

Determines whether or not testing should progress based on a review of entrance and exit criteria

• Determines the readiness of the application/project for system and acceptance testing

Types of Reviews Decision-point & Phase-end Test Completion Reviews • Determine the state of the software product

Types of Reviews Decision-point & Phase-end Important Notes • Completion of phase-end review signals beginning of next phase • In iterative development methodologies each analysis/design “package” or segment of the application may be in different phases simultaneously • Must ensure iterations are

Types of Reviews Post Implementation Reviews • Also known as “Postmortems” Review/evaluation of the product that includes planned vs. actual development results and compliance with requirements •

• Used for process improvement of software development •

Reviews and Inspections Classes of Reviews 1. Informal Review 2. Semiformal Review 3. Formal Review

Classes of Reviews Informal • Also called peer-reviews • Generally one-on-one meeting between author of a work product and a peer • Initiated as a request for input • No agenda • Results are not formally reported • Occur as needed through out each phase

Classes of Reviews Semiformal • Facilitated by the author • Presentation is made with comment at the end • Presentation is made with comment made throughout • Issues raised are captured and published in a report distributed to participants

Classes of Reviews Formal • Facilitated by a moderator (not author) • Moderator is assisted by a recorder • Defects are recorded and assigned • Meeting is planned • Materials are distributed beforehand • Participants are prepared- their preparedness dictates the effectiveness of the review

Classes of Reviews Formal -

continued

• Full participation by all members of the reviewing team is required • A formal report captures issues raised and is distributed to participants and management • Defects found are tracked through the defect tracking system and followed through to resolution

Review Rules 1. The product is reviewed, not the producer 2. Defects and issues are identified, not corrected 3. All members of the reviewing team are responsible for the results of the review

Review Notes “Stage Containment”: identifying defects in the stage in which they were created, rather than in later testing stages. Reviews are generally greater than 65% effective Testing is often less than 30% effective The earlier defects are found the less expensive they are to correct In addition to learning about a specific product/project, team members are exposed to a variety of approaches to technical issues Provides training in and enforces the use of

Participant Roles Management of V & V 1. Prepare the plans for execution of the process 2. Initiate the implementation of the plan 3. Monitor the execution of the plan

Participant Roles Management of V & V 4. Analyze problems discovered during the execution of the plan 5. Report progress of the processes 6. Ensure products satisfy requirements 7. Assess evaluation results 8. Determine whether a task is complete 9. Check the results for completeness

V & V Manager Role Tasks and Responsibilities •

Create the Software Verification and Validation plan

• Baseline Change Assessment • Conduct Management Review of V &V • Management and Technical Review Support • Interface with Organizational and Supporting Processes

V & V Manager Role Software Dev. V & V Activities 1. Conceptualization 2. Requirements Analysis 3. Design 4. Coding 5. Integration 6. Testing 7. Installation 8. Acceptance

Software Dev. Activities Software Dev. V & V Common Tasks • Criticality analysis

• Traceability analysis • Hazard analysis • Risk Analysis

Software Dev. Activities Conceptualization • Define a specific implementation solution to solve the user’s needs • Architect selected • System requirements defined and packaged for analysis • Objectives: to verify the allocation of system requirements, validate the selected solution, and ensure that no false assumptions were made

Software Dev. Activities Conceptualization - Unique Tasks

• Evaluate Concept Documentation • Analyze Hardware/Software/User Requirements Allocation

Software Dev. Activities Analysis • Refine and analyze the functional and performance requirements • Interfaces external to the software • Qualification requirements • Safety and security requirements • Human factors engineering • Data definition • User documentation for the software

Software Dev. Activities Analysis -

continued

• User operation and execution requirements • User maintenance requirements • Object Oriented – define use cases • Object Oriented – class diagram created • Object Oriented – sequence diagrams • Objectives: Ensure the correctness, completeness, accuracy, testability and

Software Dev. Activities Analysis – Unique Tasks • Software Requirements Evaluation • Interface Analysis • Acceptance Test Procedure Generation and Verification • System Test Plan Generation and Verification • Acceptance Test Plan Generation and Verification • Configuration Management

Software Dev. Activities Design • Address software architectural design • Address software detailed design • Includes databases • Includes interfaces (external to the software, between components and between software units) • Objectives: to demonstrate that the design is correct, accurate, complete transformation

Software Dev. Activities Design – Unique Tasks • Software Design Evaluation • Component Test Plan Generation and Verification • Integration Test Plan Generation and Verification • Test Design Generation and Verification

Software Dev. Activities Implementation • Transform design into code • Transform design into database structures • Transform design into related machine executable representations • Addresses software coding • Addresses unit testing • Objectives: verify and validate that the

Software Dev. Activities Implementation – Unique Tasks • Source Code and Source Code Documentation Evaluation • Interface Analysis • Test Case Generation and Verification • Test Procedure Generation and Verification • Component (unit) Test Execution and Verification

Software Dev. Activities Testing • Includes component integration • Includes system testing • Includes system integration • Includes user acceptance testing • Objectives: ensure that the functional and supplemental requirements are satisfied by execution of integration, system, and acceptance tests.

Software Dev. Activities Testing – Unique Tasks • Integration Test Execution and Verification • System Test Execution and Verification • Acceptance Test and Verification

Software Dev. Activities Installation & Checkout • Includes the installation of the software product in the target environment • Includes the acquirer’s (user’s) acceptance review and/or testing • Objectives: to verify and validate the correctness of the software installation in the target environment

Software Dev. Activities Installation & Checkout – Unique Tasks • Installation Configuration Audit • Installation Checkout • V & V Final Report Generation

Software Dev. Activities Operation • Support the use of the software by the end user in an operational environment • Addresses Operational testing, • System operation • And user support • Objectives: to evaluate new constraints on the system, assess proposed changes and

Software Dev. Activities Operation – Unique Tasks • Evaluate New Constraints • Proposed Change Assessment • Operation Procedures Evaluation

Software Dev. Activities Maintenance • Covers modifications (corrective, adaptive, perceptive) • Address migration -The movement of software to a new operating system • Address retirement of software -The withdrawal of support by the operation and maintenance organization -Partial or total replacement by a

Software Dev. Activities Maintenance - continued • Address problem and modification analysis • Address modification implementation • Address maintenance review/acceptance • Objectives: to assess proposed changes and their impact on the software, evaluate anomalies that are discovered during operation, assess migration

Software Dev. Activities Maintenance – Unique Tasks • Software V & V Plan Revision(s) • Proposed Change Assessment • Anomaly Evaluation • Migration Assessment • Retirement Assessment • Task Iteration

V & V Manager Role Acquisition V & V • Necessary if your project includes the purchase or contract development software • The V & V Plan must include tasks to address the acquisition and integration of that software into your environment

Acquisition V & V Acquisition Activities • Address project initiation • Address request for proposal • Address contract preparation • Address supplier monitoring • Address acceptance and completion • Objectives: to scope the V & V effort, plan interfaces with the supplier and acquirer, and review the draft systems

Acquisition V & V Acquisition Tasks • Scoping the V & V effort • Planning the Interface between the V &V Effort and the Supplier • System Requirements Review

V & V Manager Role Supply V & V • Necessary if you are developing custom applications or components for an external customer • Initiated by a decision to prepare a response to a request for proposal • Initiated by signing and entering into a contract to provided the system, software

V & V Manager Role Supply V & V - continued • Determination of procedures and resources required • Verify that the request for proposal requirements and contract requirements are consistent with customer needs • Uses the contract requirements and program schedules to revise and update the interface planning between the

Supply V & V Supply Activities • Addresses the initiation • Addresses the preparation of response • Addresses contract planning • Addresses execution and control • Addresses review and evaluation • Addresses delivery and completion activities

Supply V & V Supply Tasks • Planning the Interface between the V &V Effort and Supplier • Contract Verification

V & V Manager Role Reporting • Consists of task reports, activity reports, anomaly reports and the final report • Reports are provided as feedback to the software development process regarding the technical quality of each software product.

Reporting Base-set of V & V reports • Task Reports • V & V Activity Summary Reports • Anomaly Report • V & V Final Report • Optional Reports (example: special study)

V & V Summary Emphasis on Verification Concepts - definition Techniques Stressed Reviews & Inspections – types & classes In-Process Milestone Test Readiness Test Completion Post-Implementation Informal Semi-formal Formal

V & V Summary Emphasis on Verification Roles for Verification & Validation Focus on V & V Manager - activities Software Development Acquisition Supply Reporting

V & V Summary Things to Remember The earlier in the software development cycle, defects are detected, the easier and cheaper they are to fix. Over 65 % of defects are found in Verification Reviews Less than 30 % of defects are found in the testing phase

V & V Summary Things to Remember The product is reviewed, not the producer Defects and issues are identified, not corrected All team members of the review are responsible for the results of the review

V & V Summary Mentioned Validation Concepts – definition (provided by an outside source)

Mentioned Audits Concepts - definition