Reviews

Reviews Induction – Sep 2007

Protection notice / Copyright notice Version 2.0

Contents Purpose of Reviews What are Reviews? Benefits from reviews for project roles Types of Reviews Importance of Checklists Importance of Coding Guidelines Roles & Responsibilities Qualities of a Reviewer Review Metrics Tools for reviews

Page 2

Sep-07

P&Q

Protection notice / Copyright notice For Internal Use Only

Objective of Reviews • Detect Defects • Verify compliance to Specifications, Standards, Plans and Procedures • Suggest improvements and alternate solutions • Monitor progress of projects

Page 3

Sep-07

P&Q

Protection notice / Copyright notice For Internal Use Only

Purpose of Reviews

The main purpose of conducting reviews is to • improve quality • increase productivity and • reduce rework

Its purpose is not to tear down the project or the authors

Page 4

Sep-07

P&Q

Protection notice / Copyright notice For Internal Use Only

What are Reviews?

A work product is examined for defects by individuals other than the person who produced it. Reviews are considered the testing of non-executable documents.

One should do reviews and traditional testing, if possible, on all deliverable products of the lifecycle, including test plans!

Page 5

Sep-07

P&Q

Protection notice / Copyright notice For Internal Use Only

Why Reviews?

The effort to detect an obvious bug by manual reading is much much less than the effort in designing the test case to detect the same Author Anonymous

Page 6

Sep-07

P&Q

Protection notice / Copyright notice For Internal Use Only

Benefits from Reviews for Developer

Developer

 Less time spent performing rework/Increased productivity  Confidence that the right requirements are being implemented  Better techniques learned from other developers  Reduced unit testing and debugging time & less debugging during IT/ST  Exchange of information about components & overall system with other team members Page 7

Sep-07

P&Q

Protection notice / Copyright notice For Internal Use Only

Benefits from reviews for Maintenance Team Maintenance Team

 Fewer production support calls, leading to a reduced maintenance backlog  More robust designs that tolerate change  More maintainable and better documented work products that are easy to understand and modify  Better understanding of the product from having participated in design and code reviews during development

Page 8

Sep-07

P&Q

Protection notice / Copyright notice For Internal Use Only

Benefits from reviews for Project Manager Project Manager

 Shortened product development cycle time  Reduced field service and customer support costs, lifetime maintenance costs, freeing resources for new development projects  Improved teamwork (encouraged by technical reviews and walkthrus), collaboration, and development effectiveness  Better and earlier insight into project risks and quality issues  Reduced impact from staff turnover through cross-training of whole team Page 9

Sep-07

P&Q

Protection notice / Copyright notice For Internal Use Only

Benefits from reviews for Quality Manager

Quality Manager

 Infer the testability of product features under development depending on the review data received  Shortened system-testing cycles and less retesting  Use review data when making release decisions  Use data for forecast of quality assurance/review effort needed

Page 10

Sep-07

P&Q

Protection notice / Copyright notice For Internal Use Only

Project Life Cycle

Proposal Requirements Design (URS)

(DFS GIS,DIS)

Reviews

Coding

Testing

(Code)

(Test specs)

White Box Black Box Stress/Load

Maintenance

Problem Reports

Defect Prevention - Feedback and Process adjustments Defect Analysis and Process Improvement

Page 11

Sep-07

P&Q

Protection notice / Copyright notice For Internal Use Only

Review - When & What Proposal stage

Proposal related documents and estimation Project Plan, Quality Plan, Configuration Plan User Requirement Specs & Detailed Functional Specs Detailed Implementation Specs, Component Specs, Design Document Code & Unit Test Specs Integration & System test specs All documents that are updated, Code and all Test specs

Planning phase Requirements phase Design phase Coding phase Testing phase Maintenance phase

Page 12

Sep-07

P&Q

Protection notice / Copyright notice For Internal Use Only

Reviews vs Testing Advantages of Reviews over Testing Early detection of software defects Reviews expose defects, whereas testing shows only the symptom of the defect Reviews expose a batch of defects, whereas it is usually one by one in testing Some defects can be found only by reviews: Code redundancy Dead Code Violations of coding standards Reviews help reduce rework cost (Cost involved during rework is exponentially increased during Testing Phase)

Page 13

Sep-07

P&Q

Protection notice / Copyright notice For Internal Use Only

The Review Cycle VSS 1 Author Creates Document

6 Author Updates Document

2 Sends Document to reviewers for review

Review Document and send LOD in specified timeframe along with time spent

QMS Intranet

3

This cycle is repeated until comments in all LOD’s have been resolved and the Review Report was verified by the PQM

4 Author Prepares Review Report

Yes 5 PQM Verifies Review Report

Any more Deficiencies ?

No 7 Approver verifies changes and approves

Page 14

Sep-07

8 Author notifies reviewers of Approval

P&Q

9 Author/CM Baselines and uploads to PDB

Protection notice / Copyright notice For Internal Use Only

Types of Reviews

Off-Line Review 4-Eyes Review Walkthrough Inspection Collaborative Review (for Agile/Scrum projects)

Page 15

Sep-07

P&Q

Protection notice / Copyright notice For Internal Use Only

Off-line Review • Held when a meeting is not possible • Suitable for small review objects or when reviewers are at different locations

Review Object

List Of Deficiencies

Review report

Author

Input: Software work product to be reviewed, Relevant standards, specifications, plans, procedures

Reviewer

Review Comments Output: Corrected review object, Review Report & LOD Page 16

Sep-07

P&Q

Exit Criteria: Verification by approving authority for release of the software work Protection notice / Copyright notice product For Internal Use Only

4-Eyes Review • Defects are documented in a formal meeting • Corrections verified by approving authority • Suitable for small review objects/ verifying corrections in an earlier review

Review Object

Author

Input: Software work product to be reviewed, Relevant standards, specifications, plans, procedures

Output: Corrected review object, Review Report & LOD Page 17

Sep-07

P&Q

Reviewers

Exit Criteria: Reviewers’ approval for release of the software work product. Protection notice / Copyright notice For Internal Use Only

Walkthrough PM to prepare checklist, if required Review object distributed atleast 3 days in advance Issues are only raised. Alternative solutions NOT discussed. Reviewers decide whether review object is to be released/re-review required Corrections verified by approving authority

Review Object Recorder Author Moderator & Reviewers Output: Review report, Corrected review object Page 18

Sep-07

Input: Software work product to be reviewed, Relevant standards, specifications, plans, procedures P&Q

Exit Criteria: Reviewers’ approval for release of the software work product. Protection notice / Copyright notice For Internal Use Only

Walkthrough (contd.) Follow up The author investigates and addresses the issues raised in the walkthrough report The moderator verifies that all issues are appropriately addressed If re-walkthrough was recommended, then review object is corrected and another walkthrough is planned

Page 19

Sep-07

P&Q

Protection notice / Copyright notice For Internal Use Only

Inspection

Overview meeting

Planning

Preparation

Author, Reader, Moderator, Recorder & Reviewers

Meeting

Rework

• Introduction • Check preparedness

Output: Review report, LOD, Corrected review object

• Reading review object • Defect recording • Review defect list • Decision

Page 20

Sep-07

P&Q

Protection notice / Copyright notice For Internal Use Only

Inspection (contd.) This is a rigorous, formal peer examination PM selects the inspection team Material distributed at least 3 days in advance Offline pre-review is preferred This method does NOT examine alternatives/ stylistic issues Checklist is Mandatory Overview session may be arranged

Page 21

Sep-07

P&Q

Protection notice / Copyright notice For Internal Use Only

Inspection (contd.) Reader interprets the review object Moderator checks the entry criteria Moderator may reschedule meeting if reviewers are not sufficiently prepared Moderator to report inspection data Recorder reports each defect in the defect list At the end the moderator has the defect list reviewed by team for accuracy and completeness Exit Criteria: All of the defects are resolved. Each project should document the criteria ( threshold) for acceptance Page 22

Sep-07

P&Q

Protection notice / Copyright notice For Internal Use Only

Collaborative Review Applicability All Agile/Scrum Projects irrespective of size. Is applicable where all project where all project activities (like design, development, testing) are happening simultaneously For projects using pair programming Method The object is reviewed online Review through telecon or in a meeting (formal/informal) Review comments are recorded in LOD Artifact may be updated as the review progresses

Page 23

Sep-07

P&Q

Protection notice / Copyright notice For Internal Use Only

Review - Common Features Reviewer’s approval may be one of the following: Release, Release after corrections or No release Usage of Checklists Projects to prepare/use existing ones from process database Review completion to be checked by approving authority Metrics Effort data Review Effectiveness Review Yield Defects Classification Root Cause Analysis of review data Review object is under Configuration Management

Page 24

Sep-07

P&Q

Protection notice / Copyright notice For Internal Use Only

Importance of Checklists A checklist helps the Author to check whether review object is complete and conforms to standards Helps maintain consistency A checklist reflects common issues Helps the moderator to structure the review meeting and helps each reviewer to focus on important issues. From time to time, based on causal analysis and feedback, checklists must be modified Perspective-based reviews catch 35% more defects than nondirected reviews [as researched by Barry Boehm]

Page 25

Sep-07

P&Q

Protection notice / Copyright notice For Internal Use Only

Roles & Responsibilities Author  Writes the review object and corrects it after review  Distributes review material  Provide overview tutorial on product ( if necessary )  Act as another reviewer  Must not be defensive Moderator:  Manages the review meeting  Ensures that everyone gets review object  Ensures that solutions are not discussed in the review meeting  Gives everyone scope to speak out  Draws conclusion about a defect  Controls pace of the meeting Page 26

Sep-07

P&Q

Protection notice / Copyright notice For Internal Use Only

Roles & Responsibilities (contd.) Recorder Records errors Records inspection data Reads out list at end of meeting Reader (Inspections only) Not the same as the author Leads the inspection team through the review object Paraphrase while reading Set pace for inspection

Page 27

Sep-07

P&Q

Protection notice / Copyright notice For Internal Use Only

Qualities of a Reviewer Understand the product Review Individually Expert on the relevant standard Unbiased opinion Provide positive feedback

Page 28

Sep-07

P&Q

Protection notice / Copyright notice For Internal Use Only

Communication Factors during Reviews •Discussion is kept within bounds •Not extreme in opinion •Reasonable and calm •Not directed at a personal level •Concentrate on finding defects •Not discuss trivial issues •Not involve oneself in solution hunting •The participants should be sensitive to each other by keeping the synergy of the meeting high Page 29

Sep-07

P&Q

Protection notice / Copyright notice For Internal Use Only

Guidelines for Re-review/Re-Walkthrough Quality Goal – Review Yield is not as per norm (deviation is high) Review team finds critical defects or few major defects or too many minor defects Number of open issues in review object is very high (hence review object is then considered as incomplete) In special cases, if too many defects are reported during UT/IT, code review may be redone.

Page 30

Sep-07

P&Q

Protection notice / Copyright notice For Internal Use Only

Common Causes of Ineffective Reviews •Checklist not understood ( no project specific training on checklist) •No analysis on defect data ( hence no chance for improvement) •Reviews are done under time pressure ( e.g. code reviews done just before release) •No activity on defect prevention •Review results are not discussed among project members (same mistakes are repeated by all ) •Disagreement between author and reviewers

Page 31

Sep-07

P&Q

Protection notice / Copyright notice For Internal Use Only

What to look for during code review – (I) Completeness of code  Ensure the code is a complete and precise implementation of the design  Ensure that there are no unreferenced or undefined variables, constants, or data types

 Correctness of code  Ensure all variables are properly declared, initialized and used  Ensure the algorithm is logically correct  Ensure the loop termination conditions are properly specified  Ensure errors are properly handled  Ensure error messages / exception condition messages are correctly framed  Ensure resources and memory are released in all error paths  Ensure boundary conditions (lower and upper limit) are met. Page 32

Sep-07

P&Q

Protection notice / Copyright notice For Internal Use Only

What to look for during Code Review – (II)  Efficiency

of code

 Ensure the code does not have an adverse impact on size, speed, or memory use  Ensure the algorithm is efficient  Compliance  Ensure project and/or organization’s standard (coding guideline) is followed  Consistency

 Ensure the same format, convention, and structure is used throughout

Note: Static Analyzer Tools ensure compliance and consistency. Page 33

Sep-07

P&Q

Protection notice / Copyright notice For Internal Use Only

What to look for during code review – (III)  Robustness

 Ensure the code protects against detectable runtime errors. For example: Range array index values, Division by zero, Out of range variable values etc

 Structuredness  Ensure each function of the program is recognizable as a block of code  Traceability  Ensure the code contains or references a revision history of all code modifications and the reason for them  Ensure there is a cross-reference framework through which the code can be easily and directly traced to the software design documentation Page 34

Sep-07

P&Q

Protection notice / Copyright notice For Internal Use Only

What to look for during code review – (IV)

 Understandability  Ensure consistent formatting techniques (e.g., indentation, use of white space) are used to enhance clarity.  Ensure the naming reflects the type of variable  Maintainability  Ensure there are adequate comments in the code  Ensure there is no hard coding

Page 35

Sep-07

P&Q

Protection notice / Copyright notice For Internal Use Only

What to look for during code review – (V)  Scalability:  Ensure that the code is not a bottleneck that prevents the system from growing to accommodate increased load, data, users or inputs  Reusability  Ensure that the code be reused in other part of applications / other applications  Security  Ensure that code is not vulnerable to unauthorized access Page 36

Sep-07

P&Q

Protection notice / Copyright notice For Internal Use Only

Importance of Coding Guidelines

 Standardizes pattern of coding thus ensures coding consistency by developers  Reduces coding time  Helps in maintainability of code  Lessons learnt from past experience goes into the guidelines – prevents coding defects and helps code optimization.  Improves readability and thus decreases review time  Org. Level guidelines tailored to suit project specific requirements (Project Level guidelines)

Page 37

Sep-07

P&Q

Protection notice / Copyright notice For Internal Use Only

Code Review Checklists  As per studies conducted by Barry Boehm and Victor Basisli - Perspective-based reviews catch 35% more defects than non-directed reviews Checklist are used for code reviews to reflect common coding issues:

Logic and Correctness Completeness Reliability, Portability and Consistency Traceability Maintainability Code Review Checklist - c++

Page 38

Sep-07

P&Q

Protection notice / Copyright notice For Internal Use Only

Review Defects

LOD

Document and Code Review defects are captured in the List Of Deficiencies (LOD) document. Review Report (RR) is prepared an an overall report for one review object. LOD contains details on the Type of Defect, Description of defect, Injected Phase, Remarks and Status of Closure Types of Defects may be Minor, Major or Suggestions RR contains data on Reviewers, Effort Spent and Review Metrics Review Report needs to be verified once the defects are closed.

Review Report

Page 39

Sep-07

P&Q

Protection notice / Copyright notice For Internal Use Only

Review Metrics Review Yield - Number of defects found during document reviews Example: Goal = 0.3 Defects/Page SD 0.2 (range 0.1 - 0.5) Review Effectiveness - Number of defects found during reviews (both document and code) to Total Defects Example: Goal = 50% SD 10% (range 40% - 60%)

Page 40

Sep-07

P&Q

Protection notice / Copyright notice For Internal Use Only

Tools for review

For code – Static Code Checkers may be used on compiled code E.g. of Tools PCLint, RSM For documents – eReviewer (in house developed tool for reviewing on softcopy & generating LOD and review report)

Page 41

Sep-07

P&Q

Protection notice / Copyright notice For Internal Use Only

Quiz

Review Quiz

Review Quiz with Answers

Page 42

Sep-07

P&Q

Protection notice / Copyright notice For Internal Use Only

Contents Purpose of Reviews What are Reviews? Benefits from reviews for project roles Types of Reviews Importance of Checklists Importance of Coding Guidelines Roles & Responsibilities Qualities of a Reviewer Review Metrics Tools for reviews

Page 43

Sep-07

P&Q

Protection notice / Copyright notice For Internal Use Only

Questions?

Page 44

Sep-07

P&Q

Protection notice / Copyright notice For Internal Use Only

Thank You

QMS Intranet – //132.186.193.50

Page 45

Sep-07

P&Q

Protection notice / Copyright notice For Internal Use Only