Software Metrics Faculty Prof.V.Narayana K.S.Prakasa Rao P.Linga Reddy
Software Metrics - Topics ¾
Why measure
¾
What to measure
¾
When to measure
¾
Data collection
¾
Metric analysis
¾
Simple statistical tools
¾
Improvements
¾
Issues
Where Ideas Drive IT
© Ideaspace Solutions Limited All Rights Reserved
Slide 2 of 43 Version:1.0 Date :05/08/2002
Why measure ? Software project goals quality of deliverables timely delivery and cost If you do not measure, you do not know
where you are and do not know what to improve
Where Ideas Drive IT
© Ideaspace Solutions Limited All Rights Reserved
Slide 3 of 43 Version:1.0 Date :05/08/2002
Why measure ? Need Quantitative Data to monitor progress to analyze trends to identify areas that need improvement to initiate improvements Continuous improvement
Where Ideas Drive IT
© Ideaspace Solutions Limited All Rights Reserved
Slide 4 of 43 Version:1.0 Date :05/08/2002
Why measure ? Measurement - A quantified observation For eg., effort in person days in completing a task or number of defects reported in testing a module Metric - A comparative measure of the performance of software product or process For e.g.., Effort deviation percentage (planned versus actual)
Defect density = defects per unit size of work product.
Where Ideas Drive IT
© Ideaspace Solutions Limited All Rights Reserved
Slide 5 of 43 Version:1.0 Date :05/08/2002
What to measure ? Project management metrics Schedule metrics Effort metrics Phase-wise effort distribution Quality metrics Defect metrics Customer satisfaction Maintenance metrics
Where Ideas Drive IT
© Ideaspace Solutions Limited All Rights Reserved
Slide 6 of 43 Version:1.0 Date :05/08/2002
What to measure ? (cont’d) Cost metrics Cost variance Cost of Quality
Where Ideas Drive IT
© Ideaspace Solutions Limited All Rights Reserved
Slide 7 of 43 Version:1.0 Date :05/08/2002
Schedule metrics To monitor the trends in the actual calendar days
spent on completion of specific tasks/ activities against planned calendar days
Development projects - phase wise
measurements
Sustenance projects - work request wise
measurements
Used for re-planning at the project level and as
input for estimation at the company level
Where Ideas Drive IT
© Ideaspace Solutions Limited All Rights Reserved
Slide 8 of 43 Version:1.0 Date :05/08/2002
Schedule metrics (cont’d) Schedule deviation = ((Actual calendar days Planned calendar days)/Planned calendar days) * 100
Where Ideas Drive IT
© Ideaspace Solutions Limited All Rights Reserved
Slide 9 of 43 Version:1.0 Date :05/08/2002
Effort metrics To monitor the trends in the person days spent
on completion of specific tasks/ activities against planned person days
Development projects - phase wise
measurements
Sustenance projects - work request wise
measurements
Used for re-planning at the project level and as
input for estimation at the company level
Where Ideas Drive IT
© Ideaspace Solutions Limited All Rights Reserved
Slide 10 of 43 Version:1.0 Date :05/08/2002
Effort metrics (cont’d) Effort deviation = ((Actual person days Planned person days)/ Planned person days) * 100
Where Ideas Drive IT
© Ideaspace Solutions Limited All Rights Reserved
Slide 11 of 43 Version:1.0 Date :05/08/2002
Phase-wise effort distribution Analysis of effort spent on each phase for
different kinds of projects
Comparison with size of work product at
each phase
Provides norms at the company level for preparation of proposals for project planning
Where Ideas Drive IT
© Ideaspace Solutions Limited All Rights Reserved
Slide 12 of 43 Version:1.0 Date :05/08/2002
Defect metrics To monitor trends in defects at each
phase of development as well as during acceptance by the customer
Classification by severity of defect and
origin of defect helps in initiating corrective actions
Where Ideas Drive IT
© Ideaspace Solutions Limited All Rights Reserved
Slide 13 of 43 Version:1.0 Date :05/08/2002
Defect metrics (cont’d) Defect density No of defects/size of the work product Severity of defects classification of defects as critical, major & minor
and their distribution at each phase
Origin of defects classification of defects based on the origin of
defects such as requirements, design or coding
Where Ideas Drive IT
© Ideaspace Solutions Limited All Rights Reserved
Slide 14 of 43 Version:1.0 Date :05/08/2002
Defect metrics (cont’d) Defects seepage Defects reported by customer during
acceptance as a percentage of total defects reported during pre-delivery system testing
Where Ideas Drive IT
© Ideaspace Solutions Limited All Rights Reserved
Slide 15 of 43 Version:1.0 Date :05/08/2002
Customer satisfaction Based on customer satisfaction survey, analyze
the data and identify areas of strengths and weakness
Identify causes that result in customer
dissatisfaction and take corrective actions
Where Ideas Drive IT
© Ideaspace Solutions Limited All Rights Reserved
Slide 16 of 43 Version:1.0 Date :05/08/2002
Maintenance metrics Analysis of defects reported during warranty
support
Mean time taken to resolve problems reported by
the customer
Where Ideas Drive IT
© Ideaspace Solutions Limited All Rights Reserved
Slide 17 of 43 Version:1.0 Date :05/08/2002
Cost metrics Cost variance =
((Actual cost - Planned cost)/Planned cost) * 100
Cost of Quality Prevention cost Appraisal cost Failure cost (Internal failures and external failures)
Where Ideas Drive IT
© Ideaspace Solutions Limited All Rights Reserved
Slide 18 of 43 Version:1.0 Date :05/08/2002
When to measure Periodic - on a time bound basis (daily or
weekly)
Milestone - on completion of a specific
mile stone
Where Ideas Drive IT
© Ideaspace Solutions Limited All Rights Reserved
Slide 19 of 43 Version:1.0 Date :05/08/2002
Data collection Planned start date and close date for
each phase
Actual start date and close date for each
phase
Planned effort for each phase Actual effort for each phase Defects at each phase of review and
testing
Where Ideas Drive IT
© Ideaspace Solutions Limited All Rights Reserved
Slide 20 of 43 Version:1.0 Date :05/08/2002
Data collection (cont’d) Customer satisfaction survey data Customer feedback and complaints data Sustenance projects - schedule and
effort (planned and actual) for each work request
Where Ideas Drive IT
© Ideaspace Solutions Limited All Rights Reserved
Slide 21 of 43 Version:1.0 Date :05/08/2002
Metric analysis Schedule variance = ((Actual calendar days -
Planned calendar days)/ Planned calendar days)) * 100
Effort variance = ((Actual effort in person days -
Planned effort in person days)/ Planned effort in person days)) * 100
Effort distribution metric
spent for each phase
Where Ideas Drive IT
= Percentage of effort
© Ideaspace Solutions Limited All Rights Reserved
Slide 22 of 43 Version:1.0 Date :05/08/2002
Metric analysis (cont’d) Defect metrics Defect density = Number of defects/size of
the work product
Defect distribution = Percentage of defects by
defect severity and origin of defects
Defects seepage = Defects reported during
acceptance as a percentage of defects reported during internal testing
Where Ideas Drive IT
© Ideaspace Solutions Limited All Rights Reserved
Slide 23 of 43 Version:1.0 Date :05/08/2002
Metric analysis (cont’d) Customer satisfaction survey Customer satisfaction survey is conducted
quarterly/mile-stone for large projects and end of project for small projects
Parameters measured - Quality, Delivery, Service,
Communication, Relationship and Value
Rating for each parameter -
5 Excellent, 4 Fully satisfied, 3 Average, 2 Fair and 1 Poor
Where Ideas Drive IT
© Ideaspace Solutions Limited All Rights Reserved
Slide 24 of 43 Version:1.0 Date :05/08/2002
Metric analysis (cont’d) Customer satisfaction survey (cont’d) Each parameter to have a score of 3 plus and
an overall score of more than 70%
Analyzed for each customer and plotted for
score of each parameter and total score
Where Ideas Drive IT
© Ideaspace Solutions Limited All Rights Reserved
Slide 25 of 43 Version:1.0 Date :05/08/2002
Metric analysis (cont’d) Maintenance Metrics
Analysis of Time taken to respond to customer reported defects
Analysis of type of defects by type and severity
Where Ideas Drive IT
© Ideaspace Solutions Limited All Rights Reserved
Slide 26 of 43 Version:1.0 Date :05/08/2002
Metric analysis (cont’d) Cost Metrics
Cost variance = ((Actual cost - Planned cost)/ Planned cost)*100
Cost of Quality Prevention cost (Planning, training etc.,) Appraisal cost (Reviews and testing) Failure cost (Rework, customer complaints etc.,)
Where Ideas Drive IT
© Ideaspace Solutions Limited All Rights Reserved
Slide 27 of 43 Version:1.0 Date :05/08/2002
Simple statistical tools Scatter diagram Histogram Pareto Analysis Brainstorming Cause and effect diagrams Control charts
Where Ideas Drive IT
© Ideaspace Solutions Limited All Rights Reserved
Slide 28 of 43 Version:1.0 Date :05/08/2002
Scatter diagram Discover, confirm or display
relationships between two sets of data
Where Ideas Drive IT
© Ideaspace Solutions Limited All Rights Reserved
Slide 29 of 43 Version:1.0 Date :05/08/2002
Histogram Display the dispersion or spread of data
Where Ideas Drive IT
© Ideaspace Solutions Limited All Rights Reserved
Slide 30 of 43 Version:1.0 Date :05/08/2002
Histogram
Where Ideas Drive IT
© Ideaspace Solutions Limited All Rights Reserved
Slide 31 of 43 Version:1.0 Date :05/08/2002
Pareto Analysis Identify major factors and distinguish the
most important causes from less significant ones.
Where Ideas Drive IT
© Ideaspace Solutions Limited All Rights Reserved
Slide 32 of 43 Version:1.0 Date :05/08/2002
Pareto Analysis (cont’d)
Where Ideas Drive IT
© Ideaspace Solutions Limited All Rights Reserved
Slide 33 of 43 Version:1.0 Date :05/08/2002
Brainstorming Generate, clarify and evaluate a sizeable
list of ideas, problems or issues
Where Ideas Drive IT
© Ideaspace Solutions Limited All Rights Reserved
Slide 34 of 43 Version:1.0 Date :05/08/2002
Cause and effect diagram Systematically analyze cause and effect
relationships and identify potential root causes of a problem.
Where Ideas Drive IT
© Ideaspace Solutions Limited All Rights Reserved
Slide 35 of 43 Version:1.0 Date :05/08/2002
Cause and effect diagram (cont’d)
Where Ideas Drive IT
© Ideaspace Solutions Limited All Rights Reserved
Slide 36 of 43 Version:1.0 Date :05/08/2002
Control charts Every process is subject to variation Variation is part of nature Causes of variation Natural causes Assignable causes Natural Causes Cannot be avoided but can be improved Assignable causes Can be avoided Where Ideas Drive IT
© Ideaspace Solutions Limited All Rights Reserved
Slide 37 of 43 Version:1.0 Date :05/08/2002
Control charts (cont’d) Control charts help us in identifying
assignable causes and to eliminate them
Where Ideas Drive IT
© Ideaspace Solutions Limited All Rights Reserved
Slide 38 of 43 Version:1.0 Date :05/08/2002
Control charts (cont’d) Steps in control charts Define company baseline for metrics that need to be
controlled Work out the upper and lower control limits Plot the metrics on a periodic or milestone basis In case metric out control identify the case and initiate action to eliminate the cause
Where Ideas Drive IT
© Ideaspace Solutions Limited All Rights Reserved
Slide 39 of 43 Version:1.0 Date :05/08/2002
Improvements Corrective actions Preventive actions Estimation, planning & tracking methods Goal setting - company level and project
level
Where Ideas Drive IT
© Ideaspace Solutions Limited All Rights Reserved
Slide 40 of 43 Version:1.0 Date :05/08/2002
Issues
Data collection - Manual methods (cumbersome and do not assure authentic data)
Automate the data collection
Identify centralized department (like QA) for analysis and reporting
Share information among all
Metrics are for process improvements and not for evaluation of persons
Where Ideas Drive IT
© Ideaspace Solutions Limited All Rights Reserved
Slide 41 of 43 Version:1.0 Date :05/08/2002
Issues (cont’d)
Keep the metrics process SIMPLE and PRACTICAL
Openness
Communication and preparedness to improve
Report metrics in ERC and MRC
Where Ideas Drive IT
© Ideaspace Solutions Limited All Rights Reserved
Slide 42 of 43 Version:1.0 Date :05/08/2002
Thank You