Failure 1

Troubled Software Projects: War Stories and Lessons Learned Robert L. Glass Computing Trends [email protected]

My Credentials for Failure(!) • Two books published by Prentice-Hall • Four books self-published • You might say that I’m building my success on a foundation of failure!

An Overview of Failure • My biased view – There are far fewer failures than claimed – Most failures occur on huge projects – Most failures are caused by • Unstable requirements • Impossible schedules . Hyped technology promises

An Overview of Failure (2) • Intense debate: – This is either • The “software crisis” era • The “computing era,” with software fueling it

Outline for this Talk • Two runaway stories (an insider’s view of spectacular failures) – Denver Airport Baggage Handling system – Westpac Bank CS90

• One more typical 21st century story – Company S

• Tatting up loose ends

Denver Airport • Source: Prof. Ramiro Montealegre, U / CO – Originally published in Harvard Business Review

• The Problem: provide an automated baggage handling system for the new Denver airport

Denver Airport (2) • What went wrong (total project): – Highly political • Seen as a jobs creation project • Built by a committee

– “Build/design” project

• What went wrong (software): – Original design United only, then expanded – Froze requirements, then modified – Contractor said 1 year, didn’t change for mods

Denver Airport (3) • What happened: – Delays, delays, delays … with press watching – Couldn’t come even close to estimates – Forced delays in opening the airport itself! – Eventually: 18 months, $2B over budgets – Lawsuits galore

Denver Airport (4) • Parallel (2005): – Bombardier backed out of Taiwan airport rail project because the requirements were impossible to meet, couldn’t guarantee safety

• Lessons learned: – Politics and technology don’t mix – If requirements change, planning must change

Westpac Bank • Source: Robert L. Glass + Chris Craddock • The Problem: – Update the bank’s information systems • “Core System Development” – CS90 • Would market to other banks

Westpac Bank (2) • What went wrong: – IS people too busy, brought in consultants and academics (no banking experience) – Produced “brochureware” before software – Proposed every buzzword in the book: • • • . •

“CS90” engine” to generate application code building blocks “object-orientation” with “supertypes” knowledge engineering techniques expert systems CASE tools

Westpac Bank (3) • What happened: – No one knew how to do the buzzword things – Went ahead anyway! – Very little progress, very little product – Project leaders began lying to management • Release 4.2 followed 2.0! • Code listed as complete that didn’t exist • Whistle blower told to forget it

Westpac Bank (4) • What happened (concluded): – Total loss – nothing ever worked – Cost over $150M – 500 positions terminated – Bank fell from first to last in rankings

Westpac Bank (5) • Lessons Learned: – Domain knowledge essential to success – Hyped technology leads to ridiculous promises – The bigger they come, the harder they fall

• Parallel (2005): – Qantas has 700 applications more than 30 years old running on Sperry Univac equipment that it needs to rewrite – They’re not the only ones!

Company S • Source: Robert L. Glass + David Glass • The previous stories are anomalies – Typical of runaways, not the norm – The following story is more the norm

Company S (2) • The problem: the software support organization for a hardware device company is mired down – Error-plagued code – Testers, not programmers, finding errors – Two steps forward, one step back – Obscure design, fragile code

Company S (3) • What went wrong: – Impossible schedules, severe schedule pressure – Design poorly understood – Code “30-90% bad” – No sense of code ownership – Too many regression errors

Company S (4) • What happened: – Called in consultants – Were honest about problems – Wanted desperately to improve

Company S (5) • What consultants did: – Suggested PROCESS changes (not what they expected to do!) • • • •

Reconcile design, code – clean up code Improve error removal (inspections) Institute life cycle (rqmnts, design, code, test) Establish a methods/tools /processes org

Company S (5) • The results: – Most suggestions followed – “Amazing results,” but later, backsliding – Process changes caused the programmers to take back the responsibility for doing the job properly – Reworked the schedule to fit the processes – Schedule performance actually improved (less rework)! – Inspections had a surprising benefit

Company S (6) • Lessons Learned: – Impossible schedule + schedule pressure = disaster – Good process: is it an end in itself, or a means to a (product) end?

• Parallel (2005): – This is typical of all too many software projects in the 21st century – “Death March”

An Overview of Failure • Research findings about failure, from “Runaway Projects – Cause and Effects,” Software World (UK, 1989 and 1995), two studies performed by KPMG

An Overview of Failure (2) • Surprising findings: – – – – – –

No industry dominant (e.g., government) Respondees optimistic Practitioners saw failures early, top-management late Risk management seldom used or acted on Consultants, packaged software no help Technology caused 7% of failures in 1989, 45% in 1995

• Predictable findings – Overly ambitious, multiple causes, management problems most frequent, schedule (89%) more common than cost (62%)

An Overview of Failure (3) • Getting up to date (Cutter Consortium, 2005) – Conclusion – spectacular failure rare – Reasons for failure • 55% cost/schedule • 25% poor quality, customer satisfaction

– Projects severely over budget • 60% of companies report <10% of projects fail severely • 25% report no such projects at all!

An Overview of Failure (4) • Contrast this with “software crisis” thinking, Standish reports (70% failure rates), … • A speaker asked his audience – How many of you believe the 70% failure rate figure? (most raised their hands) – How many of you have 70% failure rates in your own company (no one raised their hand) – Speaker concluded attendees were lying – I conclude attendees told the truth, the 70% figure represents brainwashing

Conclusion • There are spectacular failures, but they are few in number • Most projects struggle with schedule/cost issues, requirements changes (not about building software) • Biggest problems are with huge projects, hyped promises, lack of control over schedule/cost, unstable requirements • May YOUR projects never fail!