Agile Sw Development

AGILE SOFTWARE DEVELOPMENT: INTRODUCTION, CURRENT STATUS & FUTURE Pekka Abrahamsson 23.11.2005 Jyväskylä

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

EXAMPLE OF AN INNOVATIVE LEAP - 1 •Due to the adoption of an agile production frame, F-Secure's mobile division in a new product development achieved § 3x reduction in lead-time, § 50x better quality, and § 5x cheaper! •Innovative leap due to the acceptance of radical variation in product development & management processes "From the management point of view, the

Fact corner: results were amazing." - SME of 250 developers Jari Still, Director, F-Secure, Finland - Mobile & desktop sw 1.12.2005 (C) VTT Electronics, Pekka AbrahamssonAGILE-ITEA Newsletter #2,2 2005 - Products sold globally

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

EXAMPLE OF AN INNOVATIVE LEAP - 2 •The use of diverse agile solutions lead Philips to: •Productivity: 8x Faster than industry average •Quality: 3.5x Better than industry average •Customer satisfaction: 4.9 in 5 point scale •Softfab, Rapid7, reflection, sprints, … from the agile toolbox Fact corner: - 400+ Kloc - 17.5 person years/1year 1.12.2005 (C) VTT Electronics, Pekka Abrahamsson

3

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

INNOVATIVE LEAP EXPLAINED •"The actual processes adopted were neither novel nor particularly inventive. Rather they had the virtues of being easy to explain and relatively easy to comply with, with goals easily describable as having been met or not." Ward et al. (2001)

1.12.2005 (C) VTT Electronics, Pekka Abrahamsson

4

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

Part I: Agile philosophy & rationale

1.12.2005 (C) VTT Electronics, Pekka Abrahamsson

5

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

HISTORICAL PERSPECTIVE •Software Crisis (1960’ s) - Software intensive systems delivered late, over budget and do not meet the quality requirements •Solution attempt #1: Structured Methods (in 1980’ s) •Solution attempt #2: Object-oriented methodologies •Chronic Software Crisis (1990’ s) - Software intensive systems still delivered late, over budget and do not meet the quality requirements •Solution attempt #3: Software process improvement •Solution attempt #4: Agile development methodologies

1.12.2005 (C) VTT Electronics, Pekka Abrahamsson

6

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

FEATURE USAGE RATE Always 7% Often 13 % Never 45 %

Sometimes 16 %

Rarely 19 %

Fact corner: > 60% features never or rarely used!

Source: Jim Johnson of the Standish Group, Keynote Speech XP 2002

1.12.2005 (C) VTT Electronics, Pekka Abrahamsson

7

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

HOW AGILE CAN YOU BE?

Size (# of personnel)

Fact corner: Agile is not an absolute 1.12.2005 (C) VTT Electronics, Pekka Abrahamsson Concept.

Source: Boehm & Turner (2003) 8

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

AGILE THINKING EXPLAINED

Need to respond to constant changes Values Principles Practices

Fact corner: There is no definite set of agile practices.1.12.2005 (C) VTT Electronics, Pekka Abrahamsson

The fundamental reason for a “new” paradigm Defines the set of most important beliefs of what is truly important Defines a set ways to meet the values Defines in detail how this is implemented in practice 9

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

AGILE VALUES…

Fact corner: Agile manifesto is first of its kind in software 1.12.2005 (C) VTT Electronics, Pekka Abrahamsson Source: www.agilemanifesto.org 10 engineering field

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

THE 12 AGILE PRINCIPLES (1/3) DESCRIPTION 1. Our highest priority is to satisfy the customer through early and continuous delivery of valuable software 2. Welcome changing requirements, even late in development. Agile processes harness change for the customer’ s competitive advantage. 3. Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to a shorter timescale. 4. Business people and developers must work together daily throughout the project Fact corner: See principals as “thinking-tools” 1.12.2005 (C) VTT Electronics, Pekka Abrahamsson

SUMMARY 1. Satisfy customer through early and frequent delivery. 2. Welcome changing requirements even late in the project. 3. Keep delivery cycles short (e.g., every couple of weeks). 4. Business people and developers work together daily throughout the project.

11

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

THE 12 AGILE PRINCIPLES (2/3) 5. Build project around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

5. Build projects around motivated individuals.

6. The most efficient and effective method of conveying information to and within development team is face-to-face conversation.

6. Place emphasis on face-to-face Communication.

7. Working software is the primary measure for progress. 8. Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.

1.12.2005 (C) VTT Electronics, Pekka Abrahamsson

7. Working software is the primary measure of progress.

8. Promote sustainable development pace.

12

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

THE 12 AGILE PRINCIPLES (3/3)

9. Continuous attention to technical excellence and good design enhances agility.

9. Continuous attention to technical excellence and good design.

10. Simplicity –the art of maximizing the amount of work not done –is essential.

10. Simplicity is Essential.

11. The best architectures, requirements, and designs emerge from self-organizing teams. 12. At regular intervals, the team reflect on how to become more effective, then tunes and adjusts its behavior accordingly. 1.12.2005 (C) VTT Electronics, Pekka Abrahamsson

11. The best results emerge from self-organizing teams.

12. Team reflects regularly where and how to improve.

13

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

What has improved? (Asked from the industrial developers)

Iteration planning Estimation skills Communication

Responsibility

Fact corner: Agile methods improve communication! 1.12.2005 (C) VTT Electronics, Pekka Abrahamsson

14

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

Part II: Existing agile software development approaches

1.12.2005 (C) VTT Electronics, Pekka Abrahamsson

15

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

EXISTING AGILE METHODS •Methods for agile software development: - Agile software process model [Ayoama, 1998] - Adaptive Software Development [Highsmith, 2000] - Crystal Family of Methodologies [Cockburn, 2000] - Dynamic Systems Development Method [Stapleton, 1997] - Extreme Programming [Beck, 1999] - Feature-Driven Development [Palmer & Felsing, 2002] - Lean software development [Poppendieck x 2, 2003] - Scrum [Schwaber, 1995; 2002] - … the list is growing every year… •Combination of approaches: - Agile Modeling [Ambler, 2002] Fact corner: - Internet-Speed Development [Cusumano & Yoffie, 1999; Only 3 out of 9 Baskerville et al., 2001; Truex et al., 1999] Methods has - Pragmatic Programming [Hunt & Thomas, 2000] Empirical evidence

1.12.2005 (C) VTT Electronics, Pekka Abrahamsson

16

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

AGILE PRACTICES: THE TOOLBOX XP

Scrum

Crystal

Product backlog

2

Planning game Small/short releases

Effort estimation

Staging Revision and review

3

Metaphor

Sprint

Monitoring

4

Simple design

Sprint planning meeting

Holistic diversity

5

Testing

6

Refactoring

7

Pair programming

8

Collective ownership

1

Sprint backlog Daily scrum meeting Sprint review meeting

Methodologytuning technique User viewings Reflection workshops

Agile practices FDD DSDM ASD Domain object Active user Iterative modelling involvement development Developing by Teams must be Feature-based feature empowered.. planning Individual class … frequent Customer focus(code) ownership delivery group reviews Fitness for business purpose… Feature teams Iterative and incremental development Inspection All changes are Regular builds deversible Configuration Requirements are management baselined… Progress reporting

Continuous 9 integration 10 40-hour week 11 On-site customer 12 Coding standards 13 Open workspace 14 Just rules 15 16

Testing is integrated … A collaborative and cooperative approach shared by all stakeholders

Lean Seeing waste Value stream mapping

Perceiced integrity Conceptual integrity

Feedback

Refactoring

Iterations

Testing

Syncronization Set-Based Development

Measurements

Options thinking The last responsible moment

Making decisions Pull systems Queuing theory Cost of delay Self determination Motivation Leadership Expertise

1.12.2005 (C) VTT Electronics, Pekka Abrahamsson

17

Contracts

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

SOURCE FOR AN OVERVIEW OF AGILE METHODS •Titled “Agile software development methods: Review and analysis”, 2002 •A VTT series publication freely available from http://agile.vtt.fi •For each method the following aspects are described: - Purpose & motivation - Roles & responsibilities - Development process - Development practices - Current status & known limitations 1.12.2005 (C) VTT Electronics, Pekka Abrahamsson

18

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

Part III: The Mobile-D™ for mobile software

1.12.2005 (C) VTT Electronics, Pekka Abrahamsson

19

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

FORMATION OF A BASELINE: REQUIREMENTS FOR A GOOD SOFTWARE DEVELOPMENT PROCESS •The development process should •provide systematic support for high quality software development (Production frame) •be designed for small (and medium) sized development teams (Most teams are small) •produce visible results early (Early validation) •be easy to learn, transparent, straightforward and adjustable (= makes sense) •provide a fit to company’ s strategic planning (= aligned with the business) •meet the needs of standard quality requirements (= be convincing to us & customers) 1.12.2005 (C) VTT Electronics, Pekka Abrahamsson

20

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

NEW PROGRAMMING ENVIRONMENT End-users

Provides value

Customer Learns needs

Benefits Shares end-user needs & business requirements

Coach / Mentor

Software development activities, tools, people

delivers

Results: Working software, rapid delivery, high business value

supports

Removes obstacles

Management

Learning

State-of-the art methods Process definitions

Training

1.12.2005 (C) VTT Electronics, Pekka Abrahamsson

- Daily feedback from the developers - Communication - Collaboration 21

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

MOBILE-D™ FOR MOBILE SOFTWARE •Concept: An 8-week agile development rhythm •Mobile-D™ is based on Extreme Programming (practices), Crystal methodologies (scalability) and Rational Unified Process (coverage) •Designed to meet the specific characteristics of mobile software development & industry quality standards •Designed for < 10 developers working in (close to) co-located office space •Pattern-based version: http://agile.vtt.fi/mobiled/mobiled.htm 1 WEEK

EXPLORE SET-UP

2 WEEKS

CORE

1.12.2005 (C) VTT Electronics, Pekka Abrahamsson

2 WEEKS

CORE-2

2 WEEKS

1 WEEK

STABILIZE WRAP-UP 22

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

AGILE PROGRAMMING ENVIRONMENT: THE WARROOM APPROACH

Fact corner: Software development 1.12.2005 (C) VTT Electronics, Pekka Abrahamsson Is not solo business

23

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

THE PRINCIPAL ELEMENTS OF MOBILE-D •Requirements: Off-Site Customer •Planning: Phasing and pacing in Planning Day •Modeling: Agile modeling •Architecture: Architecture Line •Metrics: Time, size and defect •Documentation: RaPiD7-method •Improvement: Agile Software Process Improvement •End-users: User-Centred Focus •Testing: Mobile Test-First development 1 WEEK

SET-UP

2 WEEKS

CORE

2 WEEKS

2 WEEKS

1 WEEK

CORE-2

STABILIZE

WRAP-UP

1.12.2005 (C) VTT Electronics, Pekka Abrahamsson

24

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

MobileD: FIT TO STRATEGIC PLANNING STRATEGIC RELEASE PLANNING RELEASE PROJECT

INCREMENT HEARTBEAT

4 CYCLES OF CONTROL Rautiainen & Vähäniitty (2004)

1 WEEK

MobileD

SET-UP

2 WEEKS

CORE

1.12.2005 (C) VTT Electronics, Pekka Abrahamsson

2 WEEKS

2 WEEKS

1 WEEK

CORE-2

STABILIZE

WRAP-UP

25

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

MobileD: FIT TO STRATEGIC PLANNING STRATEGIC RELEASE PLANNING RELEASE PROJECT

INCREMENT HEARTBEAT

4 CYCLES OF CONTROL Rautiainen & Vähäniitty (2004)

1 WEEK

MobileD

SET-UP

2 WEEKS

CORE

1.12.2005 (C) VTT Electronics, Pekka Abrahamsson

2 WEEKS

2 WEEKS

1 WEEK

CORE-2

STABILIZE

WRAP-UP

26

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

MobileD: FIT TO STRATEGIC PLANNING STRATEGIC RELEASE PLANNING RELEASE PROJECT

INCREMENT HEARTBEAT

4 CYCLES OF CONTROL Rautiainen & Vähäniitty (2004)

1 WEEK

MobileD

SET-UP

2 WEEKS

CORE

1.12.2005 (C) VTT Electronics, Pekka Abrahamsson

2 WEEKS

2 WEEKS

1 WEEK

CORE-2

STABILIZE

WRAP-UP

27

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

MobileD: FIT TO STRATEGIC PLANNING STRATEGIC RELEASE PLANNING RELEASE PROJECT

INCREMENT HEARTBEAT

4 CYCLES OF CONTROL

RELEASE PLANNING DAY DAY WORKING DAY MobileD

SET-UP

CORE

1.12.2005 (C) VTT Electronics, Pekka Abrahamsson

CORE-2

WRAP-UP 28

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

Mobile-D: The daily heartbeat PLANNING DAY

1 WEEK

SET-UP

2 WEEKS

CORE

WORKING DAY

RELEASE DAY

2 WEEKS

2 WEEKS

1 WEEK

CORE-2

STABILIZE

WRAP-UP

1.12.2005 (C) VTT Electronics, Pekka Abrahamsson

29

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

ITERATION BACKLOG PRODUCT BACKLOG

INFORMATION RADIATOR IN THEORY DEFINED

STARTED

DONE

ITERATION STATUS

VERIFIED

PRODUCT STATUS

FINISHED PRODUCT

FEATURE1 FEATURE3

TASK TASK3.N TASK

TASK3.N TASK

TASK 3.N

TASK 3.N FEATURE2

FEATURE4

TASK TASK TASK4.N TASK

FEATURE FEATURE FEATURE FEATURE FEATURE FEATURE

FUTURE BACKLOG

1.12.2005 (C) VTT Electronics, Pekka Abrahamsson

30

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

INFORMATION RADIATOR IN PRACTICE

Project team in a daily stand-up meeting

Customers and project managers working together

Fact corner: Information Radiator Is one of the most spread “Big boss”at the airport agile practices 1.12.2005 (C) VTT Electronics, Pekka Abrahamsson

31

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

Part IV: Empirical cases

1.12.2005 (C) VTT Electronics, Pekka Abrahamsson

32

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

Some empirical evidence Mobile service: Active investor

Mobile added value: Stockbroker

Mobile added value: Sales person

Fact corner: 9 projects completed 1.12.2005 (C) VTT Electronics, Pekka Abrahamsson 1 projects underway

33

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

3 CASE PROJECTS Velocity in terms of implementation (design+code+test) Task effort allocation (Case average)

Percemtage of total effort

60 % 50 %

2,0 %

40 % 48,4 %

44,4 %

30 %

3,0 % 8,1 % 5,5 %

10,7 %

Defect fix 35,3 %

9,8 % 25,1 %

20 %

Enhancement

8,6 %

New feature 21,1 %

10 % 0% 1

2

Fact corner: Explicit quantified process control, i.e. 1.12.2005 (C) VTT Electronics, Pekka Abrahamsson CMMI 5 level issues

3

4

5

34

RELEASE

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

What is not working in the process? 282 problems in the development process… 45 40 35 30

# of negative issues

Project 1

25

Project 2 Project 3 Project 4

20

Project 5

15 10 5 0

Fact corner: 1 2 5 projects, 50 person months 1.12.2005 (C) VTT Electronics, Pekka Abrahamsson

3

4

# two-week work cycles 35

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

What is working in the process? 342 positive remarks in the process.. 45 40 35

# of positive issues

30 Project 1

25

Project 2 Project 3 Project 4

20

Project 5

15 10 5 0

Fact corner: 1 2 People seldomly share 1.12.2005 (C) VTT Electronics, Pekka Abrahamsson what is going well!

3

4

# two-week work cycles 36

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

BUT, BE AWARE OF THE NEW DEVELOPMENT TECHNOLOGIES 20,0

TRADITIONAL UI a)

Lost hours

n=97 10,0

0,0

R1

-10,0

Lost hours

20,0

b)

- The same "rethought" process used - With theR2new technology: R3 R4 R5 65% savings in effort 50% reduction in the lead-time Traditional mobile platform NAKED OBJECTS 79% reduction lines-of-code = effort in 1070 hours, 8 weeks n=35

10,0

0,0

R1 -10,0

R2

Naked Objects framework = effort 380 hours, 4 weeks, 79% less app. code

Source: Keränen & Abrahamsson, Euromicro 2005 1.12.2005 (C) VTT Electronics, Pekka Abrahamsson

37

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

Part IV: Future & conclusions

1.12.2005 (C) VTT Electronics, Pekka Abrahamsson

38

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

FUTURE

•Agile hype is getting to be over (good news) •XBreed, Freedom and other interesting agile methods will still keep emerging •Yet, how to become more agile is the question now •Before it was more concentrated on extreme programming •Empirical evidence is quickly building up •Agile becomes part of standardization work as well: Working group for IEEE 1648 (recommended practices) 1.12.2005 (C) VTT Electronics, Pekka Abrahamsson

39

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

THANK YOU! Questions and comments? Contact me at: [email protected] http://agile.vtt.fi XP2006 will be organized in Oulu, Finland 17.6.-22.6.2006 featuring Kent Beck, Barry Boehm and others. Go check out www.xp2006.org 1.12.2005 (C) VTT Electronics, Pekka Abrahamsson

40