Extreme Programming

Overview on Extreme Programming Jeremy Choi [email protected] March 20th , 2002

Introduction The focus of Software Engineering is mostly on resolving technical issues on software designs. With tools such as UML deve lopers can produce detail descriptions on all parts of the end production in a technical point of view. However, little attention has been paid on formalizing the interaction between developers and end users and on assuring that design produced by developers are full understood by customers and satisfactory to the customers. Most importantly, UML is not dynamic. A change in the requirement of end-users usually results in huge effort in redesigning the structure of the end product. Along came Extreme programming (XP), a lightweight development process created by Kent Back. Unlike UML, which is used to pinpoint the details of implementation, Extreme programming is a methodology that copes with the ever-changing demands of end-users. Extreme Programming is a discipline of software development based on 4 principle values: simplicity, communication, feedback, and courage. It works by bringing the whole team together in the presence of simple practices, with enough feedback to enable the team to see where they are and to tune the practices to their unique situation. Based on these 4 principles, 12 practices are implemented: l l l l l l l l l l

Planning Game Testing Refactoring Simple Design Collective Ownership Metaphor Continuous Integration On-Site Customer Pair Programming Small Releases

l l

Coding Standards 40-hour work week

This paper attempts to first give an overview of the 4 principles and then explain the reasons behind the 12 practices

1. The 4 principles 1.1

Communication

XP encourages extreme communication. If the customer needs to see the program in action to fully formulate requirements, put the customer on the development team and release working versions every 2 to 4 weeks, developing requirements along the way. If the programmers need clarification on requirements, the customer is a member of the team -- lean back in your chair and ask the customer. Programmers work in pairs: two people to every one machine. Pair members rotate regularly. All code is owned by the team, not by individuals. This promotes communication of technical knowledge throughout the team. When technical challenges arise, the team is more able to address the problem. Pair programming also is excellent in matching up programmers of differing abilities. Less experienced members are constantly mentored, and the risk of less experienced code being added to the application is minimized. 1.2

Feedback

The faster change is identified, the faster it can be dealt with. XP strives for the fastest feedback possible in every aspect of the project. Unit Tests are written for most every piece of production code. Unit tests must run at 100% before any code is checked in. When production code needs to be changed, side-effect problems are identified. After new code is checked in, it's immediately integrated with the latest changes from the rest of the team and again, all unit tests

must be made to run at 100%. Acceptance Tests are written with the customer to verify the application is doing what the customer needs it to do. Pair Programming provides constant code reviews. No more dreary code review meetings -- put two sets of eyes on the code as it's written. Collective Ownership of the code by all members of the team helps ensure even more eyes will see the code, increasing the amount of code review performed. 1.3

Simplicity

The customer is the ultimate driving force in XP. Do what the customer needs, as simply as possible, and nothing else. Take on a mindset to reduce complexity from the beginning. All code should be refactored as often as possible. Refactoring is a process of improving a code's structure without changing its functionality. Refactoring produces highly decoupled objects which makes them easy to test, easy to use, more flexible, and therefore, more changeable. 1.4

Courage

No more fragile code. With smooth communication, quick feedback and simple code, programmers have the support they need to dive into changes when they come ... and they will come.

2. The 12 practices

2.1. Planning Game Each iteration begins with the Planning Game, an informal process that sets the agenda for the iteration. The game starts with the customer defining requirements, or 'user stories'. Technical members work with the customer to normalize these stories into manageable chunks and break them down into specific tasks, as well as introduc ing technical tasks needed to support the customer's requests (e.g. upgrading development software, automating builds, etc). Then the developers place ideal time estimates on the stories/tasks. Based on the time estimates given

combined with the team's velocity (average number of stories/tasks the team has completed in past iterations), the user stories are prioritized by the customer. Programmers then sign up for tasks and the development section of the iteration begins.

Case Study: Chrysler Gemstone Project Ken Back was invited to Chrysler Gemstone Project for performance tuning. The first step in planning, done by the business people or the end- users, was a complete low resolution pass through the system as use cases, which resulted in 135 4x6 cards of description. At the same time, developers tried to build prototypes of all the parts of the system they thought would be hard. Next step was to convert these use cases from the end-users and the experience gained from building prototypes into a schedule. First the user sorted the stories by priority, into three piles: 1. These have to be done if the system is to meet its business goals 2. These have a measurable impact on the system's business goals 3. These would make the user happy, but without explicit jus tification The priority was written in one corner of the card. Then the developers sorted them by risk into three piles: 1. We know exactly how to do this 2. We think we know how to do this 3. We have no idea what this means or how to do this The risk of each task was written in another corner of the card. These sorting activities were a key to the whole rest of development. It was in seeing all the cards laid out on a great big table that everyone understood the scope of the system. It was in discussing what the cards meant that lots of alignment began, developers to customer and developer to developer. Finally determining how many weeks of ideal developer effort each task

would take produces estimation of schedule. Without prior user-cases and prototypes, these numbers would have been hard to come by. With the exploration, estimates can be produced with high confidence. The schedule turned out to be too long, so the user fitted the tasks to the available time by setting aside low priority cards.

2.2. Testing Testing is a crucial practice on an XP project. XP succeeds by making a project resilient. Resilience means accurate and frequent feedback. Testing provides this. In XP, there are two categories of tests: unit tests and acceptance tests. A unit test is a piece of code that exercises one aspect of a piece of production code. Unit tests should be small and fast. For example, given an object that does monetary conversions, a unit test would go through the following steps: • • • •

1) Create an instance of the conversion object 2) Pass in an amount of a known currency 3) Request a conversion to another currency 4) Compare the output from the object with a hardcoded expected value

This test is now a sentry assigned to monitor this piece of functionality. At the time it's written, it may seem too small a thing to worry about. However, the amount of code in even a small project can grow very quickly. Without an automatic way to check every important detail in the code, soon the code will not be exercised regularly. Growing a collection of unit tests from day one becomes an extremely powerful tool later on in the project. All unit tests must pass 100% before programmers can check-in any new code. After check- in, integration is done on a separate machine with all of the latest changes and again all unit tests must pass. Any problems should be fixed immediately. To help ensure that a suitable suite of unit tests are built throughout development, a coding practice known as "test- first" programming is implemented. Before adding a new piece of functionality, a unit test exercising the non-existent code is written. Then, enough production code is written to allow the test to compile, but

not pass. The unit test is executed to ensure that it really does fail. If it passes at this point, there's probably a bug in the unit test. Then production code is written until the test passes. This style helps ensure a consistent test suite is developed alongside the production code. Having a consistent test suite eliminates fragile code. In the later stages of a project, programmers are usually hesitant to revisit previously written code because it's either buggy and/or messy or because it's solid -- which pretty much runs the gamut. The reason is the same regardless of the quality of the code -- fear of making a mistake and introducing new problems or breaking things that have been working. The unit tests remove this fear. Programmers can make bold changes and see the cost of the change immediately by re-running the test suite. Unit testing also promotes good design. It's easy to create highly coupled code without unit tests because for every one production object often times there's only one other production object using it, at least when the code is first written. Unit tests ensure that everything tested will have 2 clients from inception -- the production code that uses it and the unit test that tests it. Requiring code to be responsive to multiple clients forces less coupling of objects which promotes long term flexibility. This flexibility is a crucial contributor to resilience. As an added bonus, unit tests make for great usage documentation. Separate example code does not need to be written. If unit tests are written thoroughly, example code exists for free. This can greatly reduce if not eliminate the amount of internal documentation required. Acceptance tests are distinguished from unit tests in a couple of ways. First, they should test the system end-to-end. Second, the customer is involved in creating the acceptance tests. Ideally, a framework should be built that allows the customer the full ability to add new tests for the system without any technical intervention. For example, an XML file could store input/output data for each test, and a framework coded to load each case from the XML file and execute it. Unit tests ensure each technical detail is working properly. Acceptance tests ensure each customer requirement is working properly. The testing practices are a main support for eliminating up-front design. It doesn't have to be done right the first time because it's cheaper to change it on the fly, it's resilient. Refactoring also plays an important role in creating resilience.

2.3. Refactoring Refactoring is the process of improving the design of code without changing the functionality. The code should be clean and readable. Any duplication should be consolidated. Refactorings should be done on an ongoing basis throughout development of the code. As soon as structure improvements make themselves known, they should be done. In many environments this would be risky, as the tendency for developers to introduce side-effect bugs while making changes is high. But the unit tests provide the quick feedback required to steer a programmer back on track during a refactoring. Refactorings also tend to get postponed because no new functionality is being created. However, allowing poorly structured code to exist in a project is a risk that accumulates over weeks of development. Refactoring is best practiced on a regular basis.

2.4. Simple Design To help ensure frequent feedback, it's important that the application's design be kept simple and kept to providing business value. While there will always be tasks that are primarily technical and necessary to support providing business value, it's important these tasks be kept as simple as possible. The main reason for this is to eliminate unnecessary development time. It's easy for the technical members of a team to overdo aspects of the project because we might need it. Here we come to a couple of XP's acronyms: YAGNI and DTSTTCPW. YAGNI stands for "You Aren't Going To Need It." Too often, the team attempts to build in functionality they might need in the future. This is a reaction from the anticipation style of doing things. If future changes are expensive, make sure you do it right the first time and think of everything the first time. YAGNI takes advantage of the resilience created by testing and refactoring. Since the provided resilience will allow for inexpensive future changes, we only need to build for today.

DTSTTCPW stands for "Do The Simplest Thing That Could Possibly Work." One note here, Simple should not imply Poor Quality or even the Fastest, though speed is the main issue with simple design. Simple should include as much elegance as possible without getting burdensome. But the tendency is still towards simplicity over elegance. Here's why. Thorough refactoring practices will cover any initial problems. The fact is, it's hard to get it right the first time. Simple design says, don't try to get it right the first time. Write the unit tests and get those passing in a simple way. If the code just written turns out not to be too important to the overall project, then it's okay if it's less than ideal, as long as it works (passes testing). If the code is crucial, it's guaranteed to be revisited many more times. The process of refactoring allows the less than ideal but simple design to have its kinks worked out, and no one wasted a lot of time trying to get it designed right the first time.

2.5. Collective Ownership Collective ownership refers to the code. Many development efforts assign specific portions of the application to individuals. Sometimes programmers and managers like this because it can be easier to measure individual contributions to the project. However, the potential for bottlenecks with individual ownership is higher than with collective ownership. When only specific individuals are familiar with certain portions of the codebase, others on the team can be held up waiting for changes to be made. In addition, the amount of work needed for different sections of a project can change during the course of development. If individual ownership of each section is promoted too much, work loads can become lopsided. Collective ownership allows anyone on the team at any time to work with any piece of code. If a pair working with object A needs object B to change, that pair can go immediately make the change in object B to accommodate the needs of object A. The practice of quickly dipping into a related piece of code to make a quick change can be dangerous, because quick changes can often times create side-effect bugs. In an XP project, however, every piece of code is developed

test- first, ensuring each piece of functionality is unit tested. If the quick edit fixed one thing but broke several others, the test suite will be run shortly after the edit was made and report the errors immediately to the programmer who just made the edit. If a bug slips through at this level, acceptance testing provides a second net for catching the problem. Pairing will be more difficult if the code any pair is working on is 'owned' by only one member. The non-owner of the pair may not feel comfortable suggesting or making changes to code that they don't have rights to. Pair programming and collective ownership does not snuff out individual contributions (good or bad). Bonus plans and other types of performance rewards may have to be changed. XP aims to avoid hang- ups caused by individual ownership by focusing ownership of the team on the whole. Take pride in the whole, not in the individual contributions to the whole. The project itself must succeed as a whole to be effective, and that should be the goal of any development effort. One concern about collective ownership is ensuring each programmer is a positive contributor to the team and not riding on others' efforts. An XP team is very people centric, relying on a lot of interaction (especially with pairing) and verbal communication. It would be difficult for a no n-contributing programmer to hide out for long in this environment. In fact, it would be easier to hide out in an environment where individuals hole up in their cubes for weeks at a time while they code their own thing. Another concern with collective ownership is building expertise with the application. There are limits to the familiarity an individual can have with the codebase. Individual ownership helps solve this problem by limiting the amount of code a programmer creates to promote expertise in certain areas. Documentation of the code helps to cover the gaps as well. However, documentation must be maintained and while individual ownership can help with code expertise, the fact is it's still easy to out-code the limits of a programmer's memory. Collective ownership doesn't help in this regard because it's concerned about overcoming individual ownership bottlenecks by spreading knowledge of the

whole codebase to every member. XP answers this problem of code familiarity by moving the responsibility from the developers to the testing structure (unit tests and acceptance tests). Instead of relying on the programmers or documentation to remember every aspect of the code, XP makes the code responsible for maintaining its own accuracy with unit tests.

2.6. Metaphor The project metaphor is, more or less, an informal architecture of the system. The metaphor describes the system in simple concepts. The concepts can be literal or figurative, depending on the clarity of the actual system. Perhaps the system resembles a post office, an assembly line, or an anthill. The point is to pick something common enough that each member of the team can understand. Since an XP project has little to no formal architecture documentation, the metaphor can be a useful tool to aid in communication among team members, especially between programmers and customer. A good metaphor can sometimes inspire improvements to the application itself. But it's important to remember the metaphor is simply a tool of communication and should be changed as the needs of the project change.

2.7. Continuous Integration Mixing the latest code from each programmer together can be a difficult process, especially if this task is not done often. To stay resilient, after writing new code that passes all tests locally, programmers must then integrate their changes with the latest code base and ensure all the tests still pass. If not, fixes must be made right away until all tests again pass. It's recommended this task be done many times a day, usually on a dedicated integration machine.

2.8. On-Site Customer Every XP project has one or more individuals to fulfill the customer role on the team. The customer's job is to write and prioritize stories (tasks from a user's perspective that the software must perform), assist with acceptance testing and be on hand to answer questions from the development team as they arise. It's typical for the customer to continue on with regular duties of their job, but it's important they physically office with the development team. The project starts and ends with the customer. The customer determines what must go into the product and declares the work successful through approved acceptance tests which were authored by the customer. Having the customer be an active member of the team provides for frequent and cheap communication. There's a much smaller need for formal documentation with direct communication. If the customer is physically separate from the team, intermediate forms of communication (conference calls, email, documents) become more valuable because it's harder for the developers to get the information they need from the customer directly. Maintenance of this information can become a project in and of itself and distract the team from working on the actual product. An involved customer can see the product as it evolves and can redirect development efforts as requirements and the fulfillment of those requirements come into focus. Changes are identified quickly and in smaller, more manageable portions with fewer side-effects.

2.9. Pair Programming All programming on an XP team is done in pairs, two people at one machine. Each task from the Planning Game is owned by an individual. When the day starts, pairs form up, each person either pairing to help someone else, or requesting help on their own tasks. Pairs stay together until a logical break comes up. One takes a turn 'driving' while the other actively participates verbally. As ideas flow between the two, the keyboard can be swapped off as often as necessary to get the best code on the screen. Pair assignments are fluid and change throughout the course of a day.

The benefits of pairing are numerous. One thing pairing provides is constant code review -- no line of code is written without two sets of eyes. This can reduce or eliminate the need for code review meetings which can often be boring and wasteful. Pairing helps distribute knowledge of the code more evenly throughout the team. This can eliminate personnel bottlenecks due to illness, vacations or team changes. If only one team member understands how a crucial layer of the system functions, progress can be severely derailed if that individual leaves for another job. Pairing provides balancing of talent within the team. A project that assigns portions to specific individuals can suffer from varying quality as the functions written by the senior members work well, but are held back by the bugs and poor design of the functions written by junior members. With the pairing process, no junior level member has to write code alone. They can be constantly mentored through the pairing process which will educate them on the job and build up the quality of the code throughout. Individual programming moves a project towards the average abilities of the team. Pair programming moves a project towards the maximum abilities of the team. Pairing can also be faster than individual programming. When writing code, one rarely works in one mental level. Switching levels takes a bit of time. Having two brains working on a problem allows thought to occur at more than one level simultaneously. Two people can leapfrog each other. While one writes code, the other can think at a higher level, organization of code, where to go next, tests that still need to be written. A pair of programmers also has reduced distractions. There's built- in accountability to ensure the mind and mouse don't stray too far from the task at hand. Writer's block (as well as Debugger's block) is less apt to steal time away from two people. Pairing is probably one of the most controversia l XP practices because of a number of challenges facing it. Management may be a hard sell because they see pairing as a waste of resources. Many of the benefits of pairing are long term and hard to see up front. Other benefits, like increased development speed, may have

to be experienced to be believed because it's easy to assume development is a simple labor task when in fact there's a high degree of creativity and thought that must go into it. The math that says, "Two can work twice as fast as one" simply doesn't hold up for many development tasks. While management may struggle with pairing, many programmers take issue with the practice as well. Successful programmers depend on good tools and sound structure. Introducing pairing, which requires sharing of tools and practices, can create friction as differences rub up against one another. Everything from tool sets to work environment can be up for debate. In addition, many programmers believe that they do their best work alone, or don't enjoy the mentoring aspect of pairing with a less experienced person.

2.10. Small Releases Small releases are a key part of generating feedback and making a project resilient. An XP project is a series of iterations, each lasting 2 to 4 weeks. Each iteration starts with the Planning Game, an activity that determines the tasks for the current iteration, and ends with a 'finished' product: all tests pass and the product is as functional as possible. Each release may or may not be a production release, especially in the early days of a project. Most projects simply aren't useful until a minimum feature set is complete. But it's important to approach each release aiming to give the customer the most business value possible. When technical items are carried over from iteration to iteration, they can accumulate additional functionality that may not be needed.

One project I worked on spent a good bit of time constructing 26 elaborate reports for the system. Once the application got to a beta stage, it was put into a production environment in 3 locations, and all 3 quickly made good use of the product. 4 months into the beta, a problem with the reporting module was detected during development. It was a major bug -- none of the reports could be opened. It wasn't long before the development team wondered why none of the 3

beta sites had come across this problem. It turned out that none of the beta sites had even tried to use the reports. If they didn't need the reports, was it really necessary to build them? In another situation, a new data entry form was being added to an existing application. The GUI for the form had been designed to be as easy to use as possible. Implementing the design required use of some third party GUI tool that was complex and time consuming to work with. Many additional weeks were added to the timeline tweaking out the GUI. After it was complete, the new form was still useless, however, because the data entered into the system through this form had not been included into other processes required in the application. Following the practice of small releases that are as close to functional as possible would have delayed the more elaborate GUI features to focus on including the new data elsewhere in the app. If an early iteration worked well enough for the customer, the more elaborate GUI might have been easily scaled down to a less efficient, but still quite usable form, allowing the customer to move on to more important items. As noted earlier, many times customers need to see a working product to help crystalize requirements. Sometimes verbal agreements on design decisions don't stand up in the face of actual implementation. It's important to keep a working product in front of the team to generate precise feedback. Small releases also help the technical members of the team give accurate estimates. Any time estimate beyond 2 to 4 weeks tend to be very imprecise.

2.11. Coding Standards Having a coding standard for a project is a commonly accepted practice in most projects regardless of methodology. This practice is equally important within an XP team, especially in light of Collective Ownership and Refactoring practices. 2.12. 40-Hour week XP promotes a well rested team. Its founders do not believe in the extended overtimes, which creates fatique and defeats concentration. Tired workers make mistakes and their desire to work decreases.

3. Bibliography •

Beck, Kent. Extreme Programming Explained: Embrace Change, Addison-Wesley

•

Fowler, Martin. “The New Methodology”. 12 Sept 2000 http://www.martinfowler.com/articles/newMethodology.html

•

Beck, Kent; Fowler, Martin. Planning extreme programming, Addison- Wesley

•

Extreme programming: a Gentle Introduction http://www.extremeprogramming.org

•

Xprogramming.com : http://www.xprogramming.com/