Software Engineering Process Using Agile

Software Engineering Process using Agile Methodology

Software Engineering Process using Agile Methodology Author: Rakesh Wagh

Keywords "Agile software development", "Continuous Integration", "Test driven development"

Abstract Agile software development practices offer compelling advantages like streamlined processes, less “paper work”, high quality, creativity in work, shorter delivery cycles, and above all customer satisfaction. Proper adoption and implementation of agile processes is a challenge in itself. Automated tools and practices like Continuous Integration, Test Driven Development, etc can largely simplify agile adoption. http://itnow.blogspot.com

1

Software Engineering Process using Agile Methodology

Agile software development................................................................................................3 Advantages of Agile .......................................................................................................3 Agile in a nutshell ...........................................................................................................3 Ever changing scope ...................................................................................................4 Quality and Testing .....................................................................................................5 Deliverables.................................................................................................................7 Documentation ............................................................................................................7 Communication............................................................................................................8 Challenges with Agile methodology................................................................................9 Project Management ...................................................................................................9 People ..........................................................................................................................9 Other Challenges..............................................................................................................9 Agile Methods ...............................................................................................................10 Extreme Programming (XP)......................................................................................10 Feature Driven Development (FDD).........................................................................11 Dynamic Systems Development Method (DSDM)...................................................12 Rational Unified Process or Agile Unified process(RUP or AUP)...........................13 SCRUM......................................................................................................................14 Tools for Agile process......................................................................................................14 Continuous Integration (CI) Tools ................................................................................14 Commercial CI servers .............................................................................................15 Open Source or Free CI Servers ...............................................................................15 Test Driven Development Tools (TDD) .......................................................................15 Issue/Bug/Defect Tracker .............................................................................................16 Team Collaboration and Wiki Tools .............................................................................16 Java based IDE ..............................................................................................................16 Build system ..................................................................................................................16 Source Code Management (SCM) ................................................................................16 Code quality and optimization ......................................................................................17 Final words: ......................................................................................................................17 External References...........................................................................................................17

http://itnow.blogspot.com

2

Software Engineering Process using Agile Methodology

Agile software development Agile is an evolutionary software engineering methodology that addresses the drawbacks found in traditional software development practices. Agile methodology emphasizes on: individuals and interactions over processes and tools, working software over comprehensive documentation, customer collaboration over contract negotiation, responding to change over following a plan. Below are some notable quotes about Agile. "Agile software development is a conceptual framework for undertaking software engineering projects that embraces and promotes evolutionary change throughout the entire life-cycle of the project." - Wikipedia "Characterized by quickness, lightness, and ease of movement" - Dictionary Definition of the word ‘Agile’ "Agile is an iterative and incremental (evolutionary) approach to software development which is performed in a highly collaborative manner with ‘just enough’ ceremony that produces high quality software in a cost effective and timely manner which meets the changing needs of its stakeholders.” - AgileModeling.com

Advantages of Agile -

Shortened development cycle Higher stability of work-loads Higher utilization of work-load, that is, developing large-scale, software systems with a fixed number of developers, Higher flexibility to change of Management & development plans Higher quality by earlier feedback from the customers. Customer satisfaction by rapid, continuous delivery of useful software Working software is delivered frequently (weeks rather than months) Even late changes in requirements are welcomed Simplicity

Agile in a nutshell -

Develop project in small iterations. Iteration length should be fixed early on (based on the practice, example XP, RUP, etc). Each iteration (often called as time-box) is a mini project in itself. Every iteration will have its own planning, requirements analysis, design, coding, testing, and documentation phase Do not stick to a pre defined process. Process should be tailored based on project need and various other parameters like people skills, requirements, etc.

http://itnow.blogspot.com

3

Software Engineering Process using Agile Methodology

-

-

Aim for fluid long term plans and concrete short term plan. Reevaluate project priorities and process at the end of each iteration. A usable though minimal system can go in production at the end of every iteration. Questions to ask in the meeting at the end of every iteration o What did we do well? o What have we learned? o What can we do better? o What puzzles us? Good team communication and collaboration is very important. Prefer face to face, oral communication over written documentation. Use collaboration tool to share ideas and easily disseminate information. If possible, physically locate the team together. Modern communication methods like frequent telephone/video conferencing should be used if the project team is geographically distributed (e.g. onsite-offshore model). Prefer developed software (or the features completed) over documents or task list as a primary measure to track progress. Developers should have access to good quality business expertise. Welcome changes; be adaptive rather then predictive; people oriented, rather than process oriented. Control an unpredictable process through iterations. Define a good feedback system that will tell us where we are at the end of every iteration. o Example: Customer himself can provide excellent feedback. Primary concern should be "business value", did the customer get what he asked for?

Ever changing scope -

-

-

-

It is important to understand that business user (customer) does not always have a very good picture of what the production software should actually look like. Add it with the complexity of communicating these requirements back to the development team and finally blending it into a finished product. “Customer requirements” does not necessarily always translate to a “business requirement”. o Example: Customer might require a “link” on the form page. However it is important to know the business case behind having that link and providing a solution to address the business case and not the link! Scope change is the reality of software development. As the system is developed, both business and development learn new things about the problem they are trying to solve. It is not uncommon to have a huge delta between the understanding of requirements at the beginning and at the end of a project. With this delta not delivered in the software, we often end up with unhappy customer. Most valuable features are not obvious until customer have had a chance to play with the software.

http://itnow.blogspot.com

4

Software Engineering Process using Agile Methodology

-

It is not uncommon to see an unhappy customer even when the deliverables strictly adhere to the original project charter.

-

Scope changes in traditional software development o Processes are based on scenarios that are envisioned in the planning phase. New scenarios, or changes to existing ones are often considered "bad" for the project plan. o Change in scope lead to slippage of schedule, increase in cost and lot of frequent changes to the project plan baseline. o This happens because scope, cost and schedule baselines are predicted early on in the planning phase. Project is “expected” to stick on this baseline there after. Changing the baseline plan is an option but it is not very desirable neither is it always possible (especially late in the project). o Over all, high resistance towards change in scope.

-

Scope changes in Agile approach o Agile model welcome change. o Change is looked at as the driving force towards delivering a better product/service. o Change and new requirement help to define the deliverable for the next iteration. o Changes fuel incremental growth of the application in a direction desired by the customer. o Changes drive the design and development. o Customer is always happy with the end product because he has had active participation in the development process. o Cost and time baselines are fixed in the beginning of the project. Scope is never base lined and is added incrementally. o Agile process encourages the business to learn about their need as the software is build.

Quality and Testing -

-

Software quality is not only about delivering bug free code, but delivering a system that is adaptive to future changes. Quality is often considered as an integral part of scope. A good quality code (and design) will be much more adaptive to changes than its reverse. The vice versa is also true and the driving force for agile the development practice: “A piece of software that can easily adapt to new changes is inherently superior in quality”. Quality requirement for IT projects are difficult to baseline and quantify. Quality is provided by performing rigorous testing. o Testing is broadly divided into "functional testing" and "unit testing".

http://itnow.blogspot.com

5

Software Engineering Process using Agile Methodology

-

-

o Functional testing look at the bigger picture and concentrates more on functionality then on programming and design errors. Functional testing is performed by team of testers who have nothing to do with real code. o Unit testing looks into the actual code and attempt to capture bugs induced during construction and design. Development team is responsible for unit testing the application. A rigorous unit and functional testing is key to delivering a good quality software. Traditional approach towards Testing/QA o Construction and Testing (QA) are two distinct (often nonoverlapping) phases performed by separate teams. o Unit testing is often ignored or not planned properly because of this phase separation. o Creating "unit test case" is not the primary objective of construction phase; reason why unit testing is largely ignored in projects. o While the system is in QA, development resources are expected to fix the bugs found by quality team. There is no real time allotted for unit testing. o Pushing QA(testing) towards the end of project can lead to a "fat" bug fixing cycle which often result in code patches that might inject further "non unit tested" errors. o If design bugs are discovered when in QA, there is no choice but to hide them. o "Work around"(which should be avoided) is the quickest and often most adopted way of fixing bugs in this stage. o Software created by such practice can be "bug free" but "low in quality". Low quality of software is typically discovered when the system is in production. Agile approach towards Testing/QA o Testing is integral part of construction. o Functional team will do the functional testing and development team will do unit testing. This is done for every iteration. o Incremental testing and bug fixing will avoid a major bug to be discovered late in the project. o Developers get the chance to provide the best solution instead of a quick fix or work around. o No notion of final testing and bug fixing. o Quality of the final deliverable is greater or equal to the quality of deliverables at the end of each iteration. o Testing time is accounted in the iteration plan. o Code development will go hand in hand with test development. o As unwanted features are eliminated in each iteration, major testing effort is concentrated only on the real system.

http://itnow.blogspot.com

6

Software Engineering Process using Agile Methodology

Deliverables -

-

Traditional approach for project deliverables o Requirement collection is often the first (and only) phase in project life cycle. Requirements provided by the user often form the scope baseline which is verified against the final deliverable. o Lot of customer interaction is done early on to properly "capture the requirement" and to deliver the same. o This approach attempts to deliver the entire software at once, based on the requirements captured in early stage of the project. o Projects often have milestone or end of phase deliverables. But these deliverables are in form of documents. Documents can differ from the actual development and fail to give a clear picture of the developed software. Agile approach for project deliverables o Final deliverables are not "completely" planned early on. o Deliverables are planned only for the next iteration. o Entire application is delivered in "production ready" quality at the end of each iteration (with more features in each delivery). o Customer can get early access to a striped down version of the software. o Customer expectations are satisfied and re evaluated at the completion of every iteration. o Difference in understanding between the customer and development team reduces with every iteration. o No surprises in the final delivery! o Delivery in form of working system instead of documents.

Documentation -

Documentation is a form of communication. Documentation can also be used for information archival or historic purposes. Traditional approach o Traditional process will typically have lot of documentation in every phase of software dev. Note: “Code-and-fix” style software development might have no documentation but it often leads to chaos. o Documentation create in such projects will generally hide the system flaws and concentrate on what works. o While documentation might make a system look "complete", in reality it might not capture or correctly communicate the real essence of its existence. o Documentation is done either in early phase or later in the development process, leading to documents that are either not

http://itnow.blogspot.com

7

Software Engineering Process using Agile Methodology

-

synchronized with the real application or documents that poorly represent the real software. Agile Approach o Documents do not contribute directly to the working of software and should be avoided if possible. o Prefer executable work packages such as customer tests and developer tests over static work packages such as plain old documentation (POD). o Efforts should be directed to create high quality source code backed by test cases instead of creating good documents. o Over all system should be "lean" and simple enough so that new developers can kick start development by looking at the source code! o Document should be created only if the benefit of having documentation is greater than the cost of creating and maintaining it. o Prefer other forms of communication over documentation. o Require people to justify documentation requests. “I need to show/deliver something to my boss” is not a legitimate request! o Documentation should be concise: overviews/roadmaps are generally preferred over detailed documentation. o Detailed code commenting should be encouraged. o Some of the circumstances where documentation might be unavoidable: project stakeholder’s communication; B2B like communication where you have a contract model (e.g. web services); communication with an “external” group; idea collection for brain storming, audit purposes, requirement/support/user documents. o In most cases, documentation efforts should be recognized and “funded” by the business. Overall documents should be understood as cost to the project and not primary project deliverable. Primary project deliverable should be the application software/service.

Communication -

Good communication is one of the keys to successful agile projects. Bad communication can very easily drive agile projects to failure. Every team member should be aware of the advantages of communicating and the disadvantages of not doing so. Project management should provide an environment where good communication can take place. People availability (especially business analyst or domain expert) is an important factor. Face to face communication should be done where ever possible. Oral communication should be preferred over written. Special measures should be taken if the team is distributed (example offshore) or working remotely (work from home).

http://itnow.blogspot.com

8

Software Engineering Process using Agile Methodology

o (daily/weekly) reports and status update meetings, which allow the team members to synchronize, discuss achievements, and reveal problems should be conducted. o In offshore outsourcing projects, development managers should be aware of language, cultural, and time-zone barriers, and must find ways to surmount these obstacles.

Challenges with Agile methodology Project Management o Agile demand that people be recognized for their skills and be treated as an important/integral part of the application development cycle. o On the contrary modern project management looks at people as "resources" and their capabilities as "roles". o From a project management perspective, final product of the project is delivered only when the user signs it off. o Agile requires "production ready" product to be delivered at the end of each iteration. o In a typical agile scenario, management shares its responsibility with development and recognizes there importance and skills. o Many such differences should be sorted out and agreed upon between management and rest of the team.

People o Requires very efficient team of developers. o People should work together and communicate frequently and openly. o Agile recognize people for there skills, expertise, creativity and uniqueness. Every team member is important and responsible for his/her part of the project. Loss of such team members often create a void space which is not easily filled by a new replacement. o Agile projects succeed fast, but if not handled properly, they can also fail fast! o "Self-Organizing Teams" often require micro management which is not always welcomed. o People must be trusted o Organizations must live with the decisions developers make

Other Challenges -

Getting started and coming up to speed might be the most difficult part. Flawed implementation of agile could be very dangerous for the project health. Loss of privacy. It takes time for people to get used to working together. Communication and management can be challenging with offshore teams. The culture of the organization must be supportive of negotiation

http://itnow.blogspot.com

9

Software Engineering Process using Agile Methodology

Agile Methods Numerous agile methods have been proposed and used over the last decade. Listed below are some of the most popular methods, often adopted in agile projects. Explaining each method is not in scope of this document. It is recommended to select a method that fits individual project and team requirement. Once the approach is finalized, it should be properly understood by all the team members (project manager, developers, testing team, business analysts, etc).

Extreme Programming (XP) -

The main aim of XP is to reduce the cost of change. XP sets out to reduce the cost of change by introducing basic values, principles and practices. By applying XP, a system should be more flexible with respect to changes. Extreme Programming is recognized for four values: Communication, Simplicity, Feedback, Courage and Respect Best Practices: - Fine scale feedback o Pair programming o Planning Game o Test Driven Development o Whole team - Continuous process o Continuous Integration o Design Improvement o Small Releases - Shared understanding o Coding Standards o Collective Code Ownership o Simple Design o System Metaphor - Programmer welfare o Sustainable Pace

http://itnow.blogspot.com

10

Software Engineering Process using Agile Methodology

Feature Driven Development (FDD) -

-

-

-

FDD blends a number of industry-recognized best practices into a cohesive whole. These practices are all driven from a client-valued functionality (feature) perspective. Its main purpose is to deliver tangible, working software repeatedly in a timely manner. FDD is a model-driven short-iteration process that consists of five basic activities. For accurate state reporting and keeping track of the software development project, milestones that mark the progress made on each feature are defined. Ffive basic activities: o Develop overall model o Build feature list o Plan by feature o Design by feature o Build by feature. Best Practices: o Domain Object Modeling: consists of exploring and explaining the domain of the problem to be solved. o Developing by Feature: If a function requires more than 2 weeks of development time, decompose it into smaller features. o Individual Class (Code) Ownership. o Feature Teams: is a small, dynamically formed team that develops a small activity. o Configuration Management: identifying the source code for all features that have been completed o Regular Builds. o Visibility of progress and results o Inspections

http://itnow.blogspot.com

11

Software Engineering Process using Agile Methodology

Dynamic Systems Development Method (DSDM) DSDM recognizes that projects are limited by time and resources, and plans accordingly to meet the business needs. In order to achieve these goals, DSDM encourages the use of RAD with the consequent danger that too many corners are cut. DSDM applies some principles, roles, and techniques.

DSDM consists of 3 phases: pre-project phase, project life-cycle phase, and post project phase. The project life-cycle phase is subdivided into 5 stages: feasibility study, business study, functional model iteration, design and build iteration, and implementation. http://itnow.blogspot.com

12

Software Engineering Process using Agile Methodology

Rational Unified Process or Agile Unified process(RUP or AUP) The RUP is not a single concrete prescriptive process, but rather an adaptable process framework, intended to be tailored by the development organizations and software project teams that will select the elements of the process that are appropriate for their needs.

-

-

Characteristics: - Iterative and Incremental - Use Case Driven(use cases are used to capture the functional requirements and to define the contents of the iterations) - Architecture Centric - Risk Focused Project Lifecycle: Inception Phase, Elaboration Phase, Construction Phase, Transition Phase. AUP is a simplified version of RUP

http://itnow.blogspot.com

13

Software Engineering Process using Agile Methodology

SCRUM -

SCRUM is a management, enhancement and maintenance methodology for an existing system. SCRUM will address new or re-engineered systems development efforts at a later date. Software product releases are planned based on the following variables : o Customer requirements - how the current system needs enhancing. o Time pressure - what time frame is required to gain a competitive advantage. o Competition - what is the competition up to, and what is required to best them. o Quality - What is the required quality, given the above variables. o Vision - what changes are required at this stage to fulfill the system vision. o Resource - what staff and funding are available.

These variables form the initial plan for a software enhancement project. However, these variables also change during the project. A successful development methodology must take these variables and their evolutionary nature into account.

Tools for Agile process In this section we will list the most popular tools for various software engineering categories.

Continuous Integration (CI) Tools -

Continuous Integration is a software engineering process that executes the build-test-deploy cycle at regular intervals. This process guarantee the latest code in all (but production) environments. Entire process is automatically handled by the continuous integration server. Integration problems are detected and fixed continuously - no last minute hiatus before release dates; Early warning of broken/incompatible code; Immediate unit testing of all changes; Constant availability of a "current" build for testing, demo, or for release purposes; The immediate impact of checking-in incomplete or broken code acts as an incentive to developers to learn to work more incrementally with shorter feedback cycles. Modern CI tools allow advanced features like report generation, alerts via IM, email, etc.

http://itnow.blogspot.com

14

Software Engineering Process using Agile Methodology

-

CI tools can interact with other software development life cycle tools like bug tracking system and eclipse IDE to provide valuable development time information.

Commercial CI servers -

-

AnthillPro - a commercial tool from Urbancode. Along with CruiseControl, Anthill is one of the original tools in this category; it remains a market and technology leader constantly adding new features such as dependency mangement, workflow automation, etc. Bamboo - Commercial continuous integration server from Atlassian Build Forge - IBM Rational's adaptive framework to standardize and automate build and release processes. Electric Commander - a continuous integration server by Electric Cloud, John Ousterhout of Tcl fame's company. TeamCity - a commercial tool by JetBrains. Web-based with IDE (IntelliJ IDEA, Eclipse, Microsoft Visual Studio) support. Includes many innovative features. QuickBuildPro - by PMEase

Open Source or Free CI Servers -

CruiseControl - Java-based framework for a continuous build process. Hudson - MIT licensed, written in Java, runs in servlet container, supports CVS, Subversion, Ant, Maven, and shell scripts. LuntBuild - a powerful open source build automation and management tool Tinderbox - a Mozilla based product Bitten - A continuous integration plugin for Trac. BuildBot - a Python/Twisted-based continuous build system CABIE - Continuous Automated Build and Integration Environment. Open source, written in Perl, works with CVS, Subversion and Perforce. DamageControl - a continuous integration server for Ruby. Apache Gump - Apache's continuous integration tool. Apache Continuum - a continuous integration server supporting Apache Maven and Apache Ant.

Test Driven Development Tools (TDD) TDD is the key for agile practices. Strict adoption of TDD results in high quality code. - junit4 - for unit testing - StrutsTestCase - for struts testing - TestNG - junit plus some new features like annotation, data-driven testing, http://itnow.blogspot.com

15

Software Engineering Process using Agile Methodology

-

DbUnit- to test database components XMLUnit- to test XML components JUnitPerf - for performance testing JMeter- load and performance testing Cobertura - to Maximizing test coverage. JSTester - Testing javascript Selenium - Automated functional testing Jester - makes some changes to the code and try to find out if the test cases catch it!

Issue/Bug/Defect Tracker -

JIRA(commercial) - Java based issue tracking system Trac – A Wiki-based issue tracking system Mylin(old name Mylar) - to manage tasks in Eclipse

Team Collaboration and Wiki Tools -

Openfire - cross-platform real-time collaboration server based on the XMPP (Jabber) protocol from Ignite. Macromedia Breeze - open source Spark - Cross-platform real-time collaboration client optimized for business and organizations. Maven 2 - can do build, reporting and documentation from a center place. Confluence - By Atlassian is an enterprise wiki that makes it easy for your team to collaborate and share knowledge. twiki - Enterprise Wiki & Collaboration Platform

Java based IDE -

Eclipse IDea (commercial) NetBeans MyEclipse - Commercial Eclipse plugin that simplify eclipse development. Weblogic Workshop - Commercial Eclipse plugin that simplify eclipse development.

Build system -

Ant Maven2 Krysalis Centipede

Source Code Management (SCM) -

Generation 0: VSS, PVCS, … Generation 1: SVN, CVS, etc..

http://itnow.blogspot.com

16

Software Engineering Process using Agile Methodology

-

Generation 2: Git, Mercurial, Bazaar

Code quality and optimization -

Checkstyle - Defining and enforcing coding standards PMD - Detecting defects and non-optimal code FindBugs - Detecting potential bugs QaLab - Build Statistics SourceMonitor (for cyclometer complexity, etc). AppPerfect

Final words: Agile methodology has been criticized in some areas that parallel or cross management practices. However, if implemented properly and thoughtfully, agile practices can make the best use of people to deliver very high quality software that meets or exceeds customer expectations. TDD, CI and collaboration tools can help simplify the agile process.

External References Agile Documentation http://www.agilemodeling.com/essays/agileDocumentation.htm http://www.ddj.com/184415786 Agile and offshore http://msdn2.microsoft.com/en-us/library/bb245671.aspx http://www.agileadvice.com/archives/2006/09/offshoredistrib.html SCM Tool comparison http://www.relisoft.com/co_op/vcs_breakdown.html http://en.wikipedia.org/wiki/Comparison_of_revision_control_software Defining Promotion Levels http://www.cmcrossroads.com/content/view/6729/202/ Streamed Lines: Branching Patterns for Parallel Software Development http://www.cmcrossroads.com/bradapp/acme/branching/ Agile and "Traditional PMI" Methods http://leadinganswers.typepad.com/leading_answers/2006/09/agile_and_tradi.html

Using Agile alongside PMBOK http://leadinganswers.typepad.com/leading_answers/files/using_agile_alongside_the_pmbok.pdf DSDM with Prince2 http://agilealliance.org/system/article/file/904/file.pdf Agile Challenges http://www.agileadvice.com/archives/2006/09/agile_challenge.html Agile Testing What is it? Can it work? http://www.io.com/~wazmo/papers/agile_testing_challenges.pdf Functional test tools http://www.opensourcetesting.org/functional.php Wiki tools Comparison

http://itnow.blogspot.com

17

Software Engineering Process using Agile Methodology

http://en.wikipedia.org/wiki/Comparison_of_wiki_software

http://itnow.blogspot.com

18