Lecture_10.pptx

SDM 5001 SYSTEMS ARCHITECTURE LECTURE 10 ARCHITECTING SOFTWARE SYSTEMS DESIGN

© LGChan

SDM 5001 SYSTEMS ARCHITECTURE LECTURE 10.1 APPLICATIONS OF SOFTWARE AND INFORMATION TECHNOLOGY SYSTEMS ARCHITECTURE

© LGChan

Software and Information Technology Systems Characteristics o Software architecture o refers to the high level structures of a software system o is set of structures (models) need to describe about the system, which comprise software elements, relations among them, and properties of both elements and relations (Clements, Bass, Kazman 2012)

o Software and Information Technology Systems must be integrated with Hardware, it cannot operate independently o Cost Structure of Software o Software: low delivery costs, high design and certification costs o Hardware: high design, manufacturing and delivery costs

o Examples Software: Computer Architecture, Database Architecture System, Internet IT Hardware: Micro-Processor Architecture, Integrated Circuit Architecture

3 © LGChan

Market Strategies of Software Architectures Build to Change instead of building to last o Consider how the application may need to change over time to address new requirements and challenges, and build in the flexibility to support this

Use Models and Viewpoints in Design o Use design tools, modelling systems, and views to o capture requirements and architectural and design decisions o analyse their impact o communicate design efficiently with stakeholders, and to enable rapid changes

o Do not formalize model design to the extent that it suppresses the capability to iterate and modify the design easily

Identify Key Engineering Decisions o Get these key decisions right early so that the design is more flexible and less likely to be damaged by changes

4 © LGChan

HOW ARCHITECTURE WINS TECHNOLOGY WARS MORRIS & FERGUSON, HBR MARCH-APRIL 1993

5 © LGChan

Success Factors in Open Architectures of Software Products of only modest capabilities can be the basis of architectural leadership … ‘good’ products don’t always win Good implementation can improve chances of establishing a standard by at least a factor of two Successful architectures tend to be proprietary, but also open at the same time General purpose architectures absorb special purpose ones Low-end systems swallow high-end systems

6 © LGChan

5 Principal Stages of Architectural Competition Commitment

Diffusion

… provide structure to the chaos

… large profits come from broad franchises

Lock In

… competitors trained to wait for the leaders next generation

Harvest

… the winner’s share of profits

Obsolescence/ Regeneration

… The better the architecture, the longer its lifespan … but sooner or later, every architecture becomes obsolete … and leaders often fail to cannibalize their old architectures 8 © LGChan

Architectural Competition The owner of the dominant architecture can quietly and precisely raise the hurdles - (whenever a particular competitor begins to pose a threat) by modifying the architecture

‘Point product’ vendors (eg. Lotus/EXCEL) are always at risk when the architectural leader changes the rules of the game IBM opened its architecture too broadly vs Apple held its architecture too closed

9 © LGChan

3 PRINCIPLES OF ARCHITECTURE OF SOFTWARE SYSTEMS

10 © LGChan

Principle 1 Separation of Concerns Separation of Concerns (also known as “divide and conquer”) A design principle for separating a computer program into distinct sections, such that each section addresses a separate concern

A computer program that implements Separation of Concerns well is called a modular program

Modularity is achieved by encapsulating information inside a section of code that has a well-defined interface

Encapsulation is a means of information hiding (eg in a function or routine) Layered designs in information systems are another embodiment of separation of concerns (eg, presentation layer, business logic layer, data access layer, persistence layer)

11 © LGChan

Principle 2 Cohesion Cohesion refers to the degree to which the elements of a software module belong together in computer science, Cohesion can be used as a measure of how strongly related the functions in a module are It is desirable to have strong cohesion in a module Example In a module, it is prefer to keep frequently used functions within the module This will reduce time to access external functions

Strong cohesion is an aspect of the principle of the separation of concerns

12 © LGChan

Principle 3 Coupling Coupling refers to the degree of dependency between two modules We always want low coupling between modules Coupling is an aspect of the principle of the separation of concerns Example Low (or Weak/Loose) Coupling provides flexibility to the programmer to create and substitute modules without having to rewrite the entire program code

Low Coupled Some dependencies

High Coupled Many dependencies

Systems with high cohesion and low coupling would automatically have separation of concerns, and vice versa 13 © LGChan

Cohesion

Coupling

Measure of Strength of association of elements within module

Measure of Interdependence of one module to another

Degree of responsibility of a single component form a meaningful unit

Describes relationship between software components

Property or Feature of an individual module

Property or Feature of a collection of modules

14 © LGChan

SDM 5001 SYSTEMS ARCHITECTURE LECTURE 10.2 NETWORK AND CLOUD ARCHITECTURE

© LGChan

Monolith Software Architectures

Monolith Architectures contain a collection of loosely related classes

Layered Monolith Architecture

Modular Monolith contains logical grouping of functionality with well defined isolation between modules

Micro-Kernel Architecture A monolithic core system with an modular API that allows physical plug-in enhancements

2 © LGChan

Software Network Architecture 1 Point-to-Point Architecture o o o o

A collection of independent systems which are connected through a network All systems have equal rights and can both use and provide services Each system has its own storage All new systems are connected directly with existing systems: resulting in increasing number of interfaces

Application 1

Application 2

Application 3

Application 4

Application 5

Application 6

© LGChan

Software Network Architecture 2 Hub-and-Spoke Architecture o A central hub takes over responsibility for communications between systems o Minimize growing interface complexity o Allows seamless data exchanges Application 1

Application 2

Middleware

Application 6

Application 5

Application 3

Application 4

© LGChan

Software Network Architecture 3 Pipeline Architecture o Independent systems along the value-added chain are integrated using a message data bus (instead of hub) o Allows more systems to be connected as the bus can be stretched over longer distances

Application 1

Application 2

Application 3

Bus

Application 4

Application 5

Application 6

© LGChan

Enterprise Service Bus An enterprise service bus (ESB) is a software architecture model used for designing and communicating between interacting software applications in a service-oriented architecture (SOA) Encapsulation of services by ESB means that the client application does not need to know anything about the location of the services, or the different communication protocols used to call them ESB decouple/separate client applications from services This enables shared, enterprise-wide use of services, and separate normal business processes from software service implementations

6 © LGChan

Enterprise Service Bus Client and Business Services and Performance Process Services

Interaction Services

Information Services

Enterprise Service Bus Partner Services

Application and Data Access Service Application 1

Example of Layered Architecture

Application 2

Application 3

Infrastructure Services © LGChan

Service Oriented Architecture o A service-oriented architecture (SOA) is the implementation of software architecture in a business environment in which application components provide services to other components via a communications protocol, typically over an enterprise service bus

o SOA consists of independent software applications that have well defined interfaces, and are orchestrated/managed through loose coupling between software applications Infrastructure Services

Client Business Services 8 © LGChan

Event-Driven Architectures Event-driven architectures (EDA) integrate several independent systems together using message queues

Broker

Broker Event Driven Architecture

Mediator Event Driven Architecture

Broker recommends the most suitable application according to message contained in the event

Hub as co-ordinator put together a package of applications for a series of processes as outline in the message

9 © LGChan

Creating Service Oriented Architecture – Systems Pattern 1 Direct Connection SOA Simplest type of direct interaction between two applications, and is based on a 1: N topology (an individual point-to-point connection) Interactions between the source and the target applications can be as complex as necessary Examples of Connection Rules : data mapping rules, security rules, and availability rules

Source Application Application

Integration Connection Rules

Target Application Application

10 © LGChan

Creating Service Oriented Architecture – Systems Pattern 2 Broker SOA (Broker Event Architecture) Similar to Direct Connection Service call from a source application is routed through a broker, and then to several target applications (this reduces number of 1: 1 connections) Distribution rules are separate from application logic Connection Pattern (interface) : depend broker rules Source Application Application

Integration Broker Rules

Target Application Application 1 Application 2

Broker Rules

Application 3

11 © LGChan

Creating Service Oriented Architecture – Systems Pattern 3 Router SOA (Mediator Event Architecture) Router decides which target application will receive the interaction Router pattern only allows 1:1 connections (vs broker pattern supports 1:N connections) Connection Pattern (interface) - router rules determine the target application

Source Application Application

Integration Router Rules

Target Application Application 1 Application 2

Broker Rules

Application 3

12 © LGChan

Distributed Computing Distributed computing brings together a network of computers to solve a computation problem Individual computers in the grid allows its own processing time to be used to solve a large problem A distributed computing architecture consists of a number of client machines with lightweight software agents installed with one or more dedicated distributed computing management servers Distributed computing is a smaller version of grid computing Examples: block chain, high performance computing, weather forecasting, SETI project, Hadoop

Node

Node

Node

Node

Distributed Computing

13 © LGChan

Grid Computing Grid computing is an infrastructure enabling the integrated, collaborative use of multi location computing resources in different organizations It is a large scale version of distributed computing system with non-interactive workloads that involve a large number of files and applications Examples: Hadron Collider at CERN, DOE Particle Physics Data grid, torrent Source Application Application

Integration Data Grid

Target Application Application Application Application

14 © LGChan

WEB SERVICES ARCHITECTURE DESIGN PATTERNS

15 © LGChan

Web Services Layered Architecture UDDI Service Discovery

Find Web services which meet certain requirements (Universal Description, Discovery and Integration)

WSDL Service Description

Services describe properties and methods

SOAP Service Invocation

Format of requests(client) and responses (server)

HTTP Service Transport

Message transfer protocol

(Web Services Description Language)

(Simple Object Access Protocol)

(Hypertext Transfer Protocol)

XML (Extensible Markup Language) is a markup language that defines a set of rules for encoding documents in a format which is both human-readable and machine-readable WSDL UDDI (Universal Description, Discovery, and Integration) is a XML-based registry for businesses worldwide to list themselves on the Internet WSDL (Web Service Definition Language) is an XML format for describing network services as a set of endpoints operating on messages containing either document-oriented or procedure-oriented information SOAP (Simple Object Access Protocol) is a messaging protocol that allows programs that run on disparate operating systems to communicate using Hypertext Transfer Protocol (HTTP) and XML 16 (Ref: http://www.w3schools.co© m/L)GChan

Client Server Systems Architecture Pattern request

Two Tier (Classic Model) client

server

response

Three Tier

client

Server/client

server

N Tier server

client

Server/client

Server/client server

17 © LGChan

Peer-to-Peer Systems Architecture Pattern peer peer

peer

peer

peer peer peer

Example: BitTorrent Gnutella eDonkey Skype 18 © LGChan

CLOUD COMPUTING ARCHITECTURE

19 © LGChan

Cloud Computing – Concept of Operations

Social Networks, Scientific Computing, Enterprise Applications

Applications (SaaS) Frameworks for Cloud Application Development

Virtual hardware, networking, OS images, and storage

Middleware (PaaS)

Hardware and OS (IaaS)

20 © LGChan

NIST Conceptual Reference Architecture

21 NIST 2011 Cloud Computing Reference Architecture

© LGChan

Service Orchestration Service Orchestration is the composition of systems components to support the Cloud Services Providers activities in arrangement, coordination and management of computing resources in order to deliver cloud services to Cloud Consumers

Service Layer Define interfaces for Cloud Consumers to access computing services Example: Web Services

Resource Abstraction and Control Layer Provide and manage access to physical components Resource Abstraction: ensure efficient, secure, and reliable usage of physical resources Example: hypervisors, virtual machines, virtual data storage Control: enable resource pooling, dynamic allocation, measured services

Physical Resource Layer All physical components Example: computers, networks(routers, firewalls, switches, interfaces), storage (hard disks) and infrastructure (power, HVAC, communications)

22 © LGChan

Interactions of Stakeholders

Role of Cloud Broker

Role of Cloud Provider and Cloud Carrier

Role of Cloud Auditor

23 © LGChan

5 Essential Characteristics of Cloud Computing On-demand self-service

Broad network access

Resource Pooling

Rapid Elasticity

Measured Service

Sufficient agility and autonomy, that requests for more resource that are automatically and dynamically provisioned in real time, without human intervention

Available over networks such as the Internet, using established mechanisms and standard protocols using access devices such as personal computers, portable computers, mobile phones and tablet devices

Brings together aspects of grid computing (where multiple compute resources are connected together in a coordinated way) and hardware virtualisation

Requests for extra resource are selfmanaged and automatic in relation to demand From consumer perspective, the supply of compute resources is limitless

Providers dynamically optimise the underlying infrastructure and provide a transparent metering service (monitor, control, and report) at a level of abstraction that is relevant to the consumer

24 NIST 2011 Cloud Computing Reference Architecture

© LGChan

CLOUD COMPUTING SERVICE MODELS

25 © LGChan

Three Cloud Deployment Models

SaaS Software as a Service o

Capability provided to the consumer is to use the provider’s applications running on a cloud infrastructures

o

Applications are accessible from various client devices through either a thin client interface, such as a web browser (e.g., web-based email), or a program interface

o

Consumer does not manage or control the underlying cloud infrastructure including network, servers, operating systems, storage, or even individual application capabilities

PaaS Platform as a Service o

Capability provided to the consumer is to deploy onto the cloud infrastructure consumer-created or acquired applications created using programming languages, libraries, services, and tools supported by the provider

o

Consumer does not manage or control the underlying cloud infrastructure including network, servers, operating systems, or storage, but has control over the deployed applications and possibly configuration settings for the application-hosting environment

IaaS Infrastructure as a Service

o

Capability provided to the consumer is to provision processing, storage, networks, and other fundamental computing resources where the consumer is able to deploy and run arbitrary software, which can include operating systems and applications

o

Consumer does not manage or control the underlying cloud infrastructure but has control over operating systems, storage, and deployed applications; and possibly limited control of select networking components (e.g., host firewalls)

26 © LGChan

SaaS Scope of Control and Software Stack Cloud Consumer

Cloud Provider Admin Control

Application eg email Middleware eg Java Operating System

Total Control

Hypervisor

Limited Admin Control User Level Control No Control

Hardware

27 © LGChan

PaaS Scope of Control and Hardware Component Stack Cloud Consumer

Cloud Provider No Control Admin Control

Application eg email Middleware eg Java

Admin Control

Make Requests

Operating System Total Control

Hypervisor

No Control

Hardware

28 © LGChan

IaaS Scope of Control and Component Stack Cloud Consumer

Cloud Provider No Control

Application eg email Middleware eg Java

Total Control

Guest Operating System Admin Control Total Control

Hypervisor Hardware

Make Requests No Control

29 © LGChan

CLOUD COMPUTING DEPLOYMENT MODELS

30 © LGChan

Cloud Deployment Configurations

Private Cloud

Community Cloud

Hybrid Cloud

Public Cloud

Cloud within an organization

Cloud infrastructure jointly owned by several organizations

Composition of two or more cloud models

Cloud infrastructure owned by an organization, provided to general public as service

common physical server dedicated physical server

dedicated servers in cloud virtual server instance in cloud

31 © LGChan

3-4-5 Principle of Cloud Computing Public

Private

Hybrid

Community

Applicatio n

SaaS

Runtime

PaaS

Virtual M achines

IaaS

Location transparent Resource pooling

On demand self service

Consumption based charge book model

Ubiquitous network access

Rapid elasticity

32 © LGChan

INTERNET OF THINGS MODELS

33 © LGChan

Characteristics of Internet of Things Distributive o Highly distributed environment o Data from many different sources and processing centers

Interoperability o Data exchange seamlessly to achieve common goals

Scalability o Increasing number of connected devices –Machine2Machine,Machine2Human o Infrastructure scalable

Resources Scarcity o Heavy demand on power consumption of devices and computing resources

Security

o Concerns with data privacy and network security

34 © LGChan

Network of Layered Architectures 7 Application Layer

Application

Business Layer

6 Presentation Layer

Application Layer

5 Session Layer

Processing Layer

4 Transport Layer

(Host-to-Host) Transport

3 Network Layer

Internet

2 Data Link Layer 1 Physical Layer OSI Model

Network Interface

Transport Layer

Perception Layer

(Hardware) TCP/IP Model

Internet of Things Model

35 Reference Wu Lu Ling 2010 Research on Architecture of Internet of Things. 2010 3rd International Conference on Advanced Computer Theory and Engineering

© LGChan

Layered Internet of Things Architecture Business Uses final results for business application. May require additional tools to achieve this

Application Analyzes data for decision making and action. Require additional tools to enable final application, eg mobile phones, smart cards

Middleware Receives data from Network. Manage and store data for further processing. Transmits data to application layer for analysis

Network Processing collects data from various sensors. Cloud technology is used here because it has reliable and dynamic interface

Sensors Perceive physical properties of environment and convert information to digital signals. RFID, WSN, GPS, NFC technologies are used

36 © LGChan

Sensing Technologies Radio Frequency Identification (RFID) Sensors o Passive RFID Tag – power from tag reader applications: supply chains, passports, item level tracking, bank cards, toll cards o Active RFID – power from battery applications: manufacturing, remote sensing o Semi Passive RFID - battery for chip and power from reader

Auto-ID

38 © LGChan

Sensing Technologies Wireless Sensor Networks (WSN) o Distributed network of sensor nodes to monitor states such as location, movements, environment o Sensor nodes are connected in a wireless multi-hop design Sensing results are reported to a smaller number of special nodes (sinks or base stations) Applications : GE jet engines, American Airlines airplane

39 © LGChan

Middleware Software Interface between Physical Layer and Transport Layer o Hides complexity of numerous heterogeneous devices o Apply SOA based on interaction models as a virtual network in order to support an unknown and dynamic network topology

Sensor Cloud Architecture

Source: http://www.hindawi.com/journals/ijdsn/2013/917923/

40 © LGChan

Example: IoT in Road Vehicle Monitoring

41 Source: Li Cheng Zhou 2015 Road Vehicle Monitoring System Based on Intelligent Visual Internet of Things . Journal of Sensors Volume 2015

© LGChan

Example: IoT Application in Enterprise DEVICES

CLOUD

ENTERPRISE

Open, Secure, Intelligent

Mediation, Security, Management

Process Integration, Action and Control, Data Analytics

42 © LGChan

Example: IoT Application in Manufacturing

43 © LGChan

EXAMPLE: DIGITAL PLATFORM ARCHITECTURE

44 © LGChan

Example: Digital Platform Platform A platform is an open architecture, together with a governance model, designed to facilitate transactions

Open Architecture: allows third parties to participate, build, and innovate Governance: gives power to exclude bad actors, enforce community behaviour, and monetize Interactions: means by which value is created

Functions o Pull participants to the platform (by offering incentives) o Provide infrastructure on which participants create and exchange value o Match participants with each other and with content/goods/services created on the platform

Participants (Users) Producers Consumers

users offering/selling products, services, and information users seeking/buying products, services, and information 45 © LGChan

Example: Digital Platform Architecture Platform Layered Architecture

User Request

Network Infrastructure and Community Technology Tools

Producer + Consumer Transaction

Market and Consumer Data User Experience (Feedback Positive and Negative)

Functions

Architecture Forms in Layer

Pull Users to Platform Infrastructure Match Participants

Community Network Infrastructure Technology Tools with Market and Consumer Data

Architecture Form

Examples

Network Infrastructure Community Technology Tools

Website, servers, data warehouse Accounts of Users, email Data Filtering, Data Mining, Artificial Intelligence, Security, Shopping Cart, Payment System Profiles of users

Market Consumer Data

46 © LGChan

Example: 3 Basic Configurations in Digital Platform Architecture Network Infrastructure and Community Network Infrastructure and Community

Network Infrastructure and Community Technology Tools

Technology Tools Technology Tools Market and Consumer Data

Configuration 1

Examples

Strong in Community Airbnb, Uber Strong in Infrastructure Amazon, Google

Market and Consumer Data Market and Consumer Data

Configuration 2

Examples

Strong in Technology Tools Android, Google, Tencent Strong in Matching Netflix, Amazon

Configuration 3

Examples

Strong in Consumer Data Strong in Market Data

Facebook eBay, Alibaba

47 © LGChan

END OF LECTURE 10.2 NETWORK ARCHITECTURES

48 © LGChan

SDM 5001 SYSTEMS ARCHITECTURE LECTURE 10.3 ARCHITECTING SOFTWARE DEVELOPMENT PROCESS AGILE, SCRUM, XP, CD, DevOps

© LGChan

Introduction Early software development before 1990s features o huge effort during the planning phase o poor requirements conversion in a rapid changing environment o rigid and non-responsive to customer needs o treatment of staff as a factor of production Agile evolved from different lightweight software approaches in the 1990s and is a response to project managers’ dislike of the rigid, linear Waterfall methodology Agile focuses on flexibility, continuous improvement, and speed in software development 2 © LGChan

What is Agile? What is Agile? Agile (or agility) is a mindset practice (or philosophy) and ability to create and respond to change in order to succeed in an uncertain and complex environment Agile is an adjective, that is a descriptive qualifier, never a noun eg an agile team, an agile project Agile is o Defined by Values o Guided by Principles o Manifested through many different practices Practices

Practices bring Accountability to Values

Principles

Principles bring Values and Practices by guiding Behaviour

Values

Values bring purposes to Practices

3 © LGChan

Agile Manifesto Agile Manifesto 4 Values in Agile: 1. Individuals 2. Product 3. Customer 4. Change Agile Value and Focus 1.Individuals and Interactions 2.Working Software Product 3. 4. Customer Collaboration Responding to Change

Traditional Value and Focus 1. 2. 3. 4.

Processes and Tools Comprehensive Documentation Contract Negotiation Following a Plan

while there is value in the items on the right, we value the items on the left more 4 © LGChan

Agile Manifesto 12 principles 1. Our highest priority is to satisfy the customer through early and continuous delivery of valuable software 2. Welcome changing requirements, even late in development Agile processes harness change for the customer's competitive advantage 3. Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale 4. Business people and developers must work together daily throughout the project 5. Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done 6. The most efficient and effective method of conveying information to and within a development team is face-to-face conversation 7. Working software is the primary measure of progress 8. Agile processes promote sustainable development. Sponsors, developers, and users should be able to maintain a constant pace indefinitely 9. Continuous attention to technical excellence and good design enhances agility 10. Simplicity - the art of maximizing the amount of work not done - is essential

11.The best architectures, requirements, and designs emerge from self-organizing teams 12. At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly

5 © LGChan

What is Agile Software Development? o o o o

A collection of a set of methods and practices based on an incremental, iterative approach over a period of time Performed by self-organizing and cross functional teams with just-in-time activities To produce high quality software in a cost effective and timely manner To meet changing needs of the stakeholders

Example of Agile Practices (Methods) o o o o o o

Scrum Extreme Programming (XP) Adaptive Software Development (ASD) Scrumban Kaizen and Kanban Lean Software Development

54% 11% 9% 7% 4%

6 © LGChan

Agile EcoSystem of Tools

7 © LGChan

SCRUM

Ref: Hirotaka Takeuchi and Ikujiro Nonaka (1986) The New Product Development Game. Harvard Business Review, January 1986 Issue

8 © LGChan

Time

Time

Analysis

Analysis

Design

Design

Code

Code

Analysis

Traditional Process

Back to Back Continuous Work

Analysis

Lean Process 25% work done = 100% usable

9 © LGChan

Lean Principles applied to Software Development

10 © LGChan

Iterative and Incremental Iterative

Incremental

Iterative and Incremental o Iterative to find and improve solutions o Increment to add functionality

11 © LGChan

Scrum Introduction Scrum is an agile process that allows development team to deliver customer value very early, and in a highly predictable manner

o Allows rapid and repeated inspect actual working software (every two weeks to one month) o Development teams are self-organized to determine the best way to deliver the highest priority features o Everyone, in a sprint meeting held every two weeks to a month, can report, see real working software and decide to release it as is or continue to enhance for another iteration

Sequential Development

Overlapping Development

12 © LGChan

What is Scrum? Referee Half Fly

Half Fly

What is the Purpose of a Rugby Scrum?

What is the Purpose of a Scrum Development?

To bring order to the chaos of a live ball on the field when no team has clear possession of the ball yet, eg after a minor infringement of rules

To bring order to the chaos of a development project Development requirements from the customer will often change, so Scrum uses sprints to speed through known requirements, while also being able to quickly adapt to project changes

Video (1:01) https://www.youtube.com/watch?v=hFXHRb-kHt8

Referee = Scrum Master Coach = Product Owner Players = Development Team

13 © LGChan

The Chicken and the Pig Riddle Riddle : In a bacon and egg breakfast, what's the difference between the chicken and the pig?

A Pig is someone who has skin in the game Pig roles are considered core team members, that is, Performers People who “do” work, including the Product Owner and Scrum Master A Chicken is someone who has something to gain by the Pigs performing, but in the end, really do not contribute day to day to “getting things done.” Their “eggs” are a renewable resource, and many eggs get laid The chicken is Involved, but the pig is COMMITTED 14 © LGChan

Scrum Overview

15 © LGChan

Color Code used in Slides

16 © LGChan

SCRUM TERMS

17 © LGChan

Scrum Framework (3 – 3 – 5) 3 Roles :

5 Ceremonies (Events) :

Product Owner, Scrum Master, Team

Sprint, Sprint Planning, Sprint Review, Sprint Retrospective, Daily Scrum Meeting

3 Artifacts (list of work to be done or deliverable) : Product Backlog, Sprint Backlog, and Burndown Chart

18 © LGChan

Roles in Scrum Team

19 © LGChan

Scrum Roles Product Owner Responsibilities o product backlog and priority o business value of the project o profitability of the product (ROI) o Define the features of the product o Decide on release date and content o Prioritize features according to market value o Adjust features and priority every iteration, as needed o Accept or reject work results

Team The Team self-organizes to get the work done o 5-10 people committed to project o Cross-functional : generalising specialists, eg QA, Programmers, UI Designers, etc. o Members should be full-time : May be exceptions (eg., System Admin, etc.) o Teams are self-organizing o Membership can change only between sprints o Deliver value in small chunks within every sprint, focused on customer, build in quality

Scrum Master Scrum Master represents management to the project and ensures that the team is functional and productive o Responsible for enacting Scrum values and practices o Removes impediments o Ensure that the team is fully functional and productive o Enable close cooperation across all roles and functions o Shield the team from external interferences

20 © LGChan

Scaling with Scrum

21 © LGChan

4 Feedback Loops in Scrum 1 day 30 day

1. Daily feedback between development team members as they go about developing each project requirement 2. Direct and daily interaction between the product owner and the development team for on‐the‐spot question answering and feedback 3. Direct feedback from the product owner as they literally accept or reject every requirement completed 4. At the end of each sprint, feedback from internal stakeholders is received 5. At the end of every release, feedback from the external marketplace 22 © LGChan

5 Scrum Ceremonies Sprint Planning the team meets with the product owner to choose a set of work to deliver during a sprint o Each task/work estimated @ 1- 16 hours Release Planning after an initial feature list has been identified, prioritized, and potentially estimated, the team plans an overall release schedule and determine which features can likely be delivered o Used to feed individual sprint plans Sprint Scrum projects make progress in a series of “sprints” in which product is designed, coded, and tested during the sprint Daily Scrum the team meets each day to share struggles and progress Sprint Reviews the team demonstrates to the product owner what it has completed during the sprint Done features are presented to Product Owner o Scrum Master, Product Owner, Team, Client(optional). Meeting 1- 3hours Sprint Retrospectives the team looks for ways to improve the product and the process by identifying the positive and negative impacts during the sprint o Scrum Master, Team have 1– 4 hours discussion o Whole team gathers and discusses what they would like to: Start Doing, Stop Doing, Continue Doing

23 © LGChan

Scrum Artefacts Product Backlog

Sprint Backlog

Burn down/up Chart

Product Backlog ordered list of ideas for the product Prioritized by value delivered to the customer o Reprioritized at the start of each sprint

Sprint Backlog set of work from the product backlog that the team agrees to complete in a sprint, broken into tasks o Individuals sign up for work of their own choosing o Work is never assigned

Burn down/up Chart shows the cumulative work remaining in a Sprint, day-by-day o Techniques/Tools : Fibonacci Series, Pareto Analysis, Party Poker

User Story A user story is a very high-level definition of a requirement, containing just enough information so that the developers can produce a reasonable estimate of the effort to implement it.

“As a [type of user], I can [goal] so that [value]” List the assumptions, constraints, priority, and estimated time required A Story Point is used to estimate the size/importance of the User Story relative to other user stories in the project 24 © LGChan

What is Product Backlog? Product Backlog is a list of all desired work on the project managed and owned by a Product Owner o Usually a combination of o story-based work (“let user search and replace”) o task-based work (“improve exception handling”) List is prioritized by the Product Owner o Typically a Product Manager, Marketing, Internal Customer, etc. o Created during Sprint Planning Meeting o Recorded on a Spreadsheet, Kanban

25 © LGChan

What is Sprint Backlog? A Sprint Backlog is a subset of Product Backlog Items, which define the work for a Sprint o o o o o

Is created ONLY by Team members Each Item has its own status Should be updated every day Contains no more than 300 tasks in the list If a task requires more than 16 hours, it should be broken down o Team can add or subtract items from the list. Product Owner is not allowed to do this

26 © LGChan

Sprint Backlog Sprint Backlog

Development Team

Burndown Chart

27 © LGChan

SCRUM ACTITIVIES

28 © LGChan

Scrum Sprint A sprint is the basic unit of development in Scrum Each Sprint begins with the Daily Scrum Meeting The sprint is a "time boxed" effort ie. it is restricted to a specific duration. The duration is fixed in advance for each sprint and is normally between one week and four weeks

NO outside influence can interfere with the Scrum team during the Sprint

Agenda of Sprint o a planning meeting where the tasks for the sprint are identified and an estimated commitment for the sprint goal is made o a review or retrospective meeting where the progress is reviewed and lessons for the next sprint are identified

29 © LGChan

Sprint Execution Overview

30 © LGChan

What is the Daily Scrum

Sprint Goals

31 © LGChan

Daily Scrum What is Daily Scrum? o A meeting in which team members make commitments to each other and to the Scrum Master o A good way for a Scrum Master to track the progress of the Team

How to Do a Daily Scrum? oDaily o 15-minutes duration o Stand-up meeting o Not for problem solving

Three questions to Address

What is NOT a Daily Scrum? o Is NOT a problem solving session o Is NOT a way to collect information about WHO is behind the schedule

o What did you do yesterday? o What will you do today? oWhat obstacles are in your way? Scrum Master only listen, he must not ask questions

Who are Participants? Chickens and pigs are invited oHelp avoid other unnecessary meetings Only pigs can talk 32 © LGChan

Scrum Daily Standup - Videos

What is the 'Daily Scrum'? Video (2:51) https://www.youtube.com/watch?v=zxwEdbcrqD8

Scrum at Microsoft Video (3:48) https://www.youtube.com/watch?v=YR84qH6d7QE

33 © LGChan

Sprints

Daily Scrum

Product Backlog

Product Owner

Sprint Planning

Sprint Goal

Potentially Shippable

34 © LGChan

Sprint Execution

PBI = Product Backlog Item

35 © LGChan

What is Sprint Review?

36 © LGChan

Sprint Review Meeting

Sprint Review Meeting o Team presents what it accomplished during the sprint o Demo of new features or underlying architecture o Informal : 2-hour prep time rule o Participants: Stakeholders (users, management), Product Owner, Scrum Team

37 © LGChan

What is Sprint Retrospective?

38 © LGChan

Sprint Retrospective Meeting

Sprint Retrospective Meeting Scrum Team only + others Feedback meeting Three questions 1. Start 2. Stop 3. Continue

39 © LGChan

Potentially Shippable Product Increment

40 © LGChan

What is a Burn Down Chart? Sprint Burn Down Chart

Product Burn Down Chart

o Depicts the total Sprint Backlog hours remaining per day o Shows the estimated amount of time to release o Ideally should burn down to zero to the end of the Sprint o Actually is not a straight line o Can bump UP

o Is a “big picture” view of project’s progress (all the releases)

41

Video (2:51) https://www.youtube.com/watch?v=rUJsqEHRcJA

© LGChan

SCRUM VS OTHER SOFTWARE ARCHITECTING DEVELOPMENT METHODS 42 © LGChan

Advantages and Disadvantages of Scrum Advantages of Scrum

Disadvantages of Scrum

More transparency and project visibility Daily stand-up meetings improves workflow

Risk of scope creep Some Scrum projects can experience scope creep due to a lack of specific end date

Increased team accountability Self organization seeks best approach

Team requires experience and commitment Team needs to commit to the daily Scrum meetings and to stay on the team for the duration of the project.

Easy to accommodate changes With short sprints and constant feedback, it’s easier to cope with and accommodate changes

Wrong Scrum Master can ruin everything If the Scrum Master tries to control the team, the project will fail.

Increased cost savings Constant communication ensures the team is aware of all issues and changes as soon as they arise, helping to lower expenses and increase quality

Poorly defined tasks can lead to inaccuracies If the initial goals are unclear, planning becomes difficult and takes more time

43 © LGChan

Scrum vs Waterfall

Scrum method is a flexible team approach with the whole unit working together and aggressively trying to advance the ball

Waterfall method is the relay race approach (a linear process) where every individual is doing their part and then passing the baton to the next person

The scrum team need to have a cross functional people and they have to be focussed on advancing the common goal

The waterfall team need to have a specialist in each process stage

44 © LGChan

When to Use Agile and When to Use Waterfall Use Agile (Scrum, XP, Kanban)

Use Waterfall

o you want to benefit from fast feedback and burning visibility of objective data o budget and time are fixed, but requirements are changing o your payoff/downside curves could vary widely o a team passionate about it or a coach who will help them o complicated project without all the experts you need or a complex project

o project is simple o requirements are fixed o project is complicated, but you have the expertise to deliver it o it is all you know and you have no support for change o initial investment is not risky to make o you focus your performance measures on delivery date and budget

45 © LGChan

EXTREME PROGRAMMING

Ref: Beck, Kent (1999) Extreme Programming Explained. Addison Wesley

46 © LGChan

Extreme Programming Extreme Programming (also known as XP ) introduced by Kent Beck in 1990s XP is an agile software development intended to improve quality and responsiveness to evolving customer requirements Extreme Programming is similar to Scrum

Some Differences between Scrum and XP Work in Scrum Teams

Work in XP Teams

o work in iterations (sprints) which are 2 weeks to 1 month long o do not allow changes into their sprints o determines the sequence in which they will develop backlog items which are first selected by the product owner o Scrum does not prescribe any engineering practices

o work in iterations which are 1 or 2 weeks long o changes are more flexible. They swap work in between o work in a strict priority order according to features prioritized by the product owner o XP has a prescribed order to develop the features

47 © LGChan

Extreme Engineering New Requirements

Analyze

Module X

Design/ Improve

SCRUM Team X

Daily Scrum

Evaluate Test Modules Interfaces

Evaluate

Module Y

Architecture

Design/ Improve

Analyze

Integrate Test

Daily Scrum

SCRUM Team Y

Evaluate Evaluate

Test

Launch

Changes 48 © LGChan

Example: Rapid Evolution of Wikispeed Wikispeed is project created in April 12, 2007 for a competition with the stated objective to build street legal car that can o Achieve 100+ miles per gallon of fuel economy o Produce less than 200 gm/mile of CO2 emissions o Be built for the mass market

Modular Components 49 © LGChan

CONTINUOUS DELIVERY

Ref: Humble, Jez, and Farley, D (2010) Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation. Addison-Wesley

50

© LGChan

What is Continuous Delivery? Continuous Delivery is the ability to get changes of all types (including new features, configuration changes, bug fixes and experiments) into production, or into the hands of users, safely and quickly in a sustainable way Continuous Delivery ensure the code is always in a deployable state, even in the face of teams of thousands of developers making changes on a daily basis It eliminates the integration, testing and hardening phases that traditionally followed “dev complete”, as well as code freezes

Principles of Continuous Delivery Build quality in, do not leave build broken down Work and Commit in small batches Development in modules, teams can work independently and simultaneously Relentlessly pursue continuous improvement Build should return a status, a log, and a deliverable Computers perform repetitive tasks, people solve problems Use software tools to perform automated tasks 51 https://continuousdelivery.com/

© LGChan

Automation Tools used in Continuous Delivery Continuous Integration

Configuration Management

To put(pull) together all changes made in the architecture Example: Jenkins, Hudson

To keep environments consistent throughout the software development process, from the developer’s laptop until production across all the stages of the delivery pipeline Example: Puppet, Chef

Version Control To keep a record of every version of every feature, add-on or any other change to the code base Example: Git, Subversion

Code Review To help the developer to step through code and see where the differences in code are present. To note which changes are acceptable and which are not before changes are merged with the main code base Example: GitHub, Crucible

Monitoring To collect and analyse data in testing environments, as well as other parts of the infrastructure Example: Logstash/Elastic, Graphite

Orchestration To manage the changes, updates, or complete applications in any specific order Example: Chef, Puppet 52 © LGChan

DEV OPS

53 © LGChan

What is DevOps? DevOps (a combination of the words development and operations) DevOps is based on Agile and Lean principles in which business owners, development, operations, and quality assurance team collaborate to deliver software in a continuous stable manner A collaborative way of developing and deploying software method that stresses communication, collaboration and integration between software developers and information technology (IT) operation professionals working together in the entire service lifecycle, from design through the development process to production support

What is DevOps Video (7:06) https://www.youtube.com/watch?v=_I94-tJlovg 54 © LGChan

© LGChan

One Team, One Goal in DevOps

o Developers always want to deliver changes as soon as possible o Operations want reliability and stability

Development uses some tools and operation uses some other tools to perform the same stuff

Shorten Lead Time

Feedback

Dev Ops focuses on agility responsiveness and concern with quality and stability on the ultimate goal of providing business value

56 © LGChan

Software Development Processes with Lean Methods

Value Idea DevOps Agile Development

Practices

Principles

Running Lean Product / Market Analysis (Finding and Validating Ideas)

Scrum, Kanban

XP

(Process)

(Engineering)

Operations

Lean (End-to-End Flow)

57 © LGChan

Continuous Improvement = Continuous Delivery Embed Project Knowledge into Operations

Extend Delivery to Production

OPS

DEV Extend Operations Feedback to Project

Embed Operations Knowledge into Project

o o o o o

Operate for Design Build Measure Learn Repeat 58 © LGChan

END OF LECTURE 10.3 AGILE, SCRUM, EXTREME, CONTINUOUS DELIVERY

59 © LGChan