Cisco Waas

Cisco WAAS (Wide Area Application Services)

Technical Overview

Philip Nedev - SE [email protected]

WAAS4.0

© 2005 Cisco Systems, Inc. All rights reserved.

Cisco Public

1

1

Agenda • Enterprise Application Delivery Challenges • Introducing Cisco Wide Area Application Services • Network Integration and Deployment • In-Depth Examination of Optimizations • Management and WAE Platforms • Summary • Q&A

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

2

The WAN Is A Barrier To Consolidation • Applications are designed for LAN environments

Round Trip Time (RTT) ~ 0mS LAN Switch

High bandwidth

Server

Client

Low latency Reliability

• WAN characteristics hinder consolidation Round Trip Time (RTT) ~ many many milliseconds

Already congested Low bandwidth

Client

LAN Switch

Routed Network

LAN Switch

Server

Latency Packet Loss

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

3

Bandwidth • Bandwidth constraints keep applications from performing well • Too much data and too small of a pipe causes congestion, packet loss, and backpressure

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

4

Packet Loss, Congestion, and Retransmission • Packet loss and congestion cause retransmission which hinders application performance and throughput • Commonly caused by saturated device transmit queues in the network path

Packet Loss Congestion

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

5

The Impact of Packet Loss R=

4,510 4,010 Throughput (Mbps)

R

MSS 1 .2 RTT p 0.5 : Average Throughput

3,510

MSS: Packet Size

3,010

RTT: Round-Trip Time P

: Packet Loss

2,510 2,010 1,510 1,010 510 10 0.00001%

0.0001%

0.001%

0.01%

0.1%

1.0%

Packet Loss Probability Assuming 1250-Byte packet size, and 100ms RTT CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

6

Latency • Latency impairs application performance in three ways: Network latency – the amount of time necessary for a message to traverse the network Transport latency – the amount of time necessary for the transport mechanism (TCP) to acknowledge and retransmit data Application latency – “chattiness” of an application protocol causing messages to be exchanged across the network

Round Trip Time (RTT) ~ many many milliseconds

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

7

The Impact of Latency R= R

MSS 1 .2 RTT p 0.5 : Average Throughput

MSS: Packet Size

Throughput

RTT: Round-Trip Time

Expected Expected

P

: Packet Loss

1.544Mbps

Actual Actual

500Kbps

80 ms Round Trip Time (RTT) CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

8

Need for Application-Specific Acceleration • Many application protocols can not be adequately optimized through simple compression and transport optimizations alone • Application protocols are commonly developed in “utopian” environments, i.e. the client and the server are on the same LAN or very close to one another • Application-induced or protocol-induced latency and unnecessary data transfers hinder overall enduser performance

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

9

Need for Application-Specific Acceleration • The Common Internet File System (CIFS) and Network File System (NFS) are two examples of such protocols • CIFS and NFS make a portion of a local file system network accessible, and must maintain all of the semantics of the local file system itself, including: User (or process) authentication and authorization Information security Locating information, directory traversal File access control and locking semantics I/O operations, including open, close, read, write, seek CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

10

Need for Application-Specific Acceleration • The result is that hundreds upon thousands of messages must traverse the network before any usable data is served or function is completed! Protocol version selection User authentication User authorization Meta data operations Find file

File open, FID Lock segment ranges Read data

Write data

Close file

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

11

Need for Application-Specific Acceleration • Applying compression to communications between the client and server certainly minimizes the amount of bandwidth consumed by each protocol message But many hundreds or thousands messages must still go back and forth across the WAN in sequence!

• Applying transport optimizations to communications between the client and server improves the ability of each message to efficiently and fully utilize available network capacity But many hundreds or thousands of messages must still go back and forth across the WAN in sequence!

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

12

Need for Application-Specific Acceleration • In this simple example of a 1MB Word document open, over 1000 messages are exchanged • With a 40mS RTT WAN, this equates to over 52 seconds of “wait” time before the document is usable! CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

13

Agenda • Enterprise Application Delivery Challenges • Introducing Cisco Wide Area Application Services • Network Integration and Deployment • In-Depth Examination of Optimizations • Management and WAE Platforms • Summary • Q&A

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

14

Cisco WAAS Overcomes the WAN • Cisco WAAS is a solution that leverages a hardware footprint (WAE) in the remote office and in the data center to overcome application performance problems in WAN environments Remote Office

Data Center

Remote Office

WAN O Oppttiim miizzeed dC Coonnnne eccttiioon nss ionnss nneeccttio n o n C o d C e ized imiz O Oppttim

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

15

Cisco WAAS Enables Consolidation • Cisco Wide Area Application Services (WAAS) Transparent integration Robust optimizations Auto discovery

• Infrastructure Consolidation WAN

Remove costly servers Centralize data protection Save WAN resources

• Application Acceleration Application adapters Advanced compression Throughput optimizations

Server and Server and storage storage infrastructure infrastructure

Cisco Cisco WAE WAE

Policy-based configuration CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

16

Cisco WAAS Print Services • Centrally Managed Print Services Print driver distribution Client driver download repository Status and health reporting

• Supports Any Printer Full feature compatibility Job control and status monitoring Guest and disconnected printing

• Print Server Configuration Network parameters (IP, name, etc) Queue definition and ACLs

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

17

WAAS Accelerates Broad Range of Applications Application

Protocol •

Windows (CIFS)

•

UNIX (NFS)

•

Exchange (MAPI)

•

SMTP/POP3, IMAP

•

Notes

Internet and Intranet

•

Data Transfer

File Sharing

Email

Software Distribution

Database Applications

Data Protection

Other

Typical Improvement •

2X-100X

•

2X-50X

HTTP, HTTPS, WebDAV

•

2X-50X

•

FTP

•

2X-50X

•

SMS

•

Altiris

•

2X-100X

•

SQL

•

Oracle

•

2X-10X

•

Notes

•

Backup Applications

•

Replication Applications

•

2X-10X

•

Any TCP-based Application

•

2X-10X

* Performance improvement varies based on user workload, compressibility of data, and WAN characteristics and utilization. Actual numbers are case-specific and results may vary. CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

18

Cisco WAAS Performance – File Services Operations over T1 (1.544Mbps), 80mS RTT 20 Seconds

40 Seconds

60 Seconds

80 Seconds

Opening 5-MB PowerPoint Saving 5-MB PowerPoint Drag and Drop of 5MB PowerPoint

Legend Operation Over Native WAN First Operation with WAAS Future Operation with WAAS 20 Seconds

40 Seconds

60 Seconds

Download of 8MB Package Microsoft SMS

80 Seconds

Legend Operation over native WAN First operation with WAAS, no preposition First operation with WAAS, with preposition Future operation with WAAS

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

19

Cisco WAAS Optimization Architecture

L7: Application Optimization

Network Infrastructure

Unified Management Management Unified

L4: Transport Optimization

Video Web Video Web

Local Enterprise File Local File Enterprise Email Email Services Apps Services Services Services Apps

Content Content Distribution Distribution

TCP TCP Flow Flow Optimizations Optimizations (TFO) (TFO)

Persistent Persistent Session-Based Session-Based Compression Compression

Other Other Apps Apps

Data Data Redundancy Redundancy Elimination Elimination (DRE) (DRE)

Application Application Classification Classification and and Policy Policy Engine Engine Logical Logical and and Physical Physical Integration Integration Security Security

Monitoring Monitoring

Quality Quality of of Service Service

Core Core Routing Routing & & Switching Switching Services Services

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

20

WAAS File Services Introduction • Cisco Wide Area Application Services provides the industry’s most innovative and robust file services optimizations: Application protocol proxy (CIFS, NFSv2) to handle protocol message workload at the edge to mitigate the impact of latency Application data and meta data cache to serve usable content at the edge to mitigate unnecessary data transfers when safe Network compression (DRE, LZ) to minimize bandwidth usage during data transfer scenarios (read or write) TCP optimizations (TFO) to improve utilization of the available network capacity

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

21

WAAS File Services Introduction • • Intelligent Intelligentlocal localhandling handlingand andoptimization optimizationofof protocol mitigates latency protocol mitigates latency • • File Filecaching cachingtotoremove removethe theneed needtoto unnecessarily unnecessarilytransfer transferfiles, files,validation validation ensures stale data is never ensures stale data is neverserved served

• • Sessions Sessionsmaintained maintainedend-to-end end-to-endensures ensures no security reconfiguration no security reconfiguration • • Auditing, Auditing,access-control, access-control,and andquotas quotasare are fully fullypreserved preserved • • Scheduled Scheduledpreposition prepositiontotoprepopulate prepopulate DRE and edge data cache DRE and edge data cache

• • Transparent Transparentintegration integrationensures ensuresno noclient clientoror server changes to apply optimization server changes to apply optimization

WAN Files

FILE.DOC Cache

• • Disconnected Disconnectedmode modeofofoperation operation allows R/O access allows R/O accesstotofully-cached fully-cached content contentwhen whenthe theserver serverisisunreachable unreachable

• • Advanced AdvancedWAN WANoptimization optimizationlayer layer improves throughput improves throughputand andefficiency efficiency • • Data DataRedundancy RedundancyElimination Elimination(DRE) (DRE) eliminates redundant network data eliminates redundant network data • • TCP optimizations to improve TCP optimizations to improve protocol protocolability abilitytotofully fullyutilize utilizenetwork network

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

22

WAAS File Services Introduction Branch Office

IT

IT Backup

Regional Office

File Cache NAS DAS DAS Files

File Backup Cache NAS DAS DAS Files

Data Center Remote Office IT IT

Backup

WAN

NAS SAN

File Cache NAS DAS DAS

Backup

Files

Files

Decentralized Storage Centralized Centralized andStorage Optimized CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

23

Intelligent Message Suppression • IMS provides latency reduction Cache and Protocol Proxy

Eliminate unnecessary message transfer and minimize WAN RTTs

~90% msgs 10% actual storage

Batch composite commands Message prediction and pre-fetch

• File performance optimizations

IP Network

WAN Optimization DRE/TFO/LZ

Read-ahead caching during file access to increase read cache hits Asynchronous write-behind caching when safe, synchronous writebehind to ensure file integrity

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

NAS

Origin Server 100% of capacity

CISCO Public

24

Application-Specific Acceleration • Application and protocol awareness Eliminate unnecessary chatter

Application Specific Acceleration

Save WAN bandwidth

Safe Caching Read-ahead Prediction Batching

Pre-populate edge cache as necessary Enable disconnected operations

• Intelligent protocol acceleration Read-ahead, prediction, and batching

WAN

WAN Optimization DRE/TFO/LZ

Safe data and metadata caching Improves application response time Provide origin server offload

• WAASv4 application adapters CIFS (Windows File Services)

Origin Server Offloaded

Windows printing CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

25

Data Caching and Integrity • Edge file segment caching and meta data caching Data cached on-demand as files or directories are opened Prepopulation of edge cache via CDN-like preposition

• Coherency, concurrency, and ACL Cache validation guarantees no stale data served File locking and AAA handled synchronously with server

OPEN OPEN FILE.DOC FILE.DOC

IP Network

NAS

Files AAA, OPEN, LOCK AAA, OPEN, LOCK

FILE.DOC

APPROVED, LOCKED, VALIDATED APPROVED, LOCKED, VALIDATED

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

26

Integration with WAN Optimization • File services adapter leverage WAN optimization capabilities provided by DRE, TFO, and LZ DRE and LZ improves open and save operation performance through compression and data suppression TFO enables the protocol to more effectively, efficiently utilize available WAN resources

WAN FILE.DOC

DRE CACHE

DRE CACHE

LZ

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

LZ

FILE.DOC

CISCO Public

27

Intelligent File Preposition • Intelligent preposition capabilities with flexible configuration to prepopulate cache with files before the first user request • Leverage Data Redundancy Elimination (DRE) and LZ compression to improve transfer performance and user save performance IP Network

NAS

Files

Distribute Distribute FILE.DOC FILE.DOC atat3am 3am

Fetch Fetch FILE.DOC FILE.DOC

FILE.DOC

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

28

File Blocking • Cisco WAAS can be configured to prevent specific types of files from being stored on the data center file server or NAS device • Prevent non-desirable file types from consuming precious WAN resources, improve productivity Save Save SONG.MP3 SONG.MP3 IP Network

X

Files

MP3

CSBU-EBC

NAS

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

29

File Services Flexible Integration Options Non-Transparent Using Published Names, DFS Compatible Data Center NAS

\\Pluto\Demo

Branch1

Windows Client

WAN

Core WAE Name: Core1

Edge WAE Name: BR1Cache

\\BR1-Pluto\Demo

Transparent Using WCCPv2 or Policy-Based Routing Data Center

Branch1 Router

NAS

\\Pluto\Demo

CSBU-EBC

Windows Client

WAN

Core WAE Name: Core1 © 2005 Cisco Systems, Inc. All rights reserved.

\\Pluto\Demo Edge WAE CISCO Public

30

Integration Example – Software Distribution • Transparently optimize CIFS to a remote software distribution server to provide LAN-like access to hotfixes, service-packs, and other updates • Preposition to pre-populate edge cache with large packages that users will request to improve download and installation performance Data Center

Branch Office

Download Download XP-SP2.msi XP-SP2.msi from from\\pluto \\pluto

Router NAS

WAN

\\Pluto\SWUpdates

CSBU-EBC

Distribute Distribute XP-SP2.msi XP-SP2.msi

© 2005 Cisco Systems, Inc. All rights reserved.

XP-SP2

CISCO Public

31

Print Services Network Integration • Central Manager WAEs register as an Active Directory computer and provide print driver repository and driver distribution functionality • Edge WAEs register as an Active Directory computer and provide local print services to an office Data Center

Branch Office WAN

DC

PRINT

Central WAE CentralManager Manager WAE Phantom Print Server Phantom Print Server CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

PRINT

Edge WAE Edge WAE Phantom Print Phantom PrintServer Server CISCO Public

32

Cisco WAAS Print Services • Many organizations have difficulty consolidating file services because of the WAN burden that would be created due to print services traffic • Cisco WAAS provides Windows-compatible print services to eliminate the need for print jobs to traverse the WAN Data Center

Branch Office Router

NAS

WAN

Driver Driver Distribution Distribution

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

JOB JOB FILE

Print Print FILE.DOC FILE.DOC

CISCO Public

33

Print Driver Distribution • Print drivers are uploaded to the Central Manager WAE and then distributed to edge print servers or groups of devices • Printer drivers are then accessible at the edge of the network for local download (PRINT$ share) to support “Click-N-Print” functionality Data Center

Upload Upload Drivers Drivers

CSBU-EBC

Branch Office WAN

DC

PRINT

© 2005 Cisco Systems, Inc. All rights reserved.

Download Download Driver Driver and and PRINT! PRINT!

Distribute HP Distribute HP LaserJet LaserJet Driver Driver

JOB JOB PRINT FILE

CISCO Public

34

Agenda • Enterprise Application Delivery Challenges • Introducing Cisco Wide Area Application Services • Network Integration and Deployment • In-Depth Examination of Optimizations • Management and WAE Platforms • Summary • Q&A

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

35

Seamless, Transparent Integration • Integration into the network fabric with high availability, loadbalancing, and failover regardless of interception mechanism Physical inline

Src Mac AAA Dst Mac BBB

Src IP 1.1.1.10 Src TCP 15131 Dst IP 2.2.2.10 Dst TCP 80

APP DATA

WCCPv2 Policy-Based Routing CSM/ACE Modules

• Compliance with network valueadded features Preservation of packet headers Classification - QoS, NBAR, Queuing, Policing, Shaping Security - Firewall policies, Access Control Lists

Src Mac BBB Dst Mac AAA

Src IP 1.1.1.10 Src TCP 15131 Dst IP 2.2.2.10 Dst TCP 80

optimized

Reporting - NetFlow, monitoring

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

36

Cisco WAE Physical Inline Deployment • Physical inline interception Physical in-path deployment between switch and router or firewall Mechanical fail-to-wire upon hardware, software, or power failure Requires no router configuration

• Scalability and high availability Two two-port groups Serial clustering with load-sharing and fail-over Redundant network paths and asymmetric routing

• Seamless integration Transparency and automatic discovery

Cisco WAE 4-port inline card

802.1q support, configurable VLANs Supported on all WAE appliances CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

WAN WAE1 CISCO Public

37

Inline Interception Deployment Modes

In-path, single WAE, single WAN connection MGMT WAN WAE1

In-path cluster, single WAN connection MGMT WAN WAE1

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

WAE2

CISCO Public

38

Inline Interception Deployment Modes In-path, single WAE, redundant WAN links WAN MGMT

WAN WAE1

In-path cluster, redundant WAN links WAN MGMT

WAN WAE1

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

WAE2

CISCO Public

39

Cisco WAE WCCPv2 Deployment • WCCPv2 interception Out-of-path with redirection of flows to be optimized (all flows or selective via redirect-list) Automatic load-balancing, load redistribution, fail-over, and failthrough operation

Original Original Flow Flow

• Scalability and high availability Up to 32 WAEs within a service group and up to 32 routers

Interception Interception Redirection Redirection

Service Service Group Group

Linear performance and scalability increase as devices are added

• Seamless integration

Optimized Optimized Flow WAN Flow

Transparency and automatic discovery Supported on all WAE platforms CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

40

Cisco WAE PBR Deployment • Policy-Based Routing (PBR) Out-of-path with redirection of flows to be optimized (all flows or selective via access-list) WAE treated as a next-hop router

Original Original Flow Flow

• High availability Failover capability allows a secondary WAE to be used should the primary WAE fail IP SLAs ensure availability by tracking WAE liveliness

• Seamless integration

Policy Policy Route Route WAE WAE == Next Next Hop Hop

Optimized Optimized WAN Flow Flow

Transparency and automatic discovery Supported on all WAE platforms CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

41

Cisco WAE ACE Deployment • Application Control Engine (ACE) Industry-leading scalability and performance for the most demanding data center networks

WAN

Supports up to 16Gbps throughput, 4M concurrent TCP connections, and 350K connections/sec setup

• Seamless integration Fully integrated with the Catalyst 6500 series of intelligent switches Transparency and automatic discovery Supported on all WAE appliances

Optimized Optimized Flow Flow

Catalyst Catalyst 6509 6509 w/ w/ ACE ACE

Original Original Flow Flow

• Industry Leading Functionality Solution for scaling servers, appliances, and network devices Virtual partitions, flexible resource assignment, security, and control CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

42

Cisco WAAS Auto-Discovery • Cisco WAE devices automatically discover one another and negotiate optimization capabilities Performed per TCP connection Flexible optimization configuration using ATP Exchange of peer capabilities and limitations WCCPv2 WCCPv2 or or PBR PBR

A A

WCCPv2 WCCPv2 or or PBR PBR

B B

WAN

A:B A:B TCP TCP SYN SYN

B:A B:A TCP TCP SYN/ACK SYN/ACK

WAE1

CSBU-EBC

A:B TCP TCP SYN SYN/ACK B:A TCP SYN/ACK A:BB:A TCP SYN (marked) (marked)

A:B B:A SYN TCP B:A TCP SYN/ACK SYN/ACK A:B TCP TCP SYN (marked) (marked) (marked) (marked) ACCELERATION II would ACCELERATION would like like CONFIRMED! to CONFIRMED! to accelerate accelerate this this connection! connection! Here Here are are my my details details

© 2005 Cisco Systems, Inc. All rights reserved.

WAE2

II know Acknowledge WAE1 know Acknowledge WAE1 is is in the Acceleration! path, in the Acceleration! path, let’s let’s Here accelerate! are Here accelerate! are my my details details

CISCO Public

43

Non-Transparent Optimization Challenges • Complex configuration and possibility of human error

Branch Offices

Doubles network management effort Requires management of two routing topologies Requires management of duplicate feature configuration

WAN

• Compromises network features of upstream routers, switches, and firewalls Loss of visibility at L3/L4 Firewall policies, ACLs QoS, NBAR

Data Center File/Print Email Application Servers Backup

NetFlow CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

44

Network Integration Overview • With the exception of inline, Cisco WAEs attach to the LAN as an appliance • Relies on packet interception/redirection to enable application acceleration and WAN optimization Interception in each site where deployed Interception in both directions of packet flow

• Transparent optimizations maintain compatibility with most IOS features and other platforms

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

45

Use of Tertiary IFs or Sub-IFs • With non-inline modes, the WAE must not be attached to the same segment as the interface performing redirection • This is required to avoid routing loops, as we have no way to notify the router to bypass the interception and redirection (shown below) PBR or WCCPv2 IP Network

Infinite Loop

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

46

Use of Tertiary IFs or Sub-IFs (Cont.) Tertiary Interface

PBR or WCCPv2

Fa0/0

IP Network

Fa1/0 Redirect Exclude

PBR or WCCPv2

Sub-Interface

Fa0/0.10 Fa0/0.20

IP Network

Redirect Exclude

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

47

One-Arm Deployment Example 11 66

IP Network

22 33

• Pros

CSBU-EBC

44 55

• Cons

Simplicity

Performance constrained

Single interface

Higher router utilization

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

48

Two-Arm Deployment Example 11

55

• Pros

22

IP Network

44 33

• Cons

Better performance

Add’l switch port consumed

Lower router utilization

Add’l configuration Interface adjacency to node Usually feasible in branch only

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

49

Hierarchical Network Placement Considerations Locality is key • High locality to the core yields a more global level of optimization Intercept traffic going to/coming from the WAN exclusively (based on placement of interception) Closer to the WAN entry/exit prevents intrasite access from traversing the WAEs Provide optimization for all attached distribution, access layers

• High locality to the access layer yields a more focused level of optimization Optimization restricted to a specific access layer unless significant changes to network routing are introduced May cause intrasite access to traverse the WAEs which causes unnecessary WAE resource utilization

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

50

Hierarchical Network Placement Considerations • Core layer typically reserved for high performance forwarding • Distribution layer provides optimal deployment location for WAAS – close to core, aggregation for downstream access layers • Access layer can be used, but too contained to be used for large scale optimizations

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

Core

Distribution

Access

CISCO Public

51

IOS Versions for WCCPv2

•

CSBU-EBC

Per KB article #1011 http://acpluto/kb/waas/kb.asp?action=article_show&articleID=1011 © 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

52

WCCPv2 Interception Considerations • WAAS uses service groups 61 and 62 for traffic interception and redirection Service group 61 – hash bucket assignment based on source IP address of the packet Service group 62 – hash bucket assignment based on destination IP address of the packet

• One service group needs to be in the path of traffic for each direction of traffic flow Ingress interception (preferred) – analyze, intercept, and redirect as packets enter an interface – less CPU utilization Egress interception – analyze, intercept, and redirect as packets prepare to exit an interface – higher CPU utilization

• Placement of the services should not be overlooked

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

53

WCCPv2 Configuration – Routers Recommended 62/in 62/in LAN LAN and and 61/in 61/in WAN WAN keeps keeps flows flows to to aa particular particular server server pinned pinned to to the the same same WAE WAE in in both both directions of traffic flow yielding directions of traffic flow yielding better better likelihood likelihood of of compression compression per per server server

LAN

61/in 62/in

Load-balancing Load-balancing based based on on nodes nodes outside of the location outside of the location

61/in 61/in LAN LAN and and 62/in 62/in WAN WAN keeps flows from a particular keeps flows from a particular client client pinned pinned to to the the same same WAE WAE in in both both directions directions of of traffic traffic flow flow yielding yielding better likelihood of compression better likelihood of compression per per client client Load-balancing Load-balancing based based on on nodes nodes within within the the location location

WAN

LAN

WAN

62/in 61/in 62/in

•• Note: Note: most most routers routers only only support support GRE-redirect, GRE-redirect, GRE-return, GRE-return, and and hash hash assignemnt, assignemnt, which which are are default default WCCP WCCP service service configuration configuration parameters parameters CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

54

WCCPv2 Configuration – Router Isolation Branch: Branch: 62/in 62/in LAN LAN and and 61/in 61/in WAN WAN keeps keeps flows flows to to aa particular particular server server pinned to the same WAE in pinned to the same WAE in both both directions of traffic flow yielding directions of traffic flow yielding better better likelihood likelihood of of compression compression per per server server

DC: DC: 62/in 62/in WAN1 WAN1 and and 61/out 61/out WAN1 WAN1 keeps keeps flows flows to to aa particular particular server server pinned to the same WAE in pinned to the same WAE in both both directions of traffic flow yielding directions of traffic flow yielding better better likelihood likelihood of of compression compression per per server server

Load-balancing Load-balancing based based on on nodes nodes outside outside of of the the location location

No No ACLs ACLs required required to to not not redirect redirect flows to/from unoptimized flows to/from unoptimized branch branch

LAN

Load-balancing Load-balancing based based on on nodes nodes outside of the location outside of the location

WAN

WAN1

LAN

IP Network WAN2

61/in 62/in

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

61/out

CISCO Public

55

WCCPv2 Configuration – Switches Recommended 61/in 61/in LAN LAN and and 62/in 62/in WAN WAN keeps flows from a particular keeps flows from a particular server server pinned pinned to to the the same same WAE WAE in in both both directions of traffic flow yielding directions of traffic flow yielding better better likelihood likelihood of of compression compression Load-balancing Load-balancing based based on on nodes nodes within within the the location location

62/in 62/in LAN LAN and and 61/in 61/in WAN WAN keeps keeps flows flows to to aa particular particular client client pinned pinned to to the the same same WAE WAE in in both both directions of traffic flow directions of traffic flow

IP Network

62/in 61/in

Load-balancing Load-balancing based based on on nodes nodes outside outside of of the the location location

•• Note: Note: configuration configuration on on switches switches is is configured configured on on L3 L3 interfaces interfaces or or SVIs SVIs only. only. Configure Configure with with appropriate appropriate parameters parameters (L2-redirect, (L2-redirect, L2-return, L2-return, mask mask assignment) assignment)

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

56

WCCPv2 Availability Monitoring • WCCPv2 keepalive (heartbeat) information is exchanged every 10 seconds between WAEs and the router(s) • Should a WAE be unresponsive for three consecutive heartbeats, it is removed from the service group • WCCPv2 heartbeat is stateful and process-based

A

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

B

C

CISCO Public

57

WCCPv2 Failover • Should a WAE within a service group fail, the portion of the load that it was handling is automatically distributed to other WAEs within the service group • Should no additional WAEs be available, the service group is taken offline, and packets are not redirected Buckets 86–128 Buckets 1–85

Buckets 86–170

Buckets 129–170 Buckets 171–255

X A

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

B

C

CISCO Public

58

Which Interception Method to Use? WCCPv2

Inline

CSM/ACE

PBR

Number of Active WAEs

32

2

16000

1

(serial cluster, tested limit)

(not practical but possible)

Maximum Number of WAEs

32

2

16000

8

(serial cluster, tested limit)

(not practical but possible)

(IOS dependent)

Maximum Number of TCP Connections

240K

15K

4M

7.5K

Maximum Throughput

Up to 32Gbps

Up to 2Gbps

Up to 16Gbps

Up to 1Gbps

(platform dependent)

(two inline pairs)

(platform dependent)

Recommended Use

Generally Recommended

Only if WCCPv2 can not be used

Very large scale data center deployments

(with WAE-7326)

(SP managed or low-end router)

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

Last resort

CISCO Public

59

Agenda • Enterprise Application Delivery Challenges • Introducing Cisco Wide Area Application Services • Network Integration and Deployment • In-Depth Examination of Optimizations • Management and WAE Platforms • Summary • Q&A

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

60

Networks Without Compression

Congestion! Congestion!

WAN

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

61

Data Transfer Without Compression

WAN

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

62

Networks With Compression No NoCongestion Congestion or Less or LessCongestion Congestion

WAN

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

63

Data Transfer With Compression

WAN

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

64

Cisco WAAS Advanced Compression • Data Redundancy Elimination (DRE): application-agnostic compression eliminates redundant data from TCP streams providing up to 100:1 compression • Persistent LZ Compression: session-based compression provides up to an additional 10:1 compression even for messages that have been optimized by DRE

LZ

LZ

DRE

DRE

Synchronized DRE Context

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

65

Advanced Compression Block Diagram Signature Matching Add New Entries

DRE FIFO Cache Synchronization

DRE FIFO Cache Synchronization

Fingerprint Chunk Identification

Fingerprint Chunk Identification LZ

TCP Proxy

CSBU-EBC

Signature Matching Add New Entries

© 2005 Cisco Systems, Inc. All rights reserved.

LZ

TCP Proxy

CISCO Public

66

DRE Chunk Identification

Fp mod No boundary found

Fp mod No boundary found

Fp mod No boundary found

Fp mod No boundary found

Fp mod Boundary identified!

• Analyze incoming data streams using a sliding window to identify “chunks” • Each chunk assigned a 5-byte signature • Single-pass used to identify chunks at multiple levels Basic chunks Chunk aggregation (nesting)

• After chunks are identified, DRE will begin pattern matching Chunk1

Fp mod

Look for smaller chunks if necessary

5B Sig

CSBU-EBC

First look for largest chunks

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

67

DRE Chunk Identification

Level-0 Level-0Chunk Chunk “Basic “BasicChunk” Chunk” ~256 ~256bytes bytes

Level-1 Level-1Chunk Chunk ~1024 ~1024bytes bytes

Level-2 Level-2Chunk Chunk ~4096 ~4096bytes bytes

Level-3 Level-3Chunk Chunk ~16384 ~16384bytes bytes

Original Data

• Each chunk is assigned a 5-byte signature

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

68

DRE Pattern Matching

DRE Database

NO MATCH NO MATCH NO MATCH NO MATCH

Original Original Message Message

CSBU-EBC

Encoded Encoded Message Message

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

69

TFO Transport Flow Optimization

WAAS4.0

© 2005 Cisco Systems, Inc. All rights reserved.

Cisco Public

70

70

Terminology – Maximum Window Size (MWS)

4

3

2

1

ACK

• The Maximum Window Size (MWS) is the maximum amount of a data a node can have outstanding in the network unacknowledged • The node can not continue transmission until previous transmissions have been acknowledged Problematic over LFNs – Long Fat Networks “elephants” Inability to fully utilize the available network resources

• Generally, ACKnowledgements are sent when an entire TCP window has been received • Upon encountering packet loss, the node would be required to retransmit the entire window of data (guaranteed delivery) Problematic over low-speed links Problematic with large windows

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

71

Terminology – Bandwidth Delay Product (BDP) RTT 10 ms Amount of data that can be in transit at any one point in time: 155Mbps x 10 ms = 192 KB

Bandwidth 155 Mbps (OC-3)

• The Bandwidth Delay Product (BDP) of a network defines the amount of data that can be in flight within a network at any one point in time

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

72

Challenge • Common TCP implementations on client and server operating systems can be bottlenecks to application performance Inability to fill-the-pipe, i.e. utilize available bandwidth Inefficient recovery from packet loss, retransmission Bandwidth starvation for short-lived connections

• Cisco WAAS Transport Flow Optimization (TFO) utilizes industry-standard TCP optimizations to remove these application performance barriers

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

73

Cisco WAAS Transport Flow Optimizations • Cisco WAAS Transport Flow Optimizations (TFO) is designed to overcome common challenges associated with standard TCP implementations Window Scaling – capitalize on available bandwidth Large Initial Windows – maximize transmission after connection establishment Selective Acknowledgement – efficient packet loss recovery and retransmission mechanisms Binary Increase Congestion (BIC) – quick return to maximum throughput upon encountering congestion

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

74

TCP Maximum Window Size (MWS) • MWS (maximum window size) determines the maximum amount of data that can be in transit and unacknowledged at any given time • BDP (bandwidth delay product) defines the amount of data that can be contained within a network at any given time If MWS > BDP, then application may not be throughput bound (i.e. application can “fill the pipe”) If BDP > MWS, then application will not be able to fully utilize the network capacity (i.e. application can not “fill the pipe”)

• Does not account for application-layer (L7) latency such as found with protocol-specific messaging CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

75

Link Utilization and MWS, BDP BDP BDP

Bandwidth

Unusable Unusable network network capacity! capacity!

MWS MWS Link Link Utilization Utilization

Latency

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

76

WAAS TFO Window Scaling • Cisco WAAS TFO window scaling (based on RFC 1323) scales the TCP window to 2MB to overcome problems with filling LFNs (Long Fat Networks) • Window Scaling applies a binary shift to the decimal value supplied in the data field

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

77

Link Utilization After Window Scaling BDP BDP Cisco Cisco WAAS WAAS TFO TFO

Bandwidth

Able Able to to fill fill the the pipe! pipe!

Original Original MWS MWS

Latency

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

78

Cisco WAAS Large Initial Windows • While 80% of network traffic is typically associated with long-lived connections (elephants), approximately 80% of network connections are short-lived (mice) • Short-lived connections transmit smaller numbers of packets and are torn down before ever leaving the slow-start phase of TCP • Cisco WAAS Large Initial Windows, based on RFC3390, increases initial window size to expedite entry into congestion avoidance mode for high throughput

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

79

Cisco WAAS Large Initial Windows

Segments per Round Trip (cwnd)

Packet Packet Loss Loss

TFO TFO

Slow-Start Slow-Start (discovery) (discovery)

Congestion Congestion Avoidance Avoidance (high-throughput) (high-throughput)

TCP TCP

Round Trips

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

80

Standard TCP Acknowledgement • Standard TCP implementations acknowledge receipt of data by acknowledging the entire window has been received • Loss of a packet causes retransmission of the entire TCP window, causing performance degradation as the window becomes larger Transmit

Receive

1

1

2

3

2 ACK

Retransmit 1

2

3 ACK

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

1

2

3

1

2

3 CISCO Public

81

Cisco WAAS Selective Acknowledgement • Cisco WAAS employs TCP extensions to improve acknowledgement of transmitted data, improve delivery of missing segments, and unnecessary minimize retransmission • Based on RFC 2018 and extensions

Transmit

Receive

1

1

2

1

2

1

2

3

1

2

3

2

3 ACK

Retransmit 3 ACK

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

82

Without TCP Proxy

WAN

X TIMEOUT! RESEND

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

?

CISCO Public

83

TCP Proxy and TFO

WAN

Window Scaling Large Initial Windows Congestion Mgmt Improved Retransmit

X

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

84

Standard TCP Congestion Avoidance • Standard TCP implementations employ an exponential slow start to increase throughput to the slow start threshold • From the slow start threshold, the congestion window is increased linearly by one packet per round-trip until packet loss is encountered • Upon encountering packet loss, the congestion window is cut in half to return to a throughput level safe given the congested environment • The net result is “saw-tooth” throughput, and return to maximum throughput can take hours for long-lived connections and LFNs

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

85

Standard TCP Congestion Avoidance

Segments per Round Trip (Congestion Window)

Packet loss causes connection to enter into linear congestion avoidance (+1 cwnd per ACK) cwnd dropped by 50% on packet loss

loss

Linear Congestion Avoidance (+1 cwnd per ACK)

loss

Exponential Slow Start (2x pkts per RTT) Low throughput during this period 1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

Round Trips CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

86

“Saw-tooth” TCP Throughput Return to maximum throughput could take a very long time!

Packet loss

Packet loss

Packet loss

Packet loss

TCP

cwnd

Slow start Congestion avoidance

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

Time (RTT)

CISCO Public

87

Binary Increase Congestion (BIC) • Cisco WAAS employs the Binary Increase Congestion (BIC) congestion avoidance system to improve throughput in lossy environments • Uses a binary search to adaptively increase the congestion window, resulting in a stable and timely return to higher levels of throughput • Decreases congestion window only by 1/8 (rather than 1/2 as compared to TCP) when packet loss is encountered, mitigating the majority of the performance penalty

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

88

WAAS Throughput and Congestion Avoidance Adaptive AdaptiveIncrease Increasetotocwnd cwnd cwnd = cwnd + f(cwnd, history) cwnd = cwnd + f(cwnd, history)

Packet Packetloss loss

Packet Packetloss loss

Cwnd Cwnddecreased decreasedby by1/8 1/8on on packet loss vs 1/2 with TCP packet loss vs 1/2 with TCP

Packet Packetloss loss

Packet Packetloss loss

cwnd

Cisco Cisco WAAS WAASTFO TFO

Slow start Congestion avoidance

Time (RTT) Standard Standard TCP TCP

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

89

Comparing TCP and TFO

Cisco CiscoTFO TFOprovides providessignificant significantthroughput throughput improvements improvementsover overstandard standardTCP TCPimplementations implementations

TFO TFO

cwnd

TCP TCP

Slow start Congestion avoidance

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

Time (RTT)

CISCO Public

90

TCP Throughput and Latency Optimizations • TCP window scaling improves link utilization and throughput

• Compatible and friendly to other TCP connections on the network

• Optimized TCP stack improves recovery and congestion handling

• Large initial windows improves throughput for short-lived connections

• Priority for transactional traffic Link LinkUtilization Utilization

Packets Packetsper perRound-Trip Round-Trip Bandwidth scalability

Standard Standard TCP TCP

Bandwidth CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

RTT Fairness Standard Standard TCP TCP

Packets/RTT

Link Utilization

Cisco Cisco WAAS WAASTFO TFO

Cisco Cisco WAAS WAASTFO TFO

TCPFriendliness

Packet Loss Probability CISCO Public

91

Agenda • Enterprise Application Delivery Challenges • Introducing Cisco Wide Area Application Services • Network Integration and Deployment • In-Depth Examination of Optimizations • Management and WAE Platforms • Summary • Q&A

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

92

WAAS Intuitive Central Management • Comprehensive Management Central configuration Device grouping Monitoring, statistics Alerts, reporting

• Easy-to-use Interface Graphical U/I, Wizards IOS CLI Roles-based administration

• Proven Scalability 1000’s of nodes Redundancy and recovery CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

93

Cisco WAE Family Positioning Enterprise Data Center

Performance

ACE

WAE-7326 Regional Office or Small Data Center

WAE-612

Branch or Remote Office WAE-512

NME-WAE-502 NME-WAE-302

Scalability CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

94

Remote Office Hardware Platforms • NM-WAE Module Lowest CapEx and OpEx, integrated within the ISR, addresses 80% of remote branch offices Single processor system, can be clustered with WCCPv2, PBR NM-WAE

Supported in ISR models 2811, 2821, 2851, 3825, and 3845

Router-Integrated Network Module for the Cisco Integrated Services Router

• WAE-512 Appliance Remote office appliance platform Up to 20Mbps WAN connections 1500 optimized TCP connections WAE-512 Remote Office Appliance

250GB RAID-1 disk capacity Deploy w/ inline, WCCPv2, PBR, CSM/ACE

Performance Performance and and scalability scalability are are subjective subjective and and may may vary vary based based on on aa variety variety of of conditions. conditions. WAE WAE WAN WAN bandwidth bandwidth is is not not limited limited by by hardware hardware or or software software CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

95

Data Center Hardware Platforms • WAE-612 Appliance Regional hub and medium data center deployments Up to 155Mbps WAN connections WAE-612 Regional Hub and Data Center Appliance

6000 optimized TCP connections 300GB RAID-1 SAS disk capacity Deploy w/ inline, WCCPv2, PBR, CSM/ACE

• WAE-7326 Appliance Enterprise data center deployments Up to 310Mbps WAN connections 7500 optimized TCP connections WAE-7326 Enterprise Data Center Appliance

900GB RAID-1 SCSI disk capacity Deploy w/ inline, WCCPv2, PBR, CSM/ACE

Performance Performance and and scalability scalability are are subjective subjective and and may may vary vary based based on on aa variety variety of of conditions. conditions. WAE WAE WAN WAN bandwidth bandwidth is is not not limited limited by by hardware hardware or or software software CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

96

Increased Scalability • Sizing tool will do the job

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

97

Agenda • Enterprise Application Delivery Challenges • Introducing Cisco Wide Area Application Services • Network Integration and Deployment • In-Depth Examination of Optimizations • Management and WAE Platforms • Summary • Q&A

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

98

Summary • Not all application protocols can be optimized through generic WAN optimization – some require application-specific acceleration to function properly over a WAN • Cisco WAAS provides robust application-specific and network-layer optimizations to enable application delivery and file server consolidation • Cisco WAAS file services provides integration flexibility and can help enable consolidation of additional CIFS or NFSbased platforms such as software distribution servers • Cisco WAAS also provides Windows-compatible and centrally-managed print services with driver distribution and disconnected printing capabilities

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

99

Why Choose Cisco WAAS? • Application-specific optimization for file and print services helps to enable file server and data protection consolidation while enabling offline access to fully cached files • High performance WAN optimization to reduce bandwidth consumption and maximize throughput, efficiency to significantly improve application delivery over the WAN • Network transparency preserves investment in existing network feature configurations and physical integration provides industry’s best total cost of ownership model • Robust and proven secure central management platform scales to meet the needs of the largest organizations • Integration with industry-leading application networking technologies such as the ACE module for data center integration, scalability, and performance • Cisco’s world-class 24x7x365 technical assistance center

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

100

Agenda • Enterprise Application Delivery Challenges • Introducing Cisco Wide Area Application Services • Network Integration and Deployment • In-Depth Examination of Optimizations • Management and WAE Platforms • Summary • Q&A

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

101

Questions?

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

102

102

Time for break!

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

103

Presentation_ID CSBU-EBC

© Cisco Systems, Systems, Inc. Inc. All All rights rights reserved. reserved. © 2003, 2005 Cisco

CISCO Public

104

104