Master_ucsi_bootcamp_vaptos_3110.pdf

Lesson 1

Introduction to Cisco’s UCS Platform Overview In this lesson you will learn about Cisco’s Unified Computing System Platform. You will understand new trends in the datacenter and understand how Cisco has designed the UCS hardware and software to be at the forefront of these trends. Objectives The specific objectives of this lesson are to enable you to perform the following tasks: • Introduce Data Server Trends • Introduce the UCS system hardware, and address how it is addressing these trends • Introduce the UCS management hardware and software

Introduction to Unified Computing System Platform

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training – Overview

In this lesson you will learn about Cisco’s Unified Computing System Platform. You will understand new trends in the datacenter and understand how Cisco has designed the UCS hardware and software to be at the forefront of these trends. Upon completing this lesson you will be able to:

• Introduce Data Server Trends • Introduce the UCS system hardware, and address how it is addressing these trends • Introduce the UCS management hardware and software

1-1 UCSI Boot Camp

© Cisco Systems, Inc.

Scalability: The server evolution Scale Up Monolithic servers Large number of CPUs Proprietary platform Proprietary OS Many apps per server

The 90’s High cost Large failure domain

Scale Out Commoditized servers Few CPUs X86 platform Commoditized OS 1 App / server

Early 2000’s Servers under-utilized Power & cooling

Scale In Bladed servers Multi-core CPUs X86 platform Commoditized OS 1 App / virtual machine

Now Management complexity Limited scale

?

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training – Overview

Scalability: The server evolution The first servers were designed as large rack mounted systems that could at times be quite large in size (thus being termed “Big Iron”). At this time the components were expensive so a single large system that could service large numbers of users was the most cost effective means to provide services to users. The next change in servers came with commoditized servers. These were smaller servers that cost far less than the monolithic server, but they also had far fewer processing resources thus limiting them to a single service or application per physical server. Today the push for servers is to do more with less. Servers are being designed into smaller form factors utilizing shared resources. Additionally the advent of virtualization software, many servers are being consolidated onto single physical computing resource. While this aids in many of the challenges faced by the datacenter this does not solve all challenges

© Cisco Systems, Inc.

Introduction to Cisco’s UCS Platform 1-2

Scalability: Changes in server design Redundant power and networking

power

LAN/SAN/ serial

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training – Overview

Scalability: Changes in server design One of the major changes was the move from single self contained servers to multiple smaller bladed servers in a single enclosure. Larger servers require more space and also had to have its own power supplies, serial console connectivity, networking, and SAN cables. Individually this may not seem like a problem but if we have to provide identical infrastructure for 100 of these servers, the result is a complex and messy infrastructure to try and manage and maintain. By putting the compute resources of a server in a bladed form factor within a enclosure we gain the following advantages:

• • • •

Maximum use of physical space Shared power distribution Shared networking and storage access More efficient cooling

1-3 UCSI Boot Camp

© Cisco Systems, Inc.

Converged Network Fabrics With the increase in servers has come the massive increase in the amount of cables in the data center. There have been a number of companies who have found over the course of years that their raised floors are completely filled from bottom to top with cabling. This is exacerbated because network caballing is not limited to Ethernet, but also includes SAN as well. Depending on the server, it may have dual Ethernet connections and dual SAN connection for protection. That is 4 network cables per server not to mention other cable connections like power and console cables. A standard rack could contain as many as 45 servers, which means as many as 180 network cables. Recently Cisco has introduced into the market products that combine Ethernet and Fibre Channel over the same cables. The Nexus switching line uses 10 gigabit Ethernet to encapsulate Fibre Channel frames within Ethernet frames which results in the ability to reduce your cabling needs in half. The resulting savings helps with space, power, and cooling in the data center.

© Cisco Systems, Inc.

Introduction to Cisco’s UCS Platform 1-4

CPU and Memory Density single core CPU single socket, single core

core

core core

core

core core

single socket, dual core

single socket, quad core

addressable memory

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training – Overview

CPU and Memory Density Today server processor are multi-core. Offering multiple processing cores in the same space as previously used for single-core processors. In addition to this many vendors have also created memory controllers that have the ability to address large amounts of RAM. While the blade sizes have remained the same, the processing and memory has increased significantly allowing blades to run processing and memory intensive application. Terminology:

• Socket refers officially to the physical chip receptacle on the mother board. It is used informally to refer to the physical CPU module or chop itself (the unit of ordering, or the FRU).

• Core is still a physical processing element embedded in the module (socket). An OS running on a server will always see the number of processors or CPU’s as equal to the number of cores or more.

1-5 UCSI Boot Camp

© Cisco Systems, Inc.

The reason an OS could see more processors than existing cores is hyperthreading. CPU’s have the ability to have each core emulate two or more processors. These emulated processors are called threads indistinguishable from physical processors to the CPU. On the Nehalem architecture on the Cisco B-series blades, a “hyperthreading” setting is enabled by default, and it doubles the number of processors visible to an OS. Therefore, by default, an OS running in the blades will see 8 processors (threads) per socket. The word CPU, all by itself, is almost always ambiguous. Do you mean “physical module” (socket), or physical processor (core), or processor visible to the OS (core or thread). You just have to agree what you are talking about when you use the word CPU. An OS will always report the number or processors at the “lowest granularity”, ie “processors visible to the OS”. So this will always be equivalent to cores or threads. Software licensing is sometimes “per socket” (for example, VMware ESX, and Cisco Nexus1000v). Sometimes it is “per core” (for example, Oracle DB) There is nothing that prevents it from being “per thread”, but this author can’t think of any examples.

© Cisco Systems, Inc.

Introduction to Cisco’s UCS Platform 1-6

Statelessness

WWN MAC BIOS firmware

© 2009 Cisco Systems Inc. All rights reserved.

WWN MAC BIOS firmware

UCS Technical Training – Overview

Statelessness State in a server includes the behavior and identity elements that make that server unique compared to all others. These items include:

• • • • • • • •

Identity WWN MAC UUID Behavior Boot order Firmware versions QoS

A failed server moving its operating system and application to another server would require that the new server personality changes be manually changed by network admin, SAN admin, and server admin in order for the software to be able to run correctly.

1-7 UCSI Boot Camp

© Cisco Systems, Inc.

In this second example, if the servers have the ability to be stateless personality settings of the first server can be applied to the second server automatically at failover, thus allowing the operating system and application to start without administrative intervention.

© Cisco Systems, Inc.

Introduction to Cisco’s UCS Platform 1-8

Virtualization virtual machines

hypervisor

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training – Overview

Virtualization This technology has allowed companies to more easily consolidate servers in the data center. With the commoditized server model, one physical server was needed for an application. With virtualization many servers, (each running in independent virtual machines), can run on a single physical server. This is possible through the creation (in software) of virtual hardware that can function like a server made of physical hardware. There are a number of advantages to using virtualization including better use of computing resources, higher server densities, and seamless server migrations.

1-9 UCSI Boot Camp

© Cisco Systems, Inc.

Centralized Management Server Orchestrators / Manager of Manager

Chassis Mgr

Chassis Mgr

Chassis Mgr

Network Mgr

Network Mgr

Network Mgr

Server Mgr

Server Mgr

Server Mgr

Vendor A

Vendor B

Vendor C

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training – Overview

Centralized Management As server sprawl has increased the challenges of managing a diverse environment has become more difficult. To solve this both server vendors and third party companies have developed software to help with many of the aspects of the systems management. This does however present new challenges as it does create a myriad of software layers and integrations in order to get it to work properly. Providing an easy integration of vendor server management software and infrastructure management software is essential.

© Cisco Systems, Inc.

Introduction to Cisco’s UCS Platform 1-10

UCS Hardware Components

© 2009 Cisco Systems, Inc. All rights reserved.

UCS Technical Training – Overview

In this section, we will introduce each of the hardware components that comprise a UCS system. Specifically, the learning objectives are

• • • • • • • • •

Describe logical and physical perspectives of UCS Describe the blades Describe the mezzanine cards Describe the hard disk drives Describe the fabric interconnects Describe the expansion modules Describe the chassis Describe the fabric extender Describe the backplane

1-11 UCSI Boot Camp

© Cisco Systems, Inc.

Logical View of UCS SAN

© 2009 Cisco Systems Inc. All rights reserved.

LAN

UCS Technical Training – Overview

Logical View of UCS The graphic depicts a fully-populated UCS system, which includes the following:

• • • •

2 fabric interconnects 1-40 chassis 2 fabric extenders (aka IOM) per chassis 1-8 blades per chassis

Note that the fabric interconnects have uplink connectivity to both the LAN and SAN networks. Also note that support for up to 40 chassis requires use of the 40-port 6140. In an HA configuration using 2 interconnects, both switches may be configured to actively switch traffic simultaneously, thereby improving throughput.

© Cisco Systems, Inc.

Introduction to Cisco’s UCS Platform 1-12

Front View of UCS redundant fabric interconnect

2 x power supplies 2 x fan modules

4 x half-width blades

chassis 2 x fullwidth blades

4 x power supplies © 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training – Overview

Front View of UCS Note that the power supplies for both the switch and chassis are accessible via the front. Also note that a combination of half-width and full-width blades is supported in the same chassis. The fan modules for the switch are accessible in the front, while fan modules for the chassis are accessed from the rear. The diagram shows the most typical topology, with a redundant fabric interconnect. The same two fabric interconnects connect down to multiple chassis to form a UCS system. At time of writing the blade options are:

• Half-width blade: B200 • Full-width blade: B250 As shown in the diagram, you are free to mix and match half-width and full-width blades in the same chassis.

1-13 UCSI Boot Camp

© Cisco Systems, Inc.

Chassis The internal name for the chassis is Santa Clara. The chassis is 32” deep and 6RU tall. The blades and power supplies plug in from the front. The same chassis may be used for up to 8 half-width blades, 4 full-width blades, or some combination of half- and fullwidth blades. For the half-width blades, a vertical divider is used to separate horizontally adjacent blades. For the full-width blades, this divider must be removed. Power There are a maximum of 4 x 2500W hot-pluggable power supplies per chassis. The actual number of power supplies required depends on the system’s hardware configuration and desired power redundancy. They can be configured as non-redundant, N+1, or N+N (grid) redundant. They are single-phase 220V, IEC320-C 19. The power supplies provide a 550W budget per slot for up to 8 half-width blades and 1100W budget per slot for up to 4 full-width blades. While this may seem like “overkill”, the design supports future growth and power budget requirements with no service disruption. Cooling The chassis cools from front to rear with a high-efficiency, high-reliability flow-through airflow design (63% of a fully populated chassis is air). The fan modules are sized to support all blades running at full power budget. There are 8 hot-plug dual fan modules per chassis with status indicators on each module. UCS feature support: UCS features supported by the chassis include the ability to scale up to as many as 20 chassis (using the 20-port interconnect) or 40 chassis (using the 40-port interconnect).

© Cisco Systems, Inc.

Introduction to Cisco’s UCS Platform 1-14

Rear View of UCS expansion module bay 1 or 2

20/40 x fabric/border ports 2 x cluster ports

2 x power entry console port 1 x management port 8 x fan modules

2x fabric extenders

4 x 10GE SFP+ fabric ports

4 x power entry © 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training – Overview

Rear View of UCS All the cabling for a UCS for both the interconnects and the chassis (including power, Ethernet, console, and Fiber Channel) are plugged into the rear. In such a redudant setup as shown , the “left” FEX connects to the “A” switch and the “right” FEX connects to the “B” switch. Note that the interconnect shown in the slide is the 20-port 6120.

1-15 UCSI Boot Camp

© Cisco Systems, Inc.

Blade Overview Differences Type

Form Factor

Number of DIMM Slots

Number of Mezzanine Slots

Half-width

½ chassis width

12

1

Full-width

full chassis width

48

2

Similarities Type

On Board Management

CPU Sockets

Internal Drives

Half-width

BMC

2

2

Full-width

BMC

2

2

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training – Overview

Blade Overview There are two types of blades available – half-width and full-width blades. Three factors distinguish these blades: the form factor, the number of DIMM slots, and the number of mezzanine card slots. The blades have a common (x64 Nehalem) architecture and share many features such as being hot-swappable in the chassis, having 2 CPU sockets, 2 internal SAS disk drive bays, and a baseboard management controller (BMC). Only one CPU is required for normal system operation. If only one CPU is installed, then it must go in the first socket. The CPUs must be identical on the same blade but can be mixed between blades in the same chassis. The baseboard management controller (BMC) is a microcontroller on the motherboard that provides “lights out” hardware status and configurability to the system management software over dedicated Ethernet links between the BMC and each FEX. The BMC uses the IPMI (Intelligent Platform Management Interface) protocol over the I2C (inter-integrated circuit) serial bus to manage devices on the baseboard. The BMC is also responsible for providing remote KVM access to the end user through dedicated management IP addresses for each blade exposed on the fabric-interconnect management port and NAT’-ed into the BMC.

© Cisco Systems, Inc.

Introduction to Cisco’s UCS Platform 1-16

UCS feature support: Choice of blades provides increased scalability to the consumer with regard to:

• • • •

Number of CPUs to populate (1-2) Number of disk drives to populate (1-2) Number of mezzanine adapters to populate (1-2) Amount of memory to install (depending on blade, up to 384GB)

Another UCS feature supported for the blades is statelessness, where the following blade personality elements are relocatable between blades:

• Identity elements such as MACs, WWNs, and UUIDs • Behavior elements such as BIOS version and QoS settings At the time of writing the B250 supports two mezzanine slots, althought the two cards must be identical.

1-17 UCSI Boot Camp

© Cisco Systems, Inc.

Half-Width Blade Mezzanine Card

Memory Slots

Drive Bays

CPU Sockets

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training – Overview

Half-Width Blade The half-width blade (referred to internally as the Gooding blade) contains 2 SFF drive bays that support SAS/SAT disks in a hardware RAID-0 or RAID-1 configuration. It has a 550W power and cooling budget. The chassis can hold up to 8 half-width blades. The motherboard contains two CPU sockets and 12 DDR3 DIMM memory sockets. There is a single mezzanine adapter slot.

© Cisco Systems, Inc.

Introduction to Cisco’s UCS Platform 1-18

10.6GB/s

10.6GB/s

Balanced Performance

Maximum B/W

Memory Configurations: (B200)

CPU

CPU

8.5 GB/s

8.5 GB/s

CPU

© 2009 Cisco Systems Inc. All rights reserved.

CPU

E5550 and above

E5520 and above UCS Technical Training – Overview

Memory Configurations: B200 Maximum B/W: 1333 speeds require a 1333-capable CPU, as shown, AND will only be achieved with 1 DPC (Dimm Per Channel) --- ie you could a speed vs capacity tradeoff with 1333 (max 48GB RAM on whole blade server actually running at 1333)

• DDR3 1333MHz across 3 channels • 1 DPC (6 DIMMs) • Max Capacity – 48 GB

1-19 UCSI Boot Camp

© Cisco Systems, Inc.

Balanced Performance: If both slots on a channel or populated (“2 DPC”) we can run at a max speed of 1066 (the 1066 is really MTS – mega-transfers-per second)

• DDR3 1066 MHz across 3 channels • Up to 2 DPC (12 DIMMS) • Max Capacity: 96 GB You can mix and match SIZES but could the best performance, in descending order, with:

• • •

same size across whole server same size across whole CPU same size across a bank

© Cisco Systems, Inc.

Introduction to Cisco’s UCS Platform 1-20

Full-Width Blade Mezzanine Cards Memory Slots

Drive Bays

CPU Sockets

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training – Overview

Full Width Blade The full-width blade (referred to internally as the Ventura blade) has 2 SFF drive bays. It has a 1100W power and cooling budget. The chassis can hold up to 4 full-width blades. The motherboard contains two CPU sockets and 48 DDR3 DIMM memory sockets, effectively quadrupling the memory capacity of the Gooding blade. There are 2 mezzanine adapter slots, effectively doubling the I/O bandwidth and providing redundancy.

1-21 UCSI Boot Camp

© Cisco Systems, Inc.

CPU

CPU

Memory ASIC

8.5GB/s

Memory ASIC

Memory Configurations: Full-width Blade (B250)

8.5GB/s

• Memory ASIC controls 4 physical DIMM • Presents to CPU as 1 DIMM (up to 32GB)

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training – Overview

Memory Configurations: Full Width The unique memory ASIC on B250 is able to present 4 physical DIMM slots to the CPU as one “virtual DIMM”, up to 32GB. The CPU cannot tell the difference between the B200 and B250 architectures. This “virtualization” of a memory slot is fully supported by Intel.

© Cisco Systems, Inc.

Introduction to Cisco’s UCS Platform 1-22

Adapter Offerings Virtualization VM I/O Virtualization and Consolidation

10GbE/FCoE

Eth

FC

Eth

Compatibility Existing Driver Stacks

Cost Cost Effective 10GbE LAN access

10GbE/FCoE

Eth FC

vNICs 0

1

2

3

PCIe x16

© 2009 Cisco Systems Inc. All rights reserved.

127

10GbE

FC

For hosts that need LAN access

PCIe Bus

UCS Technical Training – Overview

Adapter Offerings The 3 types of adapters (aka mezzanine cards) available in a UCS blade include Palo, Menlo, and Oplin.

Palo The Palo adapter supports up to 128 Ethernet vNICs (or FC vHBAs) running at 500 KIOPs in both initiator and target mode. The Palo adapter is a converged network adapter (CNA) with dual 10GE ports and dual FC ports to the backplane. The Palo ASIC provides failover between the redundant Ethernet links. No multipathing software is required on the host OS for this functionality.

Menlo The Menlo adapter is a converged network adapter (CNA) with 2 host-side 10GE ports and 2 FC ports to the backplane. The 2 network ports can run either native Ethernet or FCoE protocols and can be configured for failover. This failover is performed by the Menlo ASIC and does not require multipathing software on the host OS. The Menlo ASIC is a Cisco-designed multiplexor and FCoE protocol offload engine with a 350MHz 24K MIPS processor. There are 2 versions of this card: Menlo-E (with an Emulex

1-23 UCSI Boot Camp

© Cisco Systems, Inc.

chipset) and Menlo-Q (with a Qlogic chipset), thereby supporting existing proven driver stacks to the customer.

Oplin The Oplin card is made by Intel. It has 2 10GE ports to the backplane which run native Ethernet. OS-implemented FCoE may be run over the Oplin to provide “free SAN access” to the host. The Oplin card does not provide failover functionality.

UCS feature support:

• Choice of the Palo adapter provides enhanced virtualization. The Palo adapter supports up to 128 vNICs presented to the ESX server which can then be used as VMNICs for virtual machines. Further, the personality (through port profiles which define MAC forging, VLANs, and QoS settings) of a VM can migrate (i.e. Vmotion) using the VNTag functionality of the Palo adapter and NX5K.

• Choice of the Menlo adapter provides increased scalability. The FC-to-Ethernet protocol encapsulation and decapsulation is offloaded from the host and performed on the Menlo card itself.

• Choice of any of these mezzanine cards simplifies management because of reduced cabling/switching infrastructure and less firmware to upgrade

© Cisco Systems, Inc.

Introduction to Cisco’s UCS Platform 1-24

Hard Disk Drives

• 2 SFF drive bays plug into SAS backplane • 73GB 15KRPM • 146 or 300GB 10KRPM • RAID-0 or RAID-1 • Diskless configuration supported (requires SAN or LAN[PXE] boot)

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training – Overview

Hard Disk Drives Both types of blades have two drive bays and an SAS backplane, regardless of whether the blade was sold as “diskless”. The supported drives – a 73GB 15KRPM SAS SFF drive , a146GB 10KRPM SAS SFF drive and a 300GB 10KRPM SAS dirive . No local hard drive is required. If two hard drives are installed, then they must both be of the same size and speed and are hot-swappable. The build-in LSI RAID controller supports RAID-0 & RAID-1.

UCS feature support:

• UCS features supported at the disk drive include statelessness by scrubbing any data on a disk drive during a service profile migration.

• Also, local disk policies (such as RAID-0 or RAID-1) may be configured that migrate with logical service profiles between blades.

1-25 UCSI Boot Camp

© Cisco Systems, Inc.

20 and 40 - Port Fabric Interconnect

1U

2U

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training – Overview

20 and 40 Port Fabric Interconnect (UCS 6100) 20 – Port Fabric Interconnect The internal name of the fabric interconnect is Springfield. The 20-port interconnect is 1U high. It has 20 SFP+ ports for 10GE/FCoE . There is 1 expansion module for connecting to the LAN and SAN networks. The 20-port interconnect supports up to 560 Gbps fabric. None of these ports are enabled or configured by default.

40 – Port Fabric Interconnect The 40-port interconnect is 2U high. It has 40 SFP+ ports for 10GE/FCoE. There are 2 expansion modules for connecting to the LAN and SAN networks. The 40-port interconnect supports up to 1.12 Tbps fabric.

© Cisco Systems, Inc.

Introduction to Cisco’s UCS Platform 1-26

UCS feature support:

• Choice of either the 20-port or 40-port version of the interconnect provides scalability to the customer

• UCS features supported by the fabric interconnects include scalability. You have the choice of using a 20-port or (shown in the next slide) a 40-port interconnect.

• Another UCS feature supported by the interconnect is enhanced phyiscal (on the fex) and logical port virtualization as implemented in the VNTag feature.

• The interconnect also natively supports FCoE to converge Fiber Channel and Ethernet fabrics.

• Finally, the management server (discussed later in this module) runs on the interconnect and simplifies management so that all UCS components may be managed through a single pane of glass.

1-27 UCSI Boot Camp

© Cisco Systems, Inc.

Expansion Modules

FC-only (8 ports @ 1/2/4 Gbsec)

FC-only (6 ports @ 1/2/4/8 Gbsec) Combo FC + Ethernet Ethernet-only © 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training – Overview

Expansion Modules The expansion modules (also referred to as GEMs) provide connectivity into your enterprise Ethernet LAN and Fiber Channel SAN networks. There are three expansion modules from which to choose: FC-only, combination FC and Ethernet, and Ethernetonly.

• FC-Only (1/2/4Gb) - This fibre channel expansion module contains 8 SFP ports that run 1, 2, and 4 Gbps FC.

• FC-Only (1/2/4/8Gb) - This fibre channel expansion module contains 6 SFP ports that run 1, 2, 4 and 8 Gbps FC. • Combo - This combination expansion module contains 4 SFP+ ports that run 10GE and 4 SFP ports that run 1, 2, and 4 Gbps FC. It is internally referred to as a diamond card. • Ethernet-Only - This Ethernet expansion module contains 6 SFP+ ports that run 10GE. It is internally referred to as an emerald card.

© Cisco Systems, Inc.

Introduction to Cisco’s UCS Platform 1-28

UCS feature support: The expansion modules support increased scalability by providing choices to the consumer for LAN and/or SAN expansion.

1-29 UCSI Boot Camp

© Cisco Systems, Inc.

I/O Module (Fabric Extender)

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training – Overview

I/O Module FEX The fabric extender (also referred to as an IO module or IOM) logically extends the fabric from the switch to the blade. The name of the ASIC in the FEX is internally referred to as Redwood. Its primary function is to connect the blades to the left or right interconnect. Secondarily, it houses a chassis management controller (CMC). There are 4 10Gbps ports that can be used to connect the chassis to the switch. You connect either 1, 2, or 4 links from the FEX to the fabric interconnect. The FEX support from 2:1 to 8:1 oversubscription rates. The chassis contains 2 FEX slots for increased redundancy and bandwidth, though only one FEX is required for minimal configuration. If only one FEX is installed, it must be placed into the “left” bay. If both are used, they are hot-swappable and fully redundant. The FEX provides 10Gbps per blade port (there are 2 ports per blade). Each blade connects to both FEXs in the chassis via the mezzanine card.

© Cisco Systems, Inc.

Introduction to Cisco’s UCS Platform 1-30

CMC The FEX has a chassis management controller (CMC) which is a key part of management infrastructure. The CMC collects status data from the FEX using the IPMI (Intelligent Platform Management Interface) protocol over the I2C (inter-integrated circuit) serial bus and then communicates that information to the management node using the Ethernet server link. The CMC controls power supply and fan speeds. The CMC serves as a proxy for UCS manager to the blades for certain functionality. We will also see that the CMC plays a part in the HA protocols.

Diag port The FEX has a diag port used for “depot-level” troubleshooting.

UCS feature support: The FEX supports increased scalability by multiplexing an up to 8:1 oversubscribed blades to ports ratio.

1-31 UCSI Boot Camp

© Cisco Systems, Inc.

Backplane I/O Modules

PSU Connectors

Blade Connectors

Redundant data and management paths © 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training – Overview

Backplane The backplane provides redundant power to the FEXs and blades that plug into it. The backplane also provides redundant data network (Ethernet) connectivity between the FEX and the blades. Last, the backplane has redundant management (I2C) paths and dedicated management network (Ethernet) connectivity and supports auto-discovery of all components plugging into it. The backplane supports up to 1.2 Tbps on the data plane. While the back plane is largely passive it does have 3 SEEPROMS on it used for managing the HA capabilities of the 6120.

© Cisco Systems, Inc.

Introduction to Cisco’s UCS Platform 1-32

Introduction to UCS Management

© 2009 Cisco Systems, Inc. All rights reserved.

UCS Technical Training – Overview

Introduction to UCS Management This section will introduce the UCS management model. We will briefly look at the management architecture, high-availability and interfaces.

• • • • • •

Management architecture What is UCS Manager and where does it run? What does UCS Manager not do Introduction to UCS Manager high-availability UCS Manager interfaces Opt-in deployment models

1-33 UCSI Boot Camp

© Cisco Systems, Inc.

High-Level Management Architecture management interfaces

redundant management service

UCSM UCSM

managed endpoints switch elements

chassis elements

server elements

multiple protocol support

redundant management plane © 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training – Overview

High-Level Management Architecture UCS Manager is the name of the management service for all the UCS components. As such, users can manage the entire UCS system through a single pane of glass, thereby reducing management complexity. UCSM runs on the management node. In an HA configuration, there is a redundant instance of UCSM running on the second. Managed switch elements include GEM modules and ports. Managed chassis elements include the CMC, fan modules, and power supplies. Managed server elements include disks, mezz cards, BMC and BIOS.

© Cisco Systems, Inc.

Introduction to Cisco’s UCS Platform 1-34

UCS Manager and XML

GUI XM L CLI

configuration state

API 3rd party tools

© 2009 Cisco Systems Inc. All rights reserved.

Cisco UCS Manager

operational state

UCS Technical Training – Overview

UCS Manager and XML UCS Manager’s native language is XML. Both the GUI and CLI are written using UCS Manager’s XML API. 3rd party tools can also use this same XML API to integrate with UCS Manager. When user’s submit management configuration requests for a managed device to UCS Manager, UCS Manager will call the appropriate device manager to deploy the management request. A device’s operational state is communicated back to UCS Manager and reflected in the UI.

1-35 UCSI Boot Camp

© Cisco Systems, Inc.

Overview of HA in UCS L1 to L1

L2 to L2

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training – Overview

Overview of HA in UCS Configuring two interconnects in your UCS can provide redundancy for both the UCS Manager functionality as well as the switching functionality that the interconnect provides. You are not required to run in HA mode, but if you do, you must follow certain rules. First, you must connect the L1 port of the “left” switch to that of the “right” switch and L2 of left to L2 of right. These two Ethernet links are used for cluster protocol traffic only. Second, you must connect the “left” IO module to the “left” switch (using between 1 and 4 server links) and connect the “right” IO module to the “right” switch. Each UCS Manager instance either runs as the ‘primary’ or ‘subordinate’ – an election algorithm runs at switch startup to determine which is which. All management information is replicated between the two instances over the private network so that it persists across failovers. A “floating” management IP address is configured for UCS Managewr and is run on the primary instance. While a UCS Manager instance running on an interconnect is either primary or subordinate, both interconnects may be configured to switch Ethernet and FC traffic simultaneously. In other words, as management nodes, the switches run “active-passive” but as switches they may run “active-active”. © Cisco Systems, Inc.

Introduction to Cisco’s UCS Platform 1-36

UCS Manager Principal Interfaces GUI Navigation

CLI Equivalent to GUI

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training – Overview

UCS Principal Interfaces GUI The GUI is the principal access mechanism to UCS Manager. The GUI is a Java Application downloaded and launched (using Java Webstart technology) from a web server inside UCS Manager. It runs as a standalone application (it can survive death of the browser that launched it, for example). Since it is 100% Java, it should be nearly identical on a variety of client OS’s and platforms.

• Downloaded from UCS Manager using Java Webstart • Runs as standalone GUI • Changes made in GUI are refle CLI The CLI is a distinct, secondary interface to UCS Manager. Almost all tasks can be performed either in the GUI or the CLI, although the two are independent (for example, the GUI does not invoke the CLI). Configuration and changes made and committed in

1-37 UCSI Boot Camp

© Cisco Systems, Inc.

the CLI are reflected immediately in the GUI (and vice versa). Both the CLI and GUI subscribe to the event channels of the other.

• Use ssh (default) or telnet directly to UCS Manager IP • Independent control mechanism from GUI • Changes made in CLI are reflected in GUI

© Cisco Systems, Inc.

Introduction to Cisco’s UCS Platform 1-38

Out-of-the-Box Protocol Support SNMP

SMASH CLP

Call-home

IPMI

CIM XML

Remote KVM

UCS CLI and GUI

Serial Over LAN

UCS XML API

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training – Overview

UCSM Protocol Support UCS Manager supports a variety of industry-standard protocols for monitoring, configuration, and call-home. Some of these interfaces (e.g. Remote KVM) are cutthrough interfaces that “go around” UCS Manager while others (e.g. Call-home) use UCS Manager for configuration and activation. Configuration and detail about these protocols is covered in a later module in this course. The UCS XML API (and corresponding SDK) allow you to perform custom wrappers around UCS configuration and monitoring. There are several use cases for this:

• Creating a multi-tenant portal with your own • • • •

presentation/authorization/authentication mechanisms Integrating with custom or industry-standard orchestration tools Populating external CMDBs to include management changes in a UCS environment into your enterprise management system Scripting and integrating into custom management solutions. Extending the base functionality provided by UCS Manager

1-39 UCSI Boot Camp

© Cisco Systems, Inc.

Opt-in Models for UCS Management  Basic – 1:1 mapping from server identities/behaviors to servers – Uses built-in (hardware-derived) identities  Stateless computing – 1:many mapping from server identities /behaviors to servers – Uses pools, policies, and templates to assign identities and behaviors to servers  Multi-tenancy – 1:1 or 1:many mapping from resources to organizations – Uses RBAC to control privileges

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training – Overview

UCS Manager Opt-in Models In order to facilitate different levels of adoption for UCS features, customers can choose from essentially three different management paradigms: basic, stateless, and multitenancy.

Basic In this paradigm, the user can treat blades as physical servers in an analogous fashion as one would would a rack-mounted server. Consumers who require that the operating systems, applications, and users using a server in a UCS are confined to a particular blade could exploit this model. While this is the easiest model to adopt, it does not make use of many of the features UCS provides (namely mobility and organizational sharing of servers).

Stateless computing In this paradigm, the user does not care on which physical blade an operating system or application runs. Instead, users can pool servers together and then have UCS Manager automatically select a server that satisfies the physical or logical requirements of a user

© Cisco Systems, Inc.

Introduction to Cisco’s UCS Platform 1-40

(e.g. amount of memory installed, CPU capacity) as well as identity requirements (e.g. WWN, MAC) and behavior (e.g. firmware versions, boot order, … etc).

Multi-tenancy In this paradigm, users (or “tenants”) of the same UCS can all share its resources. UCS Mnager uses role-based access control (or RBAC) to grant or deny particular access to a particular resource for a particular user based on privileges assigned to a role.

1-41 UCSI Boot Camp

© Cisco Systems, Inc.

Overview of Server Deployment in UCS Configuration Methods

Service Profile

Physical Blade

Identity (MAC Address, WWN, Etc.)

Behavior (Firmware, QoS, Etc.)

• Manual • Automatic • Default

Other (vNICs, vHBAs, Etc.)

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training – Overview

Overview of Deployment in UCS Pools Identity pools such as MAC and WWN pools allow the server administrator to pull from a pre-defined set of identity addresses and assign them to servers. Server pools allow the server administrator to pull from a pre-defined set of blades that satisfy certain criteria (such as RAM and CPU capacity).

Policies Policies allow the server administrator to consume pre-defined behaviors such as boot order and QoS settings.

Templates Templates allow the server administrator to consume predefined profile, NIC, and HBA configuration.

© Cisco Systems, Inc.

Introduction to Cisco’s UCS Platform 1-42

Profiles The service profile is the heart of the UCS Manager. It represents all the identity and behavior of a server. When you need a blade with a particular identity and set of behaviors, you simply assign a service profile to that blade. Insodoing, UCS Manager configures a physical blade with the appropriate behavior and identities with LAN and SAN connectivity.

1-43 UCSI Boot Camp

© Cisco Systems, Inc.

UCS C-Series Servers

© 2009 Cisco Systems, Inc. All rights reserved.

UCS Technical Training – Overview

UCS C-Series Servers UCS C-Series servers are not yet part of UCS integrated management. They are standalone rack-mounted versions with the exact same CPU/Memory horsepower as their B-series equivalents. There are plans to integrate C-series rack-mounted servers into UCS integration (connected to the 6100 fabric interconnects, either directly or through a next-generation fex). This will likely occur in late CY 2010

© Cisco Systems, Inc.

Introduction to Cisco’s UCS Platform 1-44

C-Series Overview  Expands UCS into rack mount market UCS C250 M1

 Multiple offerings for different work loads • C200 - 1RU rack-mount server • C210 – 2RU large internal storage • C250 – 2RU Memory Extending

UCS C210 M1

 Offers Path to Unified Computing

UCS C200 M1

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training – Overview

C-Series Overview Cisco expands the UCS platform to now includes a series of servers that are in a rack mount form factor. These models are known as the Unified Compute System C-Series Rack mount servers. Consumers that do not use or prefer the rack mount architecture will also be able to take advantage of the many benefits of Cisco’s UCS system without being forced to use the B-Series Blade architecture. At launch not all UCS integration will be available for the c-series servers, but Cisco plans to complete this integration in CY2010. The C-Series offers 3 models:

• C200 – First model introduced. 1RU rack-mount server designed to balance simplicity, performance, and density for production-level virtualization, web infrastructure, and other mainstream data center workloads • C210 – Large internal storage model. designed to balance performance, density, and efficiency for workloads requiring economical, high-capacity, reliable, internal storage • C250 – Large memory server. designed to increase performance and capacity for demanding virtualization and large-data-set workloads

1-45 UCSI Boot Camp

© Cisco Systems, Inc.

Because the C-Series is designed to be part of the UCS system, it offers many of the same advantages:

• Simplifying management – Use existing tools to manage the server or UCSM (when available in 2010) • Unified I/O – The C-Series will use Converged Network Adapters as well as fibre channel and Ethernet adapters • Virtualization – The C-Series will have Virtualization adapter like the B-series. Large memory server is designed for large virtualized work loads

© Cisco Systems, Inc.

Introduction to Cisco’s UCS Platform 1-46

C-200 Overview Dongle for 2USB, VGA, Console DVD Internal Disk

UCS C200 Front View Console and Management Expansion Card

Power LOM

USB and VGA

UCS C200 Rear View

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training – Overview

C-200 Overview The C-200 is the general purpose high density model of the c-series line. The C-200 offers the following features:

• • • •

Up to two Intel Xeon 5500 Series multicore processors Up to 96 GB (DDR3) Up to four internal SAS or SATA disk drives; up to 2 terabytes (TB) total RAID capable: • Built-in RAID 0 and 1 support for up to four SATA drives • RAID 0 and 1 support for up to four SAS or SATA drives with optional mezzanine card • RAID 0, 1, 5, 6, and 10 support for up to four SAS or SATA drives with optional LSI MegaRAID card • Two half length x8 PCIe slots • One full height • One low profile • Two integrated Gigabit Ethernet ports and one 10/100 Mbps management port • KVM access 1-47 UCSI Boot Camp

© Cisco Systems, Inc.

• Front-panel interface with video, two USB, and serial port connections. • Back-panel video, two USB, and serial port connections • Dual redundant power

© Cisco Systems, Inc.

Introduction to Cisco’s UCS Platform 1-48

Dongle for 2USB, VGA, Console

C-210 Overview

DVD Internal Disk

UCS C210 Front View Console and Management

Expansion Card

Power

USB and VGA

© 2009 Cisco Systems Inc. All rights reserved.

UCS C210 Rear View

LOM

UCS Technical Training – Overview

C-210 Overview The C-210 is the general purpose, high density, and large internal storage model of the cseries line. The C-210 offers the following features:

• Up to two Intel Xeon 5500 Series multicore processors • Up to 96 GB (DDR3) • Up to 16 internal small form factor (SFF), (SAS), or (SATA) disk drives; up to 8 TB total • RAID capable: • Built-in RAID 0 and 1 support for up to four SATA drives • RAID 0 and 1 support for up to four SAS or SATA drives with optional mezzanine card • RAID 0, 1, 5, 6, and 10 support for up to 16 SAS or SATA drives with optional LSI MegaRAID card • Five full-height PCIe slots: • two full-height, full-length x8 PCIe card slots • three full-height, half-length x8 PCI card slots, all with x16 connectors • Two integrated Gigabit Ethernet ports and one 10/100 Mbps management port 1-49 UCSI Boot Camp

© Cisco Systems, Inc.

• KVM access • Front-panel interface with video, two USB, and serial port connections. • Back-panel video, two USB, and serial port connections • Dual redundant power

© Cisco Systems, Inc.

Introduction to Cisco’s UCS Platform 1-50

C-250 Overview Dongle for 2USB, VGA, Console

Internal Disk

DVD

UCS C250 Front View

Power Expansion Card

UCS C250 Rear View

USB and VGA LOM Console and Management

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training – Overview

C-250 Overview The C-250 server is a high-performance, memory-intensive, two-socket, 2RU rack-mount server designed to increase performance and capacity for demanding virtualization and large-data-set workloads. It also can reduce the cost of smaller memory footprints. The C250 offers the following features:

• • • •

Up to two Intel Xeon 5500 Series multicore processors Up to 384GB (DDR3) Up to eight internal small form factor (SFF), (SAS), or (SATA) drives; up to 4 TB total RAID capable: • RAID 0 and 1 support for up to eight SAS or SATA drives through optional PCI Express (PCIe) controller • RAID 0, 1, 5, 6, and 10 support for up to 8 SAS or SATA drives with optional LSI MegaRAID card • Support for up to five PCIe cards in three low-profile, • half-length x8 and two full-height, • half-length x16 slots • Four integrated Gigabit Ethernet ports and 2 x 10/100 Mbps management port 1-51 UCSI Boot Camp

© Cisco Systems, Inc.

• KVM access • Front-panel interface with video, two USB, and serial port connections. • Back-panel video, two USB, and serial port connections • Dual redundant power

© Cisco Systems, Inc.

Introduction to Cisco’s UCS Platform 1-52

PCIe x16

127 0 1 2 3

Virtualization • • • •

UCS P81e 128 vNICs Uses VNlink NIC Teaming done by HW

PCIe Bus

CNA 10GbE FC

10GbE/FCoE

vNICs

Eth Eth Eth FC FC

10GbE/FCoE

Adapter Offerings

• • • •

Emulex and Qlogic 2 Fibre Channel 2 Ethernet NIC Teaming through bonding driver

Ethernet or HBA • Emulex / Qlogic (HBA) • Broadcom Ethernet • NIC Teaming through bonding driver © 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training – Overview

Adapter Offerings The 3 types of adapters cards available in a UCS C-Series.

Virtual Interface Card (UCS P81e) Cisco UCS P81E Virtual Interface Card is a virtualization-optimized Fibre Channel over Ethernet (FCoE) PCI Express (PCIe) 2.0 x8 10-Gbps adapter designed for use with Cisco UCS C-Series Rack-Mount Servers. The virtual interface card is a dual-port 10 Gigabit Ethernet PCIe adapter that can support up to 128 PCIe standards-compliant virtual interfaces, which can be dynamically configured so that both their interface type (network interface card [NIC] or host bus adapter [HBA]) and identity (MAC address and worldwide name [WWN]) are established using just-in-time provisioning. In addition, the Cisco UCS P81E can support network interface virtualization and Cisco VN-Link technology.

1-53 UCSI Boot Camp

© Cisco Systems, Inc.

Converged Network Adapters The converged network adapter (CNA) offers and the ability to provide converged network fabrics across 10 Gbe. The adapters have 2 host-side 10GE ports and 2 FC ports to the CAN, but then will convert all fibre channel and Ethernet traffic from these ports to be transported across 10 Gbe of the consumers network. These CNAs then would be linked to a Nexus 5K as an example in order to use the FCoE capabilities. There are two manufacturers of CAN, Qlogic and Emulex.

• Emulex OneConnect • Qlogic QLE8152 Ethernet and/or HBAs You can also opt for traditional HBA and Ethernet Connectivity. Fro Ethernet we offer Broadcom NetXtremeII cards with either 2 or 4 port connectivity. For HBA we offer both Emulex and Qlogic. Ethernet:

• Broadcom NetXtremeII 4 port • Broadcom NetXtremeII 2 port HBA:

• Emulex LightPulse LPe11002 • Qlogic SANBlade QLE2462

© Cisco Systems, Inc.

Introduction to Cisco’s UCS Platform 1-54

C-Series Storage RAID Controllers

Disks

• 1 Built in controller (ICH10R) • Option LSI 1064e based mezz controller • Option LSI 1078 based Mega RAID controller

• 3.5 inch and 2.5 inch form factors • 15K SAS (High Performance) • 10K SAS (Performace) • 7200 SAS (High Cap / Perf) • 7200 SATA (Cost and Cap) • 73GB, 146GB, 300GB, and 500GB

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training – Overview

C-Series Storage The C-Series has multiple internal drives, 4 on the C-200 , 16 on the C-210, and 8 on the C-250. These drives can be controlled by an internal RAID controller. The C-series offers a single built in RAID controller, ICH10R RAID, to do simple RAID 0 and 1. This controller is not compatible for use with VMWare ESX Server software. In addition to the built in RAID controller, you have the option of getting either:

• LSI 1064-based controller mezzanine card – Only offers 0 and 1 capabilities • LSI 1078-based MegaRAID controller card – Offers 0,1,5,6,and 10 support Disks come in multiple options. Depending on the model you have the following options C-200

• • • •

250-GB SATA; 7200 RPM 500-GB SATA; 7200 RPM 46-GB SAS; 15,000 RPM 300-GB SAS; 15,000 RPM

1-55 UCSI Boot Camp

© Cisco Systems, Inc.

• 1-TB SAS; 7200 RPM C-210

• • • •

73-GB SAS; 6G, 15,000 RPM 146-GB SAS; 6G, 10,000 RPM 300-GB SAS; 6G,10,000 RPM 500-GB SATA; 7200 RPM

C-250

• • • •

73-GB SAS; 6G, 15,000 RPM 146-GB SAS; 6G, 10,000 RPM 300-GB SAS; 6G,10,000 RPM 500-GB SATA; 7200 RPM

© Cisco Systems, Inc.

Introduction to Cisco’s UCS Platform 1-56

C-Series Power PSU Options • C-200 can operate at 450W • C-200 and 210 650W PSU • C- 250 750W PSU

C-200/210 © 2009 Cisco Systems Inc. All rights reserved.

C-250 UCS Technical Training – Overview

C-Series Power The C-Series has a number fo power options fepending on the model. For power efficiency with the C-200 you can option a 450 W PSU as opposed to the 650W PSU. Because of the potential for more disk drives the C-210 comes with 650 W PSU. Finally the C-250 with its ability to house up to 384 GB RAM uses a 750W PSU. The 650 and 450 watt PSU have the same form factor.

1-57 UCSI Boot Camp

© Cisco Systems, Inc.

C-Series Cooling Cooling Info • Front to Back • Internal Fans in the C200 and 210 • Hot Swappable External Units C-250 • PSU have separate Fans

C-200

C- 210 © 2009 Cisco Systems Inc. All rights reserved.

C-250 UCS Technical Training – Overview

C-Series Cooling The C-series have front to back cooling compliant with the warm row cool row design. In the C-200 and 210 models the fans are internal to the unit as shown in the picture above. This means it will require opening the system to replace the fans. On the C-250 the Fans are module sin the front of the server and are removable and hot-swappable.

© Cisco Systems, Inc.

Introduction to Cisco’s UCS Platform 1-58

Management and UCS C-Series

© 2009 Cisco Systems, Inc. All rights reserved.

UCS Technical Training – Overview

This section will introduce the UCS management model. We will briefly look at the management architecture, high-availability and interfaces.

• Management architecture of C-Series • What is CIMC does and does not do • Deployment of C-series

1-59 UCSI Boot Camp

© Cisco Systems, Inc.

High-Level Management Architecture management interfaces

managed endpoints

management service

Administrative tasks Communications User Management Network Configuration

CIMC BMC

Server Monitoring Inventory IPMI Logs

BIOS

RAID Configuration Boot Options Server Options

Web GUI Console © 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training – Overview

High-Level Management Architecture This is a very high level view of the management of the C-Series. While full integration with the UCSM is not available currently, you can still effectively manage the server using the same tools you use currently. Included with the C-Series are also some unique management and monitoring capabilities.

CIMC – Cisco Integrated Management Controller CIMC is a separate management module that is built onto the motherboard. The CIMC has its own processor which runs the CIMC software. The CIMC is shipped with a running version of the firmware. Users can update the CIMC firmware through the firmware Update Management page, You need not worry about installing the initial CIMC firmware. Users do not install a host OS like Windows or Linux on the CIMC. The host OS runs on the Intel host processor. You install the host OS on the host hard drive using the DVD drive, or over the network. You can use the CIMC to install the host OS using the KVM console and vMedia. You can use a web-based GUI or SSH-based CLI to access, configure, administer, and monitor the server. Almost all tasks can be performed

© Cisco Systems, Inc.

Introduction to Cisco’s UCS Platform 1-60

in either interface, and the results of tasks performed in one interface are automatically displayed in another. Tasks you can perform using the CIMC:

• • • • • • • • • • • •

Power on, power off, power cycle, reset and shut down the server Toggle the locator LED Configure the server boot order View server properties and sensors Manage remote presence Create and manage local user accounts, and enable remote user authentication through Active Directory Configure network-related settings, including NIC properties, IPv4, VLANs, and network security Configure communication services, including HTTP, SSH, and IPMI Over LAN Manage certificates Configure platform event filters Update CIMC firmware Monitor faults, alarms, and server status

Tasks you CANNOT do with CIMC: • • • • • •

Deploy an OS, such as Windows or Linux Deploy patches for software, such as an OS or an application Install base software components, such as anti-virus software, monitoring agents, or backup clients Install software applications, such as databases, application server software, or web servers Perform operator actions, including restarting an Oracle database, restarting printer queues, or handling non-CIMC user accounts Configure or manage external storage on the SAN or NAS storage

BMC Baseboard Management Controller The BMC is basically the same as the one found on the B-Series blade servers. While this device is not used directly like the CIMC to manage the server it will be critical for managing and monitoring the server when we have full UCSM integration. Currently it can be used with IPMI to control basic functions of the server as well as monitoring.

1-61 UCSI Boot Camp

© Cisco Systems, Inc.

BIOS There are a number of BIOS on the C-Series for configuring either the Server itself or the various adapter cards that can be housed inside the server.

© Cisco Systems, Inc.

Introduction to Cisco’s UCS Platform 1-62

C-Series Management Connectivity

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training – Overview

C-Series Management Connectivity The principal management feature is through redundant out-of-band network connections to each rack-mounted server’s BMC. A web-page is presented by the BMC which is similar in nature to the “server (equipment)” and “server(logical)” views in UCS --- from this page you can power on and power off the server, perform other maintenance tasks, and launch the same KVM remote console tool that is used with B-series blades. Of course on board standard video and USB console access is available as well.

1-63 UCSI Boot Camp

© Cisco Systems, Inc.

C-Series Deployment Scenarios

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training – Overview

C-Series Deployment Scenarios This slide gives you an idea of the flexibility of these particular products with Cisco networking products and how it'll be integrated from a UCS perspective. In the middle, you have displayed the UCS C250, 210 and 200, and how we see it being deployed in various scenarios either with third party products, with Cisco switches like the Nexus 5000, for example, or the Cisco Nexus 2000 Fabric Extender, or into a UCS environment moving forward. If you look at how the B-Series kind of plays in this overall portfolio, you've got the UCS, the blade server chassis and products to the left, and really the CSeries allows you the flexibility to play in a standalone environment, connect to various Cisco gear with other third party gear that may be already in place in the data center, or a migration path to the UCS in the future. Whereas you see where the Cisco Unified Computing system on the left, how that plays, it's kind of a system in its own and doesn't provide some of the flexibility, although it has tremendous TCO advantage and other advantages associated with that deployment model.

© Cisco Systems, Inc.

Introduction to Cisco’s UCS Platform 1-64

Lesson 2

Introduction to UCS User Interfaces Overview The purpose of this module is to familiarize yourself with the GUI and the CLI. Focus in the practical exercises will be on the equipment areas of the GUI and CLI; you will get plenty of practice creating and managing logical objects in the hands-on labs in the remainder of the course Objectives The specific objectives of this lesson are to enable you to explain the following about UCS GUI and CLI: •

Become familiar with navigating the GUI

•

Understand the functions of the 5 main tabs on the navigation pane

•

Become familiar with the CLI and CLI scoping

•

Use CLI commands to perform creation and deletion of managed objects

UCS Manager UI Overview Objectives

• Become familiar with navigating the GUI • Understand the functions of the 5 main tabs on the navigation pane • Become familiar with the CLI and CLI scoping • Use CLI commands to perform creation and deletion of managed objects

© 2008 Cisco, Inc. All rights reserved.

DCTC v1.0—2

The purpose of this module is to familiarize yourself with the GUI and the CLI. Focus in the practical exercises will be on the equipment areas of the GUI and CLI; you will get plenty of practice creating and managing logical objects in the hands-on labs in the remainder of the course.

• • • •

Become familiar with navigating the GUI Understand the functions of the 5 main tabs on the navigation pane Become familiar with the CLI and CLI scoping Use CLI commands to perform creation and deletion of managed objects

2-1 UCS I Boot Camp

© Cisco Systems, Inc.

Primary view of Graphical interface

NAVIGATION PANE © 2008 Cisco, Inc. All rights reserved.

CONTENT PANE DCTC v1.0—3

Primary view of Graphical interface The left or Navigation Pane consist of a fault summary bar across the top view and a series of 5 tabs that offer differing views of the various managed components in the Unified Computing System. The fault summary has four conditions ranging from critical, major, minor and warning. The faults summary contains all the cumulative totals for the entire UCS An expandable branch or tree function allows the operator to traverse the various components located in the five tabs. The right or Content Pane consist of a top toolbar with a back button, new object creation pulldown, options & questions buttons, information button and a debug pulldown menu. The second toolbar in the content pane offers the operator a breadcrumbs trail of object hierarchies already traversed with the ability to rapidly return to a previous location along the trail. At the right most portion of this bar is the current location. The largest part of the content pane offers granular details associated with the varying objects that have been highlighted in the navigation pane. And at the vary bottom the

© Cisco Systems, Inc.

Introduction to UCS User Interfaces 2-2

content pane are the function buttons associated with committing or saving as well as discarding the changes made here.

2-3 UCS I Boot Camp

© Cisco Systems, Inc.

Navigation Pane Tabs (Equipment)

This shows current UCS HW components

© 2008 Cisco, Inc. All rights reserved.

DCTC v1.0—4

Navigation Pane Tabs (Equipment) This view displays the Equipment tab contents in the Navigation pane. The display is in a hierarchical format that list chassis 1-40 and then fabric interconnect A and B Subcomponents such as blades and interface cards are then expanded out from the chassis objects to perform operations on individual components and their attributes. Each hardware object that is highlighted then has its corresponding attributes displayed on the content pane. The content pane may have 1 or more tabs depending on the amount of attributes or functions. At the top of the Navigation pane is a filter pulldown for your view. The filter is set to “all” by default but can be set to restrict the object view to the topmost objects in that tab. In the case of the Equipment view the Chassis or fabric interconnects are the topmost objects.

© Cisco Systems, Inc.

Introduction to UCS User Interfaces 2-4

Navigation Pane Tabs (Servers)

This tab is for the creation of logical servers

© 2008 Cisco, Inc. All rights reserved.

DCTC v1.0—5

Navigation Pane Tabs (Servers) This is the servers tab. As opposed to the equipment tab, this tab focuses on the logical deployment of servers by configuring and managing service profiles and their associated templates, policies and pools. This tab is the “heart” of configuration of UCS servers. The idea is that the network administrator and SAN administrator can spend “up front” time configuring global network and SAN settings on the LAN and SAN tabs. After this, all the information comes together on this servers tab during the configuration of service profiles.

2-5 UCS I Boot Camp

© Cisco Systems, Inc.

Navigation Pane Tabs (LAN)

This tab is for network related tasks like creating VLANs

© 2008 Cisco, Inc. All rights reserved.

DCTC v1.0—6

Navigation Pane Tabs (LAN) The LAN tab contains network configuration that will then be visible and available for the configuration of service profiles. This tab is focused mostly on global network configuration that will affect UCS northbound connectivity, and configure a set of choices, pools, and policies related to the network arena that will be available for incorporation into service profiles. Another way of saying this is that a lot of the information on the LAN tab is created by network administrators and then consumed by server administrators as they set up service profiles.

© Cisco Systems, Inc.

Introduction to UCS User Interfaces 2-6

Navigation Pane Tabs (SAN)

This tab is for SAN related tasks like creating VSANs

© 2008 Cisco, Inc. All rights reserved.

DCTC v1.0—7

Navigation Pane Tabs (SAN) The SAN tab contains SAN storage configuration that will then be visible and available for the configuration of service profiles. It is very analogous to the LAN tab. Alot of the information on the SAN tab (VSAN’s available, WWN pools) is created by storage administrators and then consumed by server administrators as they set up service profiles.

2-7 UCS I Boot Camp

© Cisco Systems, Inc.

Navigation Pane Tabs (VM)

This tab is specifically for Virtual Card (Palo) Passthrough switch (PTS) feature

© 2008 Cisco, Inc. All rights reserved.

DCTC v1.0—8

Navigation Pane Tabs (VM) This tab deals specifically with the Palo Pass-through switch (PTS) feature for VMware. It is not at this time used for any other purpose. The Palo PTS feature will involve making a connection between USCM and VMware VCenter (the “VM Providers” setup). VM Profiles (port profiles) will be pushed to VCenter for the attachment of virtual machines to the pass-through switch. Each virtual machine adapter will be “wired straight through” the PTS to a Palo virtualized NIC which will have the exact same visibility as physical NIC’s in the UCS system.

© Cisco Systems, Inc.

Introduction to UCS User Interfaces 2-8

Navigation Pane Tabs ( Admin)

This tab is for UCS related management tasks like RBAC

© 2008 Cisco, Inc. All rights reserved.

DCTC v1.0—9

Navigation Pane Tabs ( Admin) The Admin tab is used for administrative tasks such as setting up users and external authentication schemes. IP management (for management functionality) only is also controlled on this screen.

2-9 UCS I Boot Camp

© Cisco Systems, Inc.

Content Pane Details UCS Manager control buttons

Bread Crumbs View for Content Pane

Fault summary for selected object in NAV Pane Actions for selected object in NAV Pane

© 2008 Cisco, Inc. All rights reserved.

Content Pane tabs for selected object (different for each object)

DCTC v1.0—10

Content Pane Details Content Pane is designed to show you information of an object you select in the Nav Pane. Any changes to the object will be done in the Content Pane. There are a series of tabs near the top of the Content Pane, and depending on which you select it will allow you to monitor or change the characteristics of the object. As an example if you wish to change the firmware, then the “installed firmware” tab would be the tab you select. In addition to the Content Pane tabs, there is also a actions menu where you can perform a given action for the item. Included in the content pane are also the fault summary for the object, a “bread crumb” view to indicate where in the Nav Pane you are and buttons for controlling UCS manager settings.

© Cisco Systems, Inc.

Introduction to UCS User Interfaces 2-10

Example Content – Tabs and Sub-Tabs (Physical Server RAM)

© 2008 Cisco, Inc. All rights reserved.

DCTC v1.0—11

Example Content – Tabs and Sub-Tabs (Physical Server RAM) This is just a nice example of the cascading tabs sometimes available on the content pane. This is the content pane associated with (highlighting) a physical blade server on the nav pane. The Inventory tab gives a set of sub-tabs which have all the autodiscovered hardware information about the blade, including the position, size, and clock speed of each DIMM. None of this particular information is configured manually, it is all discovered automatically.

2-11 UCS I Boot Camp

© Cisco Systems, Inc.

UCS Command Line Interface

Command Line Interface

© 2008 Cisco, Inc. All rights reserved.

© Cisco Systems, Inc.

DCTC v1.0—12

Introduction to UCS User Interfaces 2-12

Management Command (?) devwasha-A# ? acknowledge clear commit-buffer connect decommission discard-buffer end exit recommission remove scope set show terminal top up where

Acknowledge Reset functions Commit transaction buffer Connect to Another CLI Decommission managed objects Discard transaction buffer Go to exec mode Exit from command interpreter Recommission Server Resources Remove Changes the current mode Set property values Show running system information Set terminal line parameters Go to the top mode Go up one mode Show information about the current mode

© 2008 Cisco, Inc. All rights reserved.

DCTC v1.0—13

Management Command (?) You can always use the question mark to see commands available in a particular context. If you are in the middle of typing a command (at a new word boundary), you can use “?” to see the set of subcommands available in the exact context in which you are typing.

2-13 UCS I Boot Camp

© Cisco Systems, Inc.

Management Commands (show ?) devwasha-A# show ? chassis cli clock cluster configuration eth-uplink event fabric-interconnect fault identity iom org security sel server service-profile system timezone version vif

Chassis CLI commands Display current Date Cluster mode Show information about configuration sessions Ethernet Uplink Event Manager commands Show Fabric Interconnect Fault Identity IO Module Organizations Security mode System Event Log Server Service Profile System-related show commands Set timezone System version Virtual Interfaces

© 2008 Cisco, Inc. All rights reserved.

DCTC v1.0—14

Management Commands (show ?) This is an example of “?” in the “middle of command” context. Note the “scope context” (discussed on subsequent slides) is also taken into account.

© Cisco Systems, Inc.

Introduction to UCS User Interfaces 2-14

Management Commands (scope ?) devwasha-A# scope ? adapter chassis eth-server eth-uplink fabric-interconnect fc-uplink firmware host-eth-if host-fc-if monitoring org security server service-profile system vhba vnic

Mezzanine Adapter Chassis Ethernet Server Domain Ethernet Uplink Fabric Interconnect FC Uplink Firmware Host Ethernet Interface Host FC Interface Monitor the system Organizations Security mode Server Service Profile Systems VHBA VNIC

© 2008 Cisco, Inc. All rights reserved.

DCTC v1.0—15

Management Commands (scope ?) The scope command is used to change the focus of the object to be operated upon. This will allow the traversing of the hierarchical tree of objects and their child or parent objects. Don’t forget that the show command can be used in conjunction with the scope command to see immediately the commands available in the new position. There are parts of the “scope tree” for physical objects and other parts for logically configured objects. Moving around the CLI tree using scoping is analogous to using “cd” to move around a file system.

2-15 UCS I Boot Camp

© Cisco Systems, Inc.

Management Commands (scope, where, up & top) Nav Pane Navigation

CLI Equivalent to Nav Pane

© 2008 Cisco, Inc. All rights reserved.

DCTC v1.0—16

Management Commands (scope, where, up & top) In this slide we see the difference between the GUI Nav Pane to focus on a given object, and the CLI use of the scope command to focus on the same object. Notice 2 levels are traversed and then the up command is used to ascend one level. Then the top command is used to return to the root level. Also of note is the use of the command “where”. Since we have to scope to the level of a component much like changing from one directory to another, the “where” command is useful in determining your location in the CLI “scope tree”.

© Cisco Systems, Inc.

Introduction to UCS User Interfaces 2-16

Command Line (show configuration) devwasha-A# scope org / devwasha-A /org # scope service-profile TryPXEInst devwasha-A /org/service-profile # show config enter service-profile TryPXEInst instance associate server 1/8 enter boot-definition enter lan enter path primary set vnic eth0 exit set order 2 exit enter storage enter local set order 1 exit set descr "" set enforce-vnic-name no set reboot-on-update yes exit enter default-behavior vhba set action none exit ............ © 2008 Cisco, Inc. All rights reserved.

DCTC v1.0—17

Command Line (show configuration) “show configuration” is a special command. It goes down recursively from the scope in which it is typed, and shows the configuration in the format of the commands that would have been used to create or administer the current state the object at the current scope and all sub objects. The example starts at a service profile and shows how all the recursive configuration underneath that service profile might have been created with CLI commands. The output is a reverse engineering of the CLI commands because the configuration is not really stored as a set of CLI commands (it is stored in XML).

2-17 UCS I Boot Camp

© Cisco Systems, Inc.

Management Commands (create and delete) devwasha-A# scope org / devwasha-A /org # create org marketing devwasha-A /org* # commit devwasha-A /org # top devwasha-A# show org Organizations: Name ---/ (root) /marketing devwasha-A# scope org / devwasha-A /org # delete org marketing devwasha-A /org* # commit devwasha-A /org # top devwasha-A# show org Organizations: Name ---/ (root)

© 2008 Cisco, Inc. All rights reserved.

DCTC v1.0—18

Management Commands (create and delete) When you actually modify the configuration, using “create” and “delete” and other commands, you must commit your changes, as shown, to actually make the configuration (the equivalent on the GUI is pressing an “confirm” button). The opposite of “commit” is “discard” --- this will discard all your changes since the last commit.

© Cisco Systems, Inc.

Introduction to UCS User Interfaces 2-18

UCS Software

Connect commands

© 2008 Cisco, Inc. All rights reserved.

2-19 UCS I Boot Camp

DCTC v1.0—19

© Cisco Systems, Inc.

Management command (connect) devwasha-A# connect ? adapter Mezzanine Adapter bmc Baseboard Management Controller clp Connect to DMTF CLP iom IO Module local-mgmt Connect to Local Management CLI nxos Connect to NXOS CLI

© 2008 Cisco, Inc. All rights reserved.

DCTC v1.0—20

Management command (connect) The “connect” command really takes you into one of the other shells available in the CLI. We will focus on the “local-mgmt” shell and the “nxos” shell, however here is a summary of what these connect shells do:

Adapter Allows a local connection to the mezzanine adapter on a given blade. Syntax, connect adapter 1/1/1 (Chassis 1 / server 1 / Adpater 1). This would be used only for troubleshooting.

BMC Baseboard Management Controller is an ASIC on the mother board of each blade. This allows you to connect to the BMC (which has a running Linux kernel) and perform basic information gathering from the BMC. Syntax, connect bmc 1/1 (Chassis 1 / Server 1).

© Cisco Systems, Inc.

Introduction to UCS User Interfaces 2-20

CLP Connects you to a shell for the SMASH (Systems Management Architecture for Server Hardware) CLP (Command Line Protocol). If you have a customer who uses this protocol they can interface to the UCS and collect information about the many components and what their status is. The connect command puts you in a shell that accepts SMASH CLP commands. At present it is limited in the scope of commands you can run, mostly being the show command.

IOM I/O Module also has an ASIC running a linux kernel. Through the connect command you can connect to the IOM similar to doing a ssh session to a linux server. You will have a limited number of commands you can run, but this can be useful for troubleshooting problems on an IOM.

Local-mgmt Local management allows the user to run commands that allow for maintenance activities to be done on the 6120 including things like show tech-support and erasing the current database.

NXOS connects to the underlying NXOS CLI. From here users can run typical NXOS commands and examine connectivity through the 6120.

2-21 UCS I Boot Camp

© Cisco Systems, Inc.

Management commands (connect local-mgt) devwasha-A# connect local-mgmt Cisco UCS 6100 Series Fabric Interconnect TAC support: http://www.cisco.com/tac Copyright (c) 2009, Cisco Systems, Inc. All rights reserved. devwasha-A(local-mgmt)# ? Change current directory cd clear Reset functions cluster Cluster mode connect Connect to Another CLI copy Copy a file cp Copy a file . . ping Test network reachability pwd View current directory reboot Reboots Fabric Interconnect rm Remove a file rmdir Remove a directory run-script Run a script show Show running system information ssh SSH to another system . © 2008 Cisco, Inc. All rights reserved.

DCTC v1.0—21

Management commands (connect local-mgt) “local-mgmt” tasks are mainly tasks associated with the particular fabric interconnect on which you are typing, rather than on the UCS configuration as a whole. For example, you do not reboot the entire UCS; instead you can reboot (from the local-mgmt shell) a particular interconnect (the one on which you are typing).

© Cisco Systems, Inc.

Introduction to UCS User Interfaces 2-22

Management commands (connect nx-os) devwasha-A# connect nxos Cisco UCS 6100 Series Fabric Interconnect TAC support: http://www.cisco.com/tac Copyright (c) 2009, Cisco Systems, Inc. All rights reserved. devwasha-A(nxos)# sh run int e1/1/1 version 4.1(3)N2(1.1) interface Ethernet1/1/1 switchport mode trunk switchport trunk allowed vlan none no pinning server sticky fabric-interface Eth1/7 no cdp enable no shutdown

© 2008 Cisco, Inc. All rights reserved.

DCTC v1.0—22

Management commands (connect nx-os) The “connect nxos” command allows for connection to the NXOS shell. This provides the user with the full set of “show” command functionalities associated with the underlying nxos on the fabric interconnect on which you are typing. You do not have access to any nxos configuration commands. The nxos shell is “show and tell” only. Actual management is always accomplished using the UCS manager.

2-23 UCS I Boot Camp

© Cisco Systems, Inc.

Lesson 3

UCS Connectivity and Networking Overview This module will focus both on physical connectivity of the data and management planes within UCS, and the logical configuration of LAN and SAN connectivity within UCS and northbound of UCS. Objectives The specific objectives of this lesson are to enable you to explain the following about UCS management: • Understand UCS network and management connectivity • Understand and use management services provided by blade server BMC • Understand LAN Configuration within UCS • Understand SAN Configuration within UCS

UCS Connectivity

•

Understand UCS network and management connectivity

•

Understand and use management services provided by blade server BMC

•

Understand LAN Configuration within UCS

•

Understand SAN Configuration within UCS

© 2008 Nuova, Inc. All rights reserved.

ICNX5 v1.0—2

This module will focus both on physical connectivity of the data and management planes within UCS, and the logical configuration of LAN and SAN connectivity within UCS and northbound of UCS. • • • •

Understand UCS network and management connectivity Understand and use management services provided by blade server BMC Understand LAN Configuration within UCS Understand SAN Configuration within UCS

3-1 UCSI Boot Camp Sales

© Cisco Systems, Inc.

UCS Connectivity: Fabric Interconnect Mgmt 0

L1 Clustering Clustering L2

Mgmt 5 © 2009 Cisco Systems Inc. All rights reserved.

Console Port UCS Technical Training — Networking 3

UCS Connectivity: Fabric Interconnect The fabric interconnect provides crossover ports (L1 and L2) that are used to connect to the other fabric interconnect in a dual-fabric configuration only. These links are management links only; they provide control for the clustering of the management function and replication of the managed data. As of writing, the officially supported configuration uses direct links between the fabric interconnect only. There is a single network management port for external client management (both CLI, via ssh, and the GUI) on each fabric interconnect. This is a 10/100/1000 Mb port. Serial Console runs 9600/8/N. When the management system is running, the serial console provides a CLI login. The only time that serial console access is absolutely required is on initial setup of the system.

© Cisco Systems, Inc.

Introduction to UCS Connectivity and Networking 3-2

UCS Data Connectivity (Data Plane)

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training — Networking 4

UCS Connectivity (Data Plane) Each fabric interconnect can connect to only one IOM (fex) inside the chassis, and each IOM can be connected to only one fabric interconnect. At the time of writing, there is no virtual port channel (in fact, no port channel at all) between the fabric interconnect and the IOM. You can choose 1, 2, or 4 downlinks from a fabric interconnect to an IOM. You do not need to have the same number of downlinks to each chassis. In the dual-fabric scenario, it is logical but not absolutely required that both IOM’s will have the same number of links connected to their respective fabric interconnect. The diagram shows IOM “satellite ports” (1-8) connected via dedicated 10gb FCOE traces on the midplane to specific adapter slots. In the half-width blade, there is a single adapter. A full-width blade will occupy two adjacent slots (1 and 2, or 3 and 4, for example). If a full-width blade in slots 3 and 4 has two adapters, then the “blade adapter slot 3” and the “blade adapter slot 4” in the diagram will refer to the two adapters on a single full-width blade.

3-3 UCSI Boot Camp Sales

© Cisco Systems, Inc.

UCS – Connectivity: Blade Slot to Fex Pinning 8 to 1

2 to 1

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training — Networking 5

UCS Connectivity: Blade Slot to Fex Pinning Blade adapter slots are currently pinned to IOM uplinks. While the Nexus 5K/2K relationship has optional port-channeling on its uplinks, there is no port channel between the UCS 6100 and the IOM. The blade adapters are pinned to IOM/fabric interconnect uplinks as such: 1 uplink: all 8 blades (naturally) 2 uplinks: odd adapter slots to first uplink , even adapter slots to second 4 uplinks: pinning is in groups: (1,5) , (2, 6), (3. 7) (4. 8) Note: the “first uplink, second uplink” above has nothing to do with the IOM uplink port; rather it has to do with the first fabric interconnect port configured as a fabric (server port) to the IOM, the second, etc. Why do they not pin a 4-uplink scenario using (1,2). (3.4), etc? The reason is that if you had full-width blades, that scheme would pin both adapters of a full-width blade to the same uplink! But it’s much better redundancy to pin them to different uplinks, so

© Cisco Systems, Inc.

Introduction to UCS Connectivity and Networking 3-4

that if one uplink fails, you have connectivity through the same IOM at least to the other adapter. Note that if an uplink fails, blade adapter slots do not automatically get repinned. Instead. some form of adapter failover (to be controlled either by the OS, or by UCSmanaged adapter failover) must be in place to fail application traffic over to the other IOM (and hence the other fabric interconnect).

3-5 UCSI Boot Camp Sales

© Cisco Systems, Inc.

Adapter Connectivity Details: Oplin

Eth0 To IOM 1

To IOM 2

Eth1

Teaming done by O/S

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training — Networking 6

Adapter Connectivity: Oplin The Oplin adapter card supports a single 10Gb ethernet connection to each IOM (and hence to each fabric interconnect). No failover is provided internally by UCS. Redundancy is achieved only through some kind of OS teaming / grouping only. If you have a full-width blade, of course you could configure OS redundancy across two different adapters rather than on a single adapter. If the OS teaming scheme involves some kind of OS-created virtual adapter with a single MAC address, then the teaming must be active/passive only (UCS cannot support the same MAC address simultaneously across both IOMs and fabric interconnects). If the teaming scheme still preserves a separate MAC address for each physical adapter, with some other (typically layer-3) load-balancing scheme, then there could still be an active/active redundancy scheme.

© Cisco Systems, Inc.

Introduction to UCS Connectivity and Networking 3-6

Adapter Connectivity Details: CNA (Menlo)

To IOM 1

Eth0 Eth1

HBA0 HBA1

To IOM 2

Failover done by Menlo for Eths only © 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training — Networking 7

Adapter Connectivity Details: CNA The Menlo CNA provides two 10Gb ethernets. You can only use both if you have two fabric interconnects (there is no redirection of traffic of both adapters across a single IOM connection unless you start with two separate paths and specify that UCS should provide failover). For the ethernet functionality, you can choose to configure either or both adapters with UCS provided failover. In this case you would need no OS teaming; UCS would direct traffic of eth0, for example, to the left IO Module, but it would automatically fail the traffic to the other IOM (and fabric interconnect) in the case of IOM, uplink, or possibly even northbound network failure. Note that in this case the OS is completely unaware of teaming or failover; an OS can use, for example., eth0 by itself, and UCS will automatically provide path redundancy for this single adapter. It is not required that you choose to use the redundancy. You could treat the two ethernets in the CNA just like those in an Oplin card and have the OS do teaming. One advantage of this in the half-width blade, is, once again, the ability to do teaming across two different adapters.

3-7 UCSI Boot Camp Sales

© Cisco Systems, Inc.

The fiber channel adapters presented to the BIOS and OS inside the server with the CNA adapter card are standard QLogic and Emulex adapters. The Menlo ASIC directs fc0 two one IOM and fc1 to the other. Any redundancy is achieved through OS multipathing. Once agai, in the full-width blade you can do multipathing across two different adapters.

© Cisco Systems, Inc.

Introduction to UCS Connectivity and Networking 3-8

Adapter Connectivity Details: Palo

Eth0

To IOM 1

Eth ? HBA0

To IOM 2

Fail Over done by Palo for Eths only © 2009 Cisco Systems Inc. All rights reserved.

HBA ?

1.

Appear as HW Ethernet and HBA to OS

2.

Total # of vNICS (HBA +Eth) Determined by # of uplinks

UCS Technical Training — Networking 8

Adapter Connectivity: Palo The Palo virtual adapter creates virtualized ethernet NIC’s (“vnics”) and fibre channel adapters (“vhba’s”) that are seen by the blade server BIOS and OS as real, physical adapters. The virtual adapters are created “on demand” by specification of a service profile. Each virtualized adapter can be defined to go to one IOM (fabric interconnect) or the other, and you can specify for each one whether you want to UCS to provide failover (in the same style as the CNA). While the virtual card theoretically supports 128 virtual adapters, at the time of writing there is a limit of 58 supported, per Palo card (combination of ethernet and fibre virtual nics/hbas).

3-9 UCSI Boot Camp Sales

© Cisco Systems, Inc.

UCS Connectivity Summary Management Network Fabric Interconnect 1

Fabric Interconnect 2 Eth0

Eth0 L1 and L2

NXOS

NXOS

UCSM Eth5

Blade 8

CARD Blade 1

BMC MEZZ

CARD CMC

Chassis Management Switch

MEZZ

MEZZ

I/O Mux

BMC

BMC

I/O Mux

Blade 1

I/O Mux

CARD

Eth5

Chassis Management Switch

Chassis Management Switch

MEZZ

I/O Mux

Chassis Management Switch

BMC

UCSM

CARD CMC

Chassis 1 © 2009 Cisco Systems Inc. All rights reserved.

CMC

Blade 8

CMC

Chassis 2 UCS Technical Training — Networking 9

UCS Management Connectivity Note how the same downlinks between fabric interconnect and fex provide management connectivity alongside the data connectivity. These links are the only place in UCS where data and management traffic are multiplexed on the same physical links. Within the UCS IOM (fex), a chassis management switch assumes control of all management traffic, and directs it to the BMC (service processor) for each physical blade server. The management connectivity beyond the IOM is all completely dedicated ; dedicated links connect to a dedicated BMC port on the blade. The BMC management is completely out of band with respect to blade server actual operations.

UCS Connectivity Summary While this is a bit much at first glance, the fabric interconnect connectivity is established In the following way:

© Cisco Systems, Inc.

Introduction to UCS Connectivity and Networking 3-10

• Uplinks from IOMs to the chassis carry both traffic for the logical servers (LAN and SAN) and all management traffic internal to the UCS • Traffic (Both logical server and management) is multiplexed and de-multiplexed in the IOM by the I/O mux • Logical server traffic (LAN and SAN) are sent from the I/O mux to the mezz card of a blade • Management traffic is sent from I/O mux to the chassis management switch in the IOM • Management traffic can go from the chassis management switch to the CMC • CMC have separate traces through mid-plane to SEEPROM Details about the types of traffic are listed below. Fabric A/B NXOS Networks These networks provide connectivity between the NXOS instance in each IOM, the fabric A/B CMCs, and the management entities on MEZZ_CARDs (and potentially hosts in the future). The network traffic consists of L2 protocols such as the ones used to bootstrap the IOM/IO_MODULE relationship and the VIC protocol used to support Network Interface Virtualization. Frames on this network are Tagged in IOM where they take on a VLAN ID of 4042. Fabric A/B UCSM Adapter Networks These networks provide connectivity between the UCS management software instance in IOM and the management entities on MEZZ_CARDs for the purposes of allocating resources, defining identities, and monitoring status of the adapter. L2 protocols are run with both UCS management software for general adapter functionality, and L3 (IP) is used for remote login to adapter OS’s. Frames on this network are explicitly tagged with a VLAN ID of 4043 on the adapter and the tagging is carried through IOM. IP addresses in this network are algorithmically determined by UCS management software and passed to the adapter during bootstrapping. Fabric A/B UCSM Infrastructure Networks These networks provide UCS wide connectivity between the UCS management software instance in IOM, the fabric A/B CMC, and all BMCs. IOM bootstraps the process by passing out chassis numbers to each CMC as they are discovered and integrated into the system. From there, each element derives it’s own address(es) algorithmically. The interfaces in the CMC and BMC are exposed as virtual interfaces tagged with VLAN 4044, and this VLAN ID is carried throughout each fabric. Due to the way this network is connected, traffic between a CMC and the BMCs in the UCS management software chassis are forwarded via IOM. This nuance exists so that the CMC IP interface can be brought up early in the chassis association process to facilitate CMC image download, if required. IOM switches are statically assigned to be the head of fabric A or fabric B during UCS management software cluster configuration. This configuration is stored persistently… it is not algorithmically or dynamically derived. Fabric A/B UCS management software PXE Networks

3-11 UCSI Boot Camp Sales

© Cisco Systems, Inc.

These networks are isolated networks that are used to boot host operating systems from images hosted by UCS management software. IP addresses in this set are passed out through DHCP as part of the PXE process. Frames from the host are Tagged, with a default VLAN assignment of 4047 from the logical interface which is carried throughout IOM.

© Cisco Systems, Inc.

Introduction to UCS Connectivity and Networking 3-12

VNLink and VNtag

© 2008 Nuova, Inc. All rights reserved.

ICNX5 v1.0—10

This section is intended to disambiguate VNLink and VNtag, and show how they are used in UCS

3-13 UCSI Boot Camp Sales

© Cisco Systems, Inc.

What is VNLink Marketing term (not a specific technology) “Manage Virtual Networking at DC Level” One Solution: N1Kv Virtual supervisor for all VM across DC Does not unify virtualized/non-virtualized net Other solution: VnTag and Palo © 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training — Networking 11

What is VNLink VNLink is the easiest terminology to discuss, because it is really just a marketing term and doesn’t refer to any technology specifically. VNLink features are those features able to expose virtual machine networking directly to the network administrator, across the data center or across a large domain. Nexus 1000v is one solution to VNLink. It prevents a virtual supervisor that looks just like a N5000 controlling all of the virtual machines in N1Kv enabled ESX instances across your entire datacenter. The network administrator is able to set policies and control access, VLAN’s, acl’s on all virtual machines using a familiar, unified switching interface. Note that the N1KV solution does not really integrate or unify switching amount virtual machines and non-virtual machines; it is for virtual machine networking specifically.

© Cisco Systems, Inc.

Introduction to UCS Connectivity and Networking 3-14

VNtag: Bridges and Interface Virtualizers VNtag-capable switch

Int Virtualizer

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training — Networking 12

VNtag: Bridges and Interface Virtualizers VNtag is not specifically a solution for virtual machines. Instead it is mean to be able to support an object with can multiplex ports out of actual switches without themselves actually being switches (sound familiar) These objects are called “interface virtualizers” because their downstream interfaces are virtualized as ports on the upstream, VNtag capable switch. The “virtual” part has nothing to do with being connected to a virtual machine, it has to do with a satellite port on an IV being virtualized upstream.

3-15 UCSI Boot Camp Sales

© Cisco Systems, Inc.

VNTag: Virtualizes Ports on Controlling Switch Sound familiar? N2K Ports Virtualized on 5K (E100/1/1) UCS IOM Ports Virtualized on 6100 (E1/1/1) VnTag has nothing to do with Virtual Machines

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training — Networking 13

VNTag: Virtualizes Ports on Controlling Switch This concept should be familiar. N2K, doesn’t act like a switch. It’s ports are virtualized and appear as “E100/1/x”, etc, on an N5K. It’s a physical port on the fex, but a virtualized port on the N5K Of course the UCS IOM and the UCS 6100 have the exact same relationship (IOM inward facing physical ports to each blade are virtualized as E/1/<slot> on UCS6100

© Cisco Systems, Inc.

Introduction to UCS Connectivity and Networking 3-16

An Interface Virtualizer Eg N2K, UCS IO Module table of tags(directly indexed) 12: satellite port 1 13: satellite port 2 NO MAC tables!!! (period!) src tag added

dst tag removed

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training — Networking 14

An Interface Virtualizer An IV doesn’t have MAC tables (that is the whole point, they are too slow and involve brute force lookup). Instead a tag (VNtag) represents each satellite port. The IV adds the src tag on ingress (it could never have a dst tag at this point, since that is inserted by the controlling switch (next slide). For outgoing packets, the IV maintains a directly indexed table of tags mapping to outbound ports. The IV then removes the tag on packet egress. The station attached to the remote port is not VNtag aware.

3-17 UCSI Boot Camp Sales

© Cisco Systems, Inc.

A VNTag Aware Switch Eg N5K, UCS 6100 VNtag-capable switch virt.port

table of tags(directly indexed)

virt.port

12: satellite port 1 13: satellite port 2

virt.port

NO MAC tables!!! (period!) src tag added

virt.port

•MAC switching on vports •Ingress/Egress same phys port OK •Learn tags for each virt port •Remove src tags/ add dst tags •Tagging ABSOLUTELY transparent

© 2009 Cisco Systems Inc. All rights reserved.

dst tag removed

UCS Technical Training — Networking 15

A VNTag Aware Switch This switch is also called the “controlling bridge”, it is where the satellite ports on the IV device are virtualized. All MAC switching is done between these virtual ports. While the same packet might enter the physical downlink to the IV and exit the same physical port, we are not violating the cardinal rule of switching because the phyiscal port is not the switch port, the virtual port is! A destination virtual port is chosen based on standard MAC tables (per virtual port). The N5K then inserts the destination VNTag associated with the destination virtual port to direct traffic through the IV.

© Cisco Systems, Inc.

Introduction to UCS Connectivity and Networking 3-18

VNTag Format

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training — Networking 16

VNTag Format The illustration shows the VNTag inserted into an Ethernet frame. A tagged frame is represented by a distinct ethertype, which will only be intelligible to VNTag-capable switches and IV’s. The tag is divided into the source and destination areas.

3-19 UCSI Boot Camp Sales

© Cisco Systems, Inc.

Daisy-chaining Interface Virtualizers Imagine Daisy-Chaining Fexes/IOM? YES! All interfaces virtualized on top switch Tag represents OUTERMOST satellite part Intermediate fex-like device: Table of all tags Only adds/strips its “own” tags © 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training — Networking 17

Daisy-chaining IV’s IV’s can easily be daisy-chained (N5K does not support daisy-chained, 2K’s, but the concept is eaisly supported). All satellite ports on all levels of daisy-chained IOV are virtuallized at the top-level VNtag-capable switch. The tag value on an outbound packet represents FINAL DESTINATION satellite port. There is no daisy chaining of tags. An IV which receives an already tagged packet from another IV (inbound toward the switch); (a) leaves the src tag alone (b) adds the src tag to its tables An IV which receives a tagged packet outbound: (a) uses the dst tag as usual (b) recognizes the dst tag as “not its own”, and does NOT strip it.

© Cisco Systems, Inc.

Introduction to UCS Connectivity and Networking 3-20

Menlo/Palo Just Daisy-Chained IV’s!! Palo just like an IOM with virtual satellite ports

Eth0

To IOM 1

Eth ? HBA0

To IOM 2

HBA ?

•

Ports tagged/untagged just like fex ports

•

Appear as virtual ports on top-level bridge (6100)

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training — Networking 18

Menlo/Palo Just Daisy-Chained IOV’s!! What does this have to do with UCS. Ah --- Menlo/Palo cards are just daisy-chained interface virtualizers. Menlo virtualizes its two physical interfaces as virtual ports on the 6100. it is not connected directly, it is daisy-chained through another IV, the UCS IOM (which as an IV knows exactly what to do with this daisy-chaining). Palo creates “virtual interfaces” (virtualized hardware accessible to the blade on the PCIe bus). The VNtag part has nothing do do with the fact that these interfaces happen to be virtual. It would work the same way if we invented some card that supported 100 physical blade-facing interfaces. The VNTag part just virtualizes these interfaces on ports through the controlling 6100. The Palo is connected to 6100 via daisy-chaining through the intervening IV, the UCS IOM.

3-21 UCSI Boot Camp Sales

© Cisco Systems, Inc.

What does it all have to do with Virtual Machines? Vntag part: nothing Now connect each Palo Port to Virt Machine Now: VN-Link! (VM-specific ports on 6100) Eth0

To IOM 1

VM1 Eth ?

VM2 VM3

HBA0

© 2009 Cisco Systems Inc. All rights reserved.

VM4

HBA ?

To IOM 2

•

Ports tagged/untagged just like fex ports

•

Appear as virtual ports on top-level bridge (6100) UCS Technical Training — Networking 19

What does it all have to do with Virtual Machines? How does this all relate to a VNLink feature? The VNtag part has nothing to do with it directly. However, we know that the Palo virtual interfaces are represented as virtual ports on the 6100, and managed by the network administrator across the entire UCS domain exactly as physical machine network ports are managed. The final glue is the Palo pass-thru switch in ESX, which is able to dedicate the Palo virtualized adapters (which are virtualized as ports on the 6100) to specific VM’s. voila! VN-link --- VM-specific networking is now controlled alongside and indistinguishable from physical machine network across the UCS domain.

© Cisco Systems, Inc.

Introduction to UCS Connectivity and Networking 3-22

Nexus 1000v versus Cisco M81KR Characteristics

Nexus 1000v

Cisco UCS M81KR Virtual Interface Card

Purpose

Integrate all VM switching into a single supervisor managed by the network admin

Integrate VM networking so network configuration for VM is indistinguishable from that of a physical machine in UCSM

Port Profiles

Manage port configurations through port profiles • Created on VSM • Enforced on VEM

Manage port configurations through port profiles • Created in UCSM (VM Tab) • Enforced by UCSM

Local Switching

Switched by VEM Locally

Switch By UCS 6120

QoS

Configured and Managed by VSM

QoS Policies in UCSM applied to vNIC in service profile

Network Features

• PVLAN / VLAN • QoS • ACL

Supported Environments

VSM can be a VM on the ESX Host or can run on a x86 appliance

UCS only

Implementation

In software with kernel integration to the hypervisor.

Implemented through HW

• VPC-HM • ERSPAN / SPAN • Netflow

• QoS • VLAN • Mac-Forging

© 2009 Cisco Systems Inc. All rights reserved.

• Pass Through Switch (Hypervisor Bypass

UCS Technical Training — Networking 20

Comparing Nexus 1000v with the Cisco M81KR There is some confusion that develops when talking about both the Nexus 1000v and the UCS using the Cisco M81KR virtualized adapter. This typically results from the fact that both of these products fall under the umbrella of VNlink technologies, and ultimately they share many of the same feature sets as seen in the table in the slide. Despite this they are very different solutions and are implemented in very different ways. The main purpose of both of these products is to be able to integrate all your VM networking and management thereof through a single switch managed by the network admin. With the Nexus 1000v this is implemented through software that is installed on either an ESX host or a standalone x86 appliance and is used to managed networking for VMs only. With the M81KR, this is implemented through hardware and the integration with the UCS 6120/UCSM, and as such management is centralized but unlike the Nexus 1000v you also manage you physical networking as well through this management source. In the table above we have the characteristics of each solution listed and compared. It should be noted that these products are not an either/or solution but a solution that can be implemented separately or in conjunction with one another. In other words you could run a ESX host on a blade in a UCS containing a M81KR adapter. You can then install the 3-23 UCSI Boot Camp Sales

© Cisco Systems, Inc.

Nexus 1KV on that same ESX host and use it for VM on there. Additionally on that same host you could through the UCSM create vNIC that use the PTS capability designed into the M81KR. This flexibility provides a lot of power and usefulness to both products.

© Cisco Systems, Inc.

Introduction to UCS Connectivity and Networking 3-24

BMC Management Services

© 2008 Nuova, Inc. All rights reserved.

3-25 UCSI Boot Camp Sales

ICNX5 v1.0—10

© Cisco Systems, Inc.

BMC External IP Addresses

•

•

Blade mgmt service to external client •

IP’s on ext mgmt network

•

NAT’d by 6120

Service to ext clients: • • •

© 2009 Cisco Systems Inc. All rights reserved.

KVM / Virtual media IPMI Serial Over LAN (SOL)

UCS Technical Training — Networking 11

BMC External IP Addresses The BMC for each blade server is assigned by the UCS manager an external IP address, to provide blade management services to clients running externally to UCS. These are the only IP addresses managed by UCS Manager itself (UCS Manager does not currently manage any layer 3 services whatsoever on behalf of the OS on blade servers). The BMC external IP addresses are dedicated, one per blade server. The addresses are not actually configured or known to the BMC itself; instead they are NAT’d through the fabric interconnect management port. (Note, in a dual-fabric scenario, both fabric interconnects will handle a portion of the BMC external IP addresses, assuming both interconnects are alive). At the time of writing there are several services provided by the BMC using these externally visible IP addresses to an external client. We will discuss the KVM/virtual media service on the next few slides, and the IPMI and Serial-over-LAN services later in the course.

© Cisco Systems, Inc.

Introduction to UCS Connectivity and Networking 3-26

Server External Mgmt Addresses

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training — Networking 12

Server External Mgmt Addresses In order to be able to utilize special server communication services, you will need to setup a management IP address for each blade in the UCS system. Fortunately this is very easy to do. Under the Admin in the Nav Pane if you expand the Communication Services Folder you can select the Management IP Pool object. Now in the Actions Menu in the Content Pane you can select “Create a Block of IP Addresses” which will allow you to create a management IP pool. You simply state the beginning address of the range, the number of IP addresses you wish in the pool, the netmask, and the default gateway. Once complete these addresses will be assigned automatically to newly discovered blades. The easiest way to do this is to use the same subnet as the management IP assigned to the UCS Management Nodes as the management ports are used to provide access to the blades for specific services. You can of course use any IP address scheme, but you will need to make sure that on the management network the router that the nodes attach to are aware of this. The following services are available if you create this management pool and blades get IP addresses assigned to then:

3-27 UCSI Boot Camp Sales

© Cisco Systems, Inc.

• Virtual KVM access over IP – On as soon as you apply an management IP address • Serial Over LAN (SoL) – Available only if configured in a service profile • IPMI – Available only if configured in the service profile

© Cisco Systems, Inc.

Introduction to UCS Connectivity and Networking 3-28

KVM

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training — Networking 13

KVM You can access KVM console by going to the Nav Pane, selecting a server of your choice, and then from the Actions Menu in the Content Pane selecting “KVM Console”. This will give you a KVM window with “No Signal” because the blade in this example is not associated with a service profile and booted. You also would be able to access this KVM console from the Service Profile view as well, assuming one has been associated to the blade. The KVM Console is a Video-over-IP representation of the video output on the blade. The service is provided by the blade’s BMC via the external IP address only. Earlier in the module, you verified this BMC IP address. You do not need to specify it to bring up the KVM Console, but it needs to be up and contactable from the system on which you are running the UCS GUI. The KVM is also provided as as standalone application, although it still authenticates through the UCS Manager. There are shutdown / boot / reset buttons on the KVM GUI. This is useful for server administrators who may be given access to the standalone KVM application but not have any ability to actually log in to the UCS manager GUI itself. 3-29 UCSI Boot Camp Sales

© Cisco Systems, Inc.

KVM Console Virtual Media

ISO/IMG

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training — Networking 14

KVM Console Virtual Media The KVM console provides a sub-utility which allows you to use a physical CD/DVD or an .iso file on the client (on the machine on which you are running the GUI and KVM Console) which presents itself to the blade as Virtual CD medium. The blade can boot off of this medium for installations (or for later emergency repairs). Note that if you are using VMware on the blade, the UCS KVM console virtual media is used by the blade itself ( for example, for installing ESX itself), not for the guest machines. The guest machines have their own Virtual Console / Virtual media methods just like they do in any ESX installation. • Single Virtual CD/DVD on blade, mapped to: • Client (PC running KVM) ISO file • Client physical CD/DVD medium • Single Virtual floppy on blade, mapped to:

© Cisco Systems, Inc.

Introduction to UCS Connectivity and Networking 3-30

• Client IMG file • Client physical/virtual floppy medium

• Single Virtual removable HD on blade, mapped to: • Client IMG file • Client physical removable medium (USB stick)

3-31 UCSI Boot Camp Sales

© Cisco Systems, Inc.

Virtual Media Manager

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training — Networking 15

Virtual Media Manager The screenshot shows how you invoke the Virtual Media manager from the KVM Console. In this example, we are going to use a .iso file on the client as virtual media for the blade…

© Cisco Systems, Inc.

Introduction to UCS Connectivity and Networking 3-32

Blade Server and IOM Dongles

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training — Networking 16

Blade Server and IOM Dongles The blade server dongle gives direct access to the video, USB, and serial port of a physical blade. The dongle is often just a “backup” or “crash-cart” blade access mechanism, since serial, video, and USB access can be provided over the network using the BMC services. There is typically no advantage of using the physical video dongle; you will see the exact same video rendered as you would using the remote KVM service. One possible advantage of the dongle, even when everything is healthy and you have access to all BMC services, could be the desire to directly attach a USB DVD player for more optimal access to an entire install medium.

3-33 UCSI Boot Camp Sales

© Cisco Systems, Inc.

UCS Networking Introduction

© 2008 Nuova, Inc. All rights reserved.

© Cisco Systems, Inc.

ICNX5 v1.0—17

Introduction to UCS Connectivity and Networking 3-34

VLAN Objects in UCS

Note VLAN “Name” Abstracts numerical configuration from server.

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training — Networking 18

VLAN Objects in UCS Network configuration in the UCS environment is distinct from any other switching environment. All configuration is done using managed objects in the UCS model, and switch ports are automatically configured to match the model. There could be a temptation to want to configure UCS 6100 as a traditional switch, although the theory of treating the entire UCS system as a unified, managed environment using its own data model has huge advantages. At the center of ethernet configuration is the VLAN object. A VLAN object combines a traditional VLAN number with a name. The “extra level of indirection” required by VLAN names is intentional. Server-bound VLAN configuration (through vNIC, as seen later in the module) is done using VLAN names rather than numbers. This allows a great amount of flexibility --- VLAN objects can be deleted and recreated with the same name and a different number without modifying server configurations. All servers using the named VLAN will automatically be connected to the new number. Creation of VLAN objects automatically allows the VLANs on northbound ports from the UCS 6100 fabric interconnect. There is no separate port-for-port configuration required for northbound ethernet.

3-35 UCSI Boot Camp Sales

© Cisco Systems, Inc.

The screenshot shows 3 configured VLAN objects. VLAN “default(1)” preexists in the configuration and cannot be deleted (or renamed or renumbered) . As other VLANs get created, they are automatically allowed northbound. You could connect to the nxos shell on the command line and run a command such as “show interface e1/1” (matching the number of an ethernet uplink) to demonstrate to yourself how new VLAN numbers are automatically allowed northbound upon VLAN object creation. By default VLAN “default(1)” is the native VLAN northbound. You can change this to a different VLAN. As discussed later, the configuration will be the same across all uplinks on the same fabric interconnect and typically the same across both in a redundant configuration.

© Cisco Systems, Inc.

Introduction to UCS Connectivity and Networking 3-36

Northbound Networking

VLANs: Native – 1 Allowed – 1, 39, 214

VLANs: Native – 1 Allowed – 1, 39, 214

L2-Cluster

L1-Cluster

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training — Networking 19

Northbound Networking The drawing summarizes northbound VLANs in a redundant switch scenario. The native VLAN (1) has not been changed from the default. All VLANs corresponding to other VLAN objects are allowed across all multiple VLANs.

3-37 UCSI Boot Camp Sales

© Cisco Systems, Inc.

Northbound Networking

VLANs: Native – 1 Allowed – 1, 39, 214

VLANs: Native – 1 Allowed – 1, 39, 214

L2-Cluster

L1-Cluster

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training — Networking 20

Northbound Networking The drawing summarizes northbound VLANs in a redundant switch scenario. The native VLAN (1) has not been changed from the default. All VLANs corresponding to other VLAN objects are allowed across all multiple VLANs.

© Cisco Systems, Inc.

Introduction to UCS Connectivity and Networking 3-38

Northbound Networking Details • All uplink same VLAN s • Changeable Native VLAN (NB) • You can have different VLANs on other 6120 • Per-switch VLAN • Not recommended L2-Cluster

L1-Cluster

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training — Networking 21

Northbound Networking Details Identical VLAN configuration across all uplinks on same switch is required and enforced by the model (you have no mechanism of violating this model “manually”). Typically you will have identical configuration on switches A and B in a redundant switch configuration. Besides management redundancy, the whole point of the dualswitch scenario is path redundancy, all the way from the servers through the different switches northbound. It is possible to have different VLANs on switches A and B (but not on ports of the same switch). In order to accomplish this, you can create “switch-specific” VLAN objects rather than the more typical “dual-switch” VLAN objects. Switch-specific VLANs will affect the uplinks on that switch only. Keep in mind the following : • All uplink ports on same fabric interconnect (A or B) must have same VLAN configuation (native/allowed) • Could change native VLAN northbound • Usually have same config. across A and B 3-39 UCSI Boot Camp Sales

© Cisco Systems, Inc.

• Possible to have different VLANs on uplinks on different fabric interconnects • Per-switch VLANs rather than typical “dual” VLAN • Less likely since whole point of A-B is high-availability of server connectivity

© Cisco Systems, Inc.

Introduction to UCS Connectivity and Networking 3-40

Creating New VLAN

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training — Networking 22

Creating New VLAN All that is required to create a VLAN is the name and number ,and whether it will be be a typical dual-switch VLAN (“Common/Global”) or a rarer, switch-specific VLAN.

3-41 UCSI Boot Camp Sales

© Cisco Systems, Inc.

Northbound: Port Channels  Just choose ports (same switch only)  Automatically uses LACP  PC then behaves like physical port

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training — Networking 23

Northbound: Port Channels Port Channels can be created for northbound ethernet traffic. On UCS Manager, all you have to do is choose the ports for the port channel and give the port channel a number. You do not have any choice about protocols or VLANs. The UCS manager always configures the port channel to use LACP, the Link Aggregation Control Protocol. As discussed on a subsequent page, northbound configuration (on the next switch up) will have to match (ports and protocol). LACP is not a port-channel discovery protocol, just a port-channel control protocol. Once port channels are created, they are treated like individual uplink ports in all respects: • •

they will match allowed and native VLANs with all other ports and port channels on same switch (and usually across switchs) they will be available for pinning server traffic (discussed later) just like an individual port

© Cisco Systems, Inc.

Introduction to UCS Connectivity and Networking 3-42

Creating Port Channels

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training — Networking 24

Creating Port Channels The screenshots demonstrate how creating a northbound port channel is simply a matter of selecting the ports. (the harder part is making sure the configuration matches on the northbound switch. The actual implementation of this depends on your switch, but whatever it is it must support LACP).

3-43 UCSI Boot Camp Sales

© Cisco Systems, Inc.

Northbound Networking with Port Channels VLANs: Native – 1 Allowed – 1, 39, 214

VLANs: Native – 1 Allowed – 1, 39, 214

L2-Cluster

L1-Cluster

VLANs: Native – 1 Allowed – 1, 39, 214

VLANs: Native – 1 Allowed – 1, 39, 214

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training — Networking 25

Northbound Networking with Port Channels Port channels are now added to the same illustration that we had before. Once again, in this typical example, VLAN configuration is identical across both switches. The drawing illustrates individual uplinks on each switch and two ports configured into a port channel on each switch.

© Cisco Systems, Inc.

Introduction to UCS Connectivity and Networking 3-44

Matching Configurations on Switch North of UCS Regular Ports – Match native VLAN – Match allowed VLANs

Port Channels – Match connected ports – May have vPC northbound (transparent to 6100)

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training — Networking 26

Matching Configurations on Switch North of UCS The following summarizes all the matching ethernet configuration required on the northbound ethernet switch(es): •

•

•

Matching native VLANs: this will actually work if the native VLAN northbound from UCS has a mismatch with the native VLAN toward UCS on the northbound switch, but having a mismatch would be strange. Since the traffic is untagged, it would still work, but you would be “taking a mental translation” from the native VLAN number southbound into UCS to the native northbound VLAN within UCS. This is likely unhealthy and not a good practice from a management standpoint. It is much easier to match the native VLANs. Matching allowed VLANs. The deal is the only VLANs that will work across the uplink are the intersection of allowed VLANs on the endpoint ports. It is legal to have more or fewer VLANs allowed on the northbound switch. Fewer VLANs allowed northbound will cause VLANs which “should work” in UCS drop into the bit bucket. More VLANs allowed northbound will appear not to exist in UCS, since all configuration in UCS is done through the VLAN objects. Port Channels: ports much match, and LACP must be enabled on the port channel northbound to match the automatic configuration in UCS.

3-45 UCSI Boot Camp Sales

© Cisco Systems, Inc.

Inbound Networking (to servers) Controlled by vNIC object in Service Profile Applied automatically as SP is associated Different way of thinking of things

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training — Networking 27

Inbound Networking (to servers) Inbound networking to servers is all specified through the vNIC object in a service profile. Note this means all connectivity into a server is specified as properties of the “theoretical server” represented by a service profile. When this service profile is then applied to a physical blade server, the fabric ports are automatically reconfigured to direct the proper VLAN traffic to the server. This is a different mode of thinking for many, who object that it is “easier to configure the fabric switch ports directly”. However, if you think about it, this administrative model more closely maps to the actual requirements. Actual VLAN requirements are on servers, not on ports.

© Cisco Systems, Inc.

Introduction to UCS Connectivity and Networking 3-46

vNIC in Service Profile

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training — Networking 28

vNIC in Service Profile The vNIC object represents a “theoretical or virtual specification” for a single nic port in a server. When a service profile containing a vNIC is applied to a blade server with physical NIC hardware (Menlo or Oplin card), the properties specified by the vNIC are applied to the physical adpater (and connectivity through the fabric directed to that physically adapter). The “v” in vNIC in this case should be thought of as “virtual specification for a NIC”. When a vNIC is applied to the virtual card, the story is slightly different. The presence of the service profile vNIC causes the virtual card to manufacture the corresponding virtualized hardware – and then the vNIC properties are applied to it as if it were a physical device. The illustration shows two vNICs in a single service profile, corresponding to specification for two physical or virtualized nic adapters on whatever blade the service profile happens to be associated with.

3-47 UCSI Boot Camp Sales

© Cisco Systems, Inc.

The properties of the vNIC specify its connectivity (A or B switch, with or without failover), VLANs, and other adapter properties and Qos (really Class of Service) parameters.

© Cisco Systems, Inc.

Introduction to UCS Connectivity and Networking 3-48

Inbound VLANs (for vNIC)

Vlans: Native 39 Allowed 1, 39, 214

 VLANs same as northbound links  vNICs can connect to more than 1 VLANs

Eth0 To IOM 1

 vNICs can have own native VLAN – Not required to have native VLAN (0 or 1 VLAN native)

To IOM 2

Eth1

– Unrelated to native VLAN on uplinks

Vlans: Native 1 Allowed 1, 39, 214

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training — Networking 29

Inbound VLANs (for vNIC) VLANs available for connectivity into blade servers via the vNICs in server profiles are the same set that are allowed northbound (via creation of VLAN objects). If you have only “dual” VLANs (same configuration on both switches) you will be less confused. If you have single-switch VLANs, then these will also be available only to non-failover vNICs specified with connectivity to that particular switch. Any vNIC can be connected to any number of VLANs, zero or one of which can be native (untagged down to the server adapter). Native VLAN into a server is completely unrelated to the native VLAN on northbound links. The UCS fabric logic will automatically make the tagging/untagging transitions. Any non-native VLANs specified for a server (via its vNIC) must be handled by a tagging-aware device driver in the OS.

3-49 UCSI Boot Camp Sales

© Cisco Systems, Inc.

Connecting Northbound and Inbound Ethernet Traffic: End-Host Mode Default is UCS 6100 is in end-host mode No MAC address-tables maintained northbound UCS 6100 pins northbound Can change to switching mode (Not Recommended)

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training — Networking 30

Connecting Northbound and Inbound Ethernet Traffic: End-Host Mode By default the UCS6100 is set up in “end host mode”. In this mode, UCS 6100 does not act like a traditional switch on the northbound ports. From the point of view of a switch upwards from UCS, the UCS 6100 will appear as an end-host that happens to be forwarding a large variety of MAC addresses. There is no need to run Spanning Tree Protocol (and UCS will not in end-host mode) and no chance that UCS can be the cause of layer-2 loops. Traffic is directed from UCS server to particular uplinks by a pinning process that is described on the next few slides. This pinning process never takes account of the destination (northbound) MAC address; there are no MAC tables maintained northbound. Return traffic to the UCS server is naturally just directed down the same pinned link, since that is the only link on which the northbound switch would have learned the MAC address for that particular server.

© Cisco Systems, Inc.

Introduction to UCS Connectivity and Networking 3-50

UCS 6100 does (always) act like a switch inbound toward the servers, for the purpose of server-server communication in the same UCS system, or inbound packets from a northbound source into a UCS server. There is an option to change the mode of the entire UCS switch to a traditional switching mode. This would allow direct connection of file servers and the like on UCS northbound ports. In the absence of such a need, it is highly recommended to leave UCS in end-host mode, for simplicity’s sake.

3-51 UCSI Boot Camp Sales

© Cisco Systems, Inc.

Automatic Pinning

vNIC in SP (going to switch A) pinned to uplink port or channel repinned on uplink failure

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training — Networking 31

Automatic Pinning Northbound traffic form servers is pinned to a specific uplink port or port channel, on a vNIC by vNIC basis. If you do not specify pinning, the pinning is chosen automatically by UCS. If there are multiple uplinks from a switch, UCS will assign pinning to the uplinks in a round-robin fashion for each vNIC using automatic pinning. Note that the choice of uplink on a particular switch is never based on VLAN, since all uplinks on the same UCS switch must support the same VLANs The drawing illustrates automatic pinning of server traffic specified by a particular service profile vNIC to a particular uplink. In this (automatic) pinning scenario, if the uplink chosen for pinning fails, the vNIC traffic (and that of all vNIC that UCS has chosen to pin to the uplink) will fail over to the next uplink on the same switch.

© Cisco Systems, Inc.

Introduction to UCS Connectivity and Networking 3-52

Static (Manual Pinning) You create Pin Group – 1 port or port channel on one switch – 1 port or port channel on each switch (for failover) – Used only for static (manual) pinning

You then manually specify vNIC pinning in vNIC Config

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training — Networking 32

Static (Manual Pinning) If you choose static or manual pinning, you must first create a Pin Group object to represent the choice of a single uplink or port channel on one switches (or one on each switch for a vNIC with failover). The Pin Group object by itself doesn’t do anything until it is referenced by a particular vNIC. This vNIC traffic will then be directed only to that one uplink port or port channel on that switch. However, a vNIC for which A-B (or B-A) failover is specified can refer to a pin group with one uplink port or port channel on each switch. If there is failure on the chosen uplink on one switch, traffic from that vNIC will automatically fail over to the single chosen port or port channel on the other switch. Traffic may require pinning if you had uplinks that were actually attached to different layer 2 fabrics northbound. Your traffic from your server would not be directed correctly in this case unless you pinned the traffic to the correct uplink. Even when (as is recommended) all uplinks go northbound to the same layer 2 fabric; you may want to choose static pinning in order to have more control over the traffic on 3-53 UCSI Boot Camp Sales

© Cisco Systems, Inc.

the uplinks. You may know you have one vNIC with heavy traffic that you want to pin to a dedicated uplink, while pinning combinations of other vNICs to different uplinks.

© Cisco Systems, Inc.

Introduction to UCS Connectivity and Networking 3-54

Pin Group Creation

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training — Networking 33

Pin Group Creation The screenshot shows the specification of a Pin Group. It happens to reference a single port channel on switch A and a single physical port on switch B, which may be unlikely but still works. The Pin Group is now available for binding form a vNIC.

3-55 UCSI Boot Camp Sales

© Cisco Systems, Inc.

Static (Manual Pinning) Pin Group: ZippythePinGroup

vNIC in SP (going to switch A) pinned to port/port channel on A can use no other uplink on A can failover to B © 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training — Networking 34

Static (Manual Pinning) The drawing illustrates the manual/static pinning scenario. The vNIC is manually pinned to the Pin Group named ZippythePinGroup. Traffic from the vNIC is directly only to the single port channel on switch A, but can fail over to the single port (in this example) on switch B

© Cisco Systems, Inc.

Introduction to UCS Connectivity and Networking 3-56

vNIC screen in Service Profile Wizard identity (MAC)

connectivity and VLANs

Pin Group (not set = auto pinning)

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training — Networking 35

vNIC screen in Service Profile Wizard The screenshot shows the vNIC creation screen which is part of the Service Profile creation wizard. This is presented here just to summarize the properties of vNIC which are used to direct ethernet traffic to the UCS server. The vNIC specifies a virtualized identity (MAC address). We will see more in later modules how this could just be specified to use the physical nic identity, could be an individual virtualized identity (your birthday or whatever), or could be a virtualized identity taken from a pool. Note the single VLAN can be connected to any of the VLANs supported on the switch, as specified by the same VLAN objects used to configure the switch uplinks. Exactly zero or one of the VLANs can be native (untagged down to the server). The final part of the screenshot shown above demonstrates where you would specify the choice of automatic pinning (pin group not set) or static/manual pinning (via choice of a Pin Group).

3-57 UCSI Boot Camp Sales

© Cisco Systems, Inc.

UCS SAN Introduction

© 2008 Nuova, Inc. All rights reserved.

© Cisco Systems, Inc.

ICNX5 v1.0—36

Introduction to UCS Connectivity and Networking 3-58

SAN Concepts Similarities to Ethernet – VSAN objects tie NB-traffic to server – vHBA object is fc port in SP – No switching northbound  Automatic  Manual pinning

Differences (simpler) – Each uplink and vHBA on only one VSAN – vHBA goes to IOM A or B – No failover) – HA through OS multipathing on multiple vHBA © 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training — Networking 37

SAN Concepts SAN connectivity concepts in UCS are similar to LAN concepts and analogous to them in many cases, although SAN is typically simpler. VSAN objects represent the choices of VSAN northbound and down to the server. A vHBA object in a service profile is used to virtualize identity and specify connectivity for an fc port in a blade server to which the service profile is defined. Today, each uplink can be associated with only one SAN, and each vHBA can be associated with only one SAN. Each vHBA specifies connectivity to either switch A or B but not both. UCS does not perform any failover for fiber channel. Instead, it is typical that the OS provide highly available access to external LUNs on the SAN using multipathing device drivers in the OS across pairs of vHBAs

3-59 UCSI Boot Camp Sales

© Cisco Systems, Inc.

VSAN Objects

ID is VSAN Number FCoE VLAN used inbound FCoE not used northbound FCoE VLAN not exposed as VLAN available for vNIC

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training — Networking 38

VSAN Objects Like VLAN objects, VSAN objects have a name and VSAN ID. The name indirection is intentional; vHBAs will refer to the VSAN by name rather than number. Deletion of a VSAN and recreation with the same name but different number will automatically affect traffic for all vHBAs referring to that VSAN name. The FCoE VLAN id is an implementation detail of sorts. Since FC traffic runs over FCOE within UCS only, we typically choose an FCOE VLAN number that does not overlap with a “regular VLAN number”. Since this VLAN will be specific to FCOE, it need not be allowed northbound or exposed as a VLAN that can be connected to a vNIC. It is actually allowed (not necessarily recommended for management sanity) to run regular VLAN traffic and FCOE traffic over the same VLAN number. The default VSAN uses VSAN id 1 and runs over FCOE VLAN Id 1.

© Cisco Systems, Inc.

Introduction to UCS Connectivity and Networking 3-60

VSAN Objects and Northbound FC Ports Northbound FC must run in NPV mode Today each uplink supports only one VSAN

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training — Networking 39

VSAN Objects and Northbound FC Ports Northbound fiber channel uplinks are set automatically (and irrevocably) to run in NPV node. You cannot directly attach a storage array to the UCS uplink ports; you must attach to a fiber switch and enable the NPV feature on that northbound switch (run npiv-enable, for example, on a Cisco MDS switch). You must choose the correct VSAN for each UCS fiber channel uplink. The uplink port will log into the upstream fibre switch regardless, but if the upstream switch is VSANaware, the uplink will not enable itself for use for UCS server fibre channel traffic unless the VSAN setting matches the port configuration northbound.

3-61 UCSI Boot Camp Sales

© Cisco Systems, Inc.

Inbound FC (to servers) Controlled Service Profile 1 vHBA = one FC Port Physical Cards (Menlo-Q and Menlo-E only): – vHBA applied to specific physical fc port – 2 per service profile (if dual fabric) – Physical fc port without vHBA is OS visible not usable

Virtual Card: (Palo) – vNIC spec causes creation of virtualized hardware – Many per service profile (*128)

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training — Networking 40

Inbound FC (to servers) Fiber Channel connections are properly exposed to blade servers via specification of vHBA objects in service profiles, analogous to vNIC objects for LAN. A vHBA represents a single fibre channel port, and is applied to a physical FC port on a Menlo Adapter, or causes creation of a virtualized fibre channel port on Palo. *Note: Theoretical Max

© Cisco Systems, Inc.

Introduction to UCS Connectivity and Networking 3-62

vHBA in Service Profile

WW Node Name (one per Svc Prof)

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training — Networking 41

vHBA in Service Profile The diagram illustrates vHBA properties. Each vHBA as a separate identity – this is the World-wide Port Name. The virtualized World-wide Node Name identity is a property of the service profile as a whole. Each vHBA specifies connectivity to only one switch (with no failover) and one VSAN.

3-63 UCSI Boot Camp Sales

© Cisco Systems, Inc.

Connecting Northbound and Inbound Ethernet Traffic: NPV Mode FC must run in NPV mode UCS 6100 does not switch FC traffic – Each vHBA is pinned to an uplink port or port channel – Automatic pinning  only uplink with matching VSAN  Round-robin assignment

– Manual static pinning (analogous to vNIC with Pin Grp)

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training — Networking 42

Connecting Northbound and Inbound Ethernet Traffic: NPV Mode Since the uplinks are required to run in NPV mode, UCS 6100 cannot itself act as a traditional fibre channel switch. You cannot attach end-storage directly to UCS 6100 fibre channel uplinks. Outbound traffic from a vHBA is directed to a particular uplink via automatic or manual/static pinning. However, all pinning is restricted by the choice of VSAN; vHBA traffic is always mapped to an uplink sharing the same VSAN specification. Manual pinning can be accomplished via the creation of FC Uplink Pin Groups. This is exactly analogous to LAN Pin Groups, except that there is no possibility of failover from switch to switch. Reasons for choosing static pinning are similar for SAN and LAN --- you may want to dedicate a particular uplink on a particular VSAN to a heavily-used vHBA, for example.

© Cisco Systems, Inc.

Introduction to UCS Connectivity and Networking 3-64

vHBA Configuration vHBA identity (WWPN)

single VSAN and optional FC uplink pin group

© 2009 Cisco Systems Inc. All rights reserved.

UCS Technical Training — Networking 43

vHBA Configuration This screenshot shows a piece of the service profile creation wizard that we will use over the next several module. It highlights the identity portion of the vHBA (the WW Port Name), as well as the choice of connectivity, VSAN, and pin group (if any).

3-65 UCSI Boot Camp Sales

© Cisco Systems, Inc.

Lesson 4

UCS Basic Opt In Model Overview This module discusses full usage (installing, booting, accessing) of the UCS blade servers using the basic opt-in model. The intention of this paradigm is to use each blade server as a traditional server, rather than taking advantage of the “service profile mobility” and blade pooling models presented in the next module. One main theme of this module is that the “Service Profile” configuration, whose design is intended to support mobility and pooling, is still required even if you want to boot, install, and use an individual blade as you would use a traditional server. Objectives Upon completion participants should be able to: • Identify the Relationship between blade servers and service profiles •

Review the primary purposes of service profiles

•

Create a simple service profile for a single blade with appropriate vNIC

•

Use the expert wizrd

•

Understand association

Service Profiles and the Basic Opt-in Model

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 1

Using Blade Servers with the Basic Opt-in Model This module discusses full usage (installing, booting, accessing) of the UCS blade servers using the basic opt-in model.. The intention of this paradigm is to use each blade server as a traditional server, rather than taking advantage of the “service profile mobility” and blade pooling models presented in the next module. One main theme of this module is that the “Service Profile” configuration, whose design is intended to support mobility and pooling, is still required even if you want to boot, install, and use an individual blade as you would use a traditional server.

4-1 UCSI Boot Camp

© Cisco Systems, Inc.

Lesson Objectives • Identify the Relationship between blade servers and service profiles • Review the primary purposes of service profiles • Create a simple service profile for a single blade with appropriate vNIC • Use the expert wizrd • Understand association

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 2

In this lesson Students will: • • • • •

Identify the Relationship between blade servers and service profiles Review the primary purposes of service profiles Create a simple service profile for a single blade with appropriate vNIC Use the expert wizrd Understand association

© Cisco Systems, Inc.

UCS Basic Opt In Model4-2

Service Profiles and Blade Servers Service Profile: ls-RH53Oracle Client Server: RH53Oracle

Physical Blade: 5

Identity (MAC Address, WWN, Etc.)

=

Associated to

Behavior (Firmware, QoS, Etc.) Other (vNICs, vHBAs, Etc.)

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 3

Service Profiles and Blade Servers Understanding the service profile, or “logical server” is key to understanding blade management on UCS. The service profile represents a logical view of a single blade server, without needing to know exactly which blade you are talking about. The profile object contains the server personality (identity and network information and the like, as discussed on later slides) and connectivity requirements. The profile can then be associated with a single blade at a time. The concept of profiles was invented to support the notion of mobility --- or transferring of identity transparently from one blade to another, as well as pooling concepts discussed in detail in the next module. Even if you intend to manage the blade server as a traditional individual server (not taking advantage of mobility or pooling) you still have to create and manage a service profile for the blade. While you could boot a blade without a service profile, it would have no network or SAN connectivity.

4-3 UCSI Boot Camp

© Cisco Systems, Inc.

Blade Mobility with Service Profiles Blade Failure

Time A

Identity LAN/SAN Config Time B Service Profile: MyDBServer

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 4

Blade Mobility with Service Profiles This graphic is just intended to review the concept of logical server mobility. A logical server is defined with identity (e.g. UUID, MAC/WWN addresses, VLAN/VSAN requirements). The profile can be associated with only one blade at a time, but the association can be changed if there is a problem with a particular blade, or hardware maintenance required on a particular blade. The mobile logical server concept works well when the blade’s OS and data reside on external SAN storage. A “replacement blade” for the profile will inherit the WWN/VSAN information and boot specifications from the profile just like the original blade. From the point of view of the LAN/SAN infrastructure, the new blade is the same server as the old blade, and it will be exposed to and boot the exact same SAN LUN’s without any further reconfiguration. Blade local disks in this scenario could still be used for temporary storage and/or swap space.

© Cisco Systems, Inc.

UCS Basic Opt In Model4-4

Service Profiles: Multiple Servers Blade Failure Profile A1 Time A

Time A

© 2008 Cisco Systems Inc. All rights reserved.

Profile A2

Course Title —Module Name 5

Service Profiles: Multiple Servers Every blade that is booted simultaneously must have its own service profile, even if in your mind the profiles are very similar (exact same connectivity requirements, for example). There are two easy ways to create large numbers of very similar service profiles: cloning and templates. These will be discussed in more detail in the next module.

4-5 UCSI Boot Camp

© Cisco Systems, Inc.

Contents of a Service Profile

• UUID/WWNN • Boot order • Other Policies

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 6

Contents of a Service Profile This is just a summary of the contents of a service profile. Examples of blade identity information unrelated to specific network or disk adapters are the UUID and World Wide Node Name. The profile also holds network adapter identity information. All of this information is physically applied to the blade associated at a particular time with the profile. In this manner, blades associated with a particular profile at different points in time are indistinguishable from one another, from the point of view of any identity information for the purpose of licensing schemes or network and external storage configuration.

© Cisco Systems, Inc.

UCS Basic Opt In Model4-6

Profile Elements in Basic Opt-in Model

• UUID/WWNN • Boot order • Other Policies

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 7

Profile Elements in Basic Opt-in Model In the basic model the transference of identity information is not an issue. In a service profile you can use a special value of “derived” . This implies that the hardware identities in a blade’s BIOS and adapters will be used “as is”. If you want to supply actual values in the profile, you still can, and they still will be transferred to the blade server when it is associated with the profile. You still must create vNIC and vHBA objects in your profile to define connectivity of the adapters in your blades on the network. This is discussed further later in the module.

4-7 UCSI Boot Camp

© Cisco Systems, Inc.

Creating a Service Profile (GUI) Look at simplified wizard later...

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 8

Creating a Service Profile (GUI) There are two wizards for creating service profiles, the expert wizard and the simplified wizard. It is helpful to step through the expert wizard in order to understand the nature of the service profile.

© Cisco Systems, Inc.

UCS Basic Opt In Model4-8

Service Profile

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 9

Service Profile This screenshot shows the first page of the service profile creation process. You must set a unique name for the profile and an optional description. The UUID can be virtualized (more about this in the next module) or just set to the Hardware Default, which is sufficient for a basic service profile.

4-9 UCSI Boot Camp

© Cisco Systems, Inc.

Service Profile: Storage We’ll Focus on it more in next module

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 10

Service Profile: Storage The body of the storage section focuses on SAN connectivity via vHBA. We will focus more on that, and on the local storage and scrub policies, in the next module. In the module, for simplicity’s sake, we will be creating service profiles without SAN connectivity.

© Cisco Systems, Inc.

UCS Basic Opt In Model4-10

Service Profile: Networking Simple sub-wizard ---1 vNIC per fabric Single native VLAN per vNIC

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 11

Service Profile: Networking The Networking page has single and expert “sub-modes”. The simple “sub-network-wizard” allows creation of up to two vNIC’s, and lets you choose a single Native VLAN for each (but no more VLAN’s). It also does not let you choose a virtualized identity. This is sufficient for physical adapter cards (it can use the hardware default as the identity), but will have to be changed for Palo cards which require a virtualized nic identity. Note that this simple sub-wizard only allows you to create one vNIC per fabric. If you do not have a redundant fabric interconnect (as in the system on which the screenshot was taken), you only have a single fabric.

4-11 UCSI Boot Camp

© Cisco Systems, Inc.

Service Profile: Networking (expert)

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 12

Service Profile: Networking (expert) The expert “sub-network-wizard” allows you to invoke the detailed form for each vNIC. This is the form we saw in the previous module. You can specify a virtualized MAC address, and trunk as many VLANs as you like to the vNIC, with at most one being native. You could also optionally choose a pin group.

© Cisco Systems, Inc.

UCS Basic Opt In Model4-12

Service Profile: vNIC Placement

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 13

Service Profile: vNIC Placement This piece of a service profile is meaningful only when association with a blade with two physical adapters (B250 full-width blade). It lets you specify with which physical adapter each of your vNIC definitions should be associated. You can choose to let the UCS Manager automatically perform placement.

4-13 UCSI Boot Camp

© Cisco Systems, Inc.

Service Profile: Boot Order

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 14

Service Profile: Boot Order Boot order lets you specify the boot priority as a “virtualized configuration” in service profile – this boot order will be applied to the BIOS of the specific blade server as the service profile is associated. It is still possible to set the boot order manually by escaping into the BIOS control screen as you can on any traditional server.

© Cisco Systems, Inc.

UCS Basic Opt In Model4-14

Service Profile: Server Assignment

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 15

Server assignment is optional in the wizard. It is probably best practice, especially with complicated service profiles, to skip server assignment in the wizard (or if you get to this page, leave it at “assign later”), and examine and massage your service profile as a strictly logical configuration before actually assigning to a blade. On this screen you can also specify whether you want the physical server to be left powered down after the actual service profile association process, or whether you want UCS manager to automatically boot the server (using the boot priority specified by the boot order), after the actual association process.

4-15 UCSI Boot Camp

© Cisco Systems, Inc.

Service Profile: Policies

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 16

Service Profile: Policies We will talk about many of the policies to which you can refer from a service profile near the end of the next module. It is fine to click “finish” before you reach this page for now, or just not interact with this page.

© Cisco Systems, Inc.

UCS Basic Opt In Model4-16

Simplified Wizard

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 17

Simplified Wizard The simplified wizard for creating service profiles works well with the basic opt-in model. The wizard will produce a service profile with the basic characteristics outlined on the slide. If you want to customize the service profile once the wizard is finished (add more vNIC’s, attach vNIC’s to different or multiple VLAN’s, etc.) you could still do so.

4-17 UCSI Boot Camp

© Cisco Systems, Inc.

Create SP From “Blade Server” View

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 18

Create SP From “Blade Server” View From the blade server (equipment) point of view, you can pop up a window to create a service profile and immediately associate it with that blade (you could still design a service profile or pick a template not appropriate for the blade). The most basic operation creates a “hardware based service profile” for the blade, using only derived identities. vNICs and VHBAs will be based on the actual physical blade adapter (ie it will only create those for the fabrics that you actually have, and would not create a vHBA if the blade server adapter is an Oplin). There are other options on this form to instantiate a template and associate with this blade, or to branch to the expert service profile wizard. With these other two options, there is no actual guarantee that the service profile you create is actually compatible with this blade.

© Cisco Systems, Inc.

UCS Basic Opt In Model4-18

Profile: Created but not Assigned or Associated

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 19

Profile: Created but not Assigned or Associated The screenshot shows our profile. We have not yet specified any blade to associate with the profile, so the Selected Blade and Associated Blade fields are blank.

4-19 UCSI Boot Camp

© Cisco Systems, Inc.

Associating Blade Actually two step process (but all done with “Associate” command): 1. Assigning blade to service profile (just a UCS config)  Could still fail if blade not compatible with profile

2. Blade Configuration through PnuOS

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 20

Associating Blade When you give the command to associate a blade, UCSM first tries to assign the blade to the configuration. This does nothing to modify the blade itself. However the assignment still checks that a blade is compatible with a profile. If it is not compatible, it will fail. Examples of incompatible blades: • • • •

Too few physical network adapters for number of vNICs vNIC specified for failover but you have an Oplin in the blade vNIC specified for failover, but you have only one fabric interconnect vHBA specified, but you have an Oplin

Once the blade is successfully assigned, the actual association configuration process begins. This involves UCSM causing a mini-OS called PnuOS (processing node utility OS) to be booted on the blade.

© Cisco Systems, Inc.

UCS Basic Opt In Model4-20

Configurating Blade through PnuOS Physical process of applying profile to blade  UCSM automatically powers on blade if necessary  UCSM automatically boots or reboots blade  UCSM provides network boot of PnuOS to blade  UCSM communicates over network to blade’s PnuOS  Once finished, identity/boot order has been written to blade  Association process also configures UCS fabric for proper VLAN (and VSAN) traffic to reach adapter on blade

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 21

Configurating Blade through PnuOS PnuOS servers as a “pre-OS configuration Agent” for the blade. UCSM automatically powers on the blade and sets it up for a PXE (network) boot. UCSM itself provides the DHCP response and PnuOS download on a special dedicated management VLAN (4047) to the blade. Once PnuOS is booted UCSM communicates with it to configure the blade according to the specifications of the profile (adapter identities and the like). Once PnuOS is finished, the blade boots using its BIOS boot order (which may or may not have been provided by the profile).

4-21 UCSI Boot Camp

© Cisco Systems, Inc.

Associating Blade with Profile

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 22

Associating Blade with Profile As depicted in the screenshot, you use the Change Service Profile Association to access a form to select the blade. When committed, this will initiate the assignment and association of the blade. The “Pre-provision a slot” is interesting in that it allows you to choose to “pre-associate” a service profile with an empty slot. When a blade server is actually inserted into that slot, it will undergo automatic discovery, and then UCS Manager will automatically try to associate the service profile with the blade server (there is no guarantee, still, that the service profile and new blade server are compatible).

© Cisco Systems, Inc.

UCS Basic Opt In Model4-22

Profile Assigned and Associating

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 23

Profile Assigned and Associating The GUI indicates that the selected blade is successfully assigned. There were no incompatibilities between the blade and the profile. It is in the process of associating.You could just wait for the association to complete (the state will change to associated). You could also already access the blade console and actually watch it booting PnuOS, just to make sure everything looks normal.

4-23 UCSI Boot Camp

© Cisco Systems, Inc.

KVM: Watching PnuOS on Blade

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 24

KVM: Watching PnuOS on Blade It is possible to use the KVM Console watch the blade server booting PnuOS as part of the association process. While you could interact or interrupt PnuOS, doing so might conceivably interfere with the association process. Even though you see a login prompt from PnuOS, the intention is that you leave it alone and let UCS manager direct it over the network to do its blade configuration.

© Cisco Systems, Inc.

UCS Basic Opt In Model4-24

Booting a Blade from Virtual Media

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 25

Booting a Blade from Virtual Media The KVM, together with the virtual media sub-feature discussed in an earlier module, are the “manual” way to install an OS on a blade server that has a service profile. Note once we configure networking in the OS we are likely to have less need for the KVM console since we can just access the OS over the network (Remote Desktop for Windows, for example).

4-25 UCSI Boot Camp

© Cisco Systems, Inc.

Managing Blade from Profile (Logical Server) view

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 26

Managing Blade from Profile (Logical Server) view Note that once a blade is associated with a profile, you can administer the server from the “profile point of view”. In other words, you can treat the profile as if it were the server, without having to even know or remember which blade you are talking about. This is obviously more valuable in the “mobile logical server” and “pooling” paradigms, and is a good segue to the next module.

© Cisco Systems, Inc.

UCS Basic Opt In Model4-26

Lesson 5

Server Profiles and Stateless Computing Overview This module discusses the full implementation of the stateless computing model using service profiles. The discussion is divided into two themes: • Mobile (relocatable) service profiles: same logical server can be booted on different blades at different times. When a blade is associated with a service profile, it will inherit all its identity and boot information from the profile. Booting off a SAN LUN may be used as an easy way to achieve statelessness of the OS. • Server farms using server pools: multiple very similar logical service profiles can be used to create servers in a farm. Actual blade hardware and identities are chosen from pools.. Objectives Upon completion of this lesson participants should be able to: •

Understand service profiles with stateless model

•

Configure VSANs, vHBAs and SAN boot

•

Understand server pools

•

Use virtual identity pools

•

Use cloning and templates

•

Summarize some of the advanced service profile policiesenclosure

Server Profiles and Stateless Computing

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 1

This module discusses the full implementation of the stateless computing model using service profiles. The discussion is divided into two themes: Mobile (relocatable) service profiles: same logical server can be booted on different blades at different times. When a blade is associated with a service profile, it will inherit all its identity and boot information from the profile. Booting off a SAN LUN may be used as an easy way to achieve statelessness of the OS. Server farms using server pools: multiple very similar logical service profiles can be used to create servers in a farm. Actual blade hardware and identities are chosen from pools.

5-1 UCSI Boot Camp

© Cisco Systems, Inc.

Lesson Objectives •Understand service profiles with stateless model •Configure VSANs, vHBAs and SAN boot •Understand server pools •Use virtual identity pools •Use cloning and templates •Summarize some of the advanced service profile policies

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 2

In this lesson students will: • • • • • •

Understand service profiles with stateless model Configure VSANs, vHBAs and SAN boot Understand server pools Use virtual identity pools Use cloning and templates Summarize some of the advanced service profile policies

© Cisco Systems, Inc.

Server Profiles and Stateless Computing5-2

Blade Mobility with Service Profiles (Review) Blade Failure

Time A

Identity LAN/SAN Config Time B Service Profile: MyDBServer

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 3

Blade Mobility with Service Profiles (Review) This graphic is just intended to review the concept of logical server mobility. A logical server is defined with identity (UUID, MAC/WWN addresses, VLAN/VSAN requirements). The profile can be associated with only one blade at a time, but the association can be changed if there is a problem with a particular blade, or hardware maintenance required on a particular blade. SAN boot can be part of the design of a completely mobile logical server. A “replacement blade” for the profile will inherit the WWN/VSAN information and boot specifications from the profile just like the original blade. From the point of view of the LAN/SAN infrastructure, the new blade is the same server as the old blade, and it will be exposed to and boot the exact same SAN LUN’s without any further reconfiguration. Blade local disks in this model could still be used for temporary storage and/or swap space.

5-3 UCSI Boot Camp

© Cisco Systems, Inc.

Identity Information for Mobile Profiles

• UUID/WWNN • Boot order • Other Policies

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 4

Identity Information for Mobile Profiles In the rack-mounted paradigm it was typical to use the derived values for UUID and for adapter identities. In the full logical server paradigm, you need to have identities defined with the logical service profile that will then be applied to the blade. UUID is a 128-bit number (32 hex digits, 16 groups of 2). It is supposed to uniquely identify a component worldwide. There are various UUID generation algorithms. You can also use a UUID suffix pool. UCS Manager will presumably automatically generate a unique prefix so that you will be guaranteed a unique UUID for each logical server. Today the prefix is all 0’s. For MAC and WWN, you can also make up values, or you can use pools. Pools will guarantee uniqueness at least among all the profiles that use the same pool.

© Cisco Systems, Inc.

Server Profiles and Stateless Computing5-4

SAN and SAN Boot

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 5

SAN and SAN Boot SAN usage, and especially SAN boot, or often central parts of stateless model design. There will be one (always virtualized) WW Node Name per service profile, and, in the fully stateless model, an always virtualized WW Port Name per vHBA. When a service profile moves from blade to blade the SAN host identities will move with it. Service profile running on a new blade will be indistinguishable from an external SAN fabric and storage array perspective from the same profile running on the orignal blade. You will properly access the same external storage, without any reconfiguration on the external fibre fabric (zones, for example) or remapping of storage LUN’s.

5-5 UCSI Boot Camp

© Cisco Systems, Inc.

Revisiting SAN Part of Wizard

Add VHBA’s

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 6

Revisiting SAN Part of Wizard On the storage page of the expert service profile eizard, you can choose expert mode to enter a single virtualized WW Node Name, and click the Add button to invoke the form for the properties for each vHBA.

© Cisco Systems, Inc.

Server Profiles and Stateless Computing5-6

Revisiting SAN Part of Wizard

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 7

Revisiting SAN Part of Wizard For each vHBA, you specify a virtualized WW Port Name. TYou also choose the VSAN and Pin Group, if any.

5-7 UCSI Boot Camp

© Cisco Systems, Inc.

SAN Boot in Service Profile Boot Order

3 Key Items: vHBA Name WW PN of Storage Array LUN # – seen by the server – LUM mapping is transparent

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 8

SAN Boot in Service Profile Boot Order To specify a SAN boot in service profile, you specify a “triplet” (vHBA name, target-array-WWPN, LUN #). This is the LUN # as exposed by the storage array mapping to the server, based typically on combination of the server’s (for us, virtualized) WWNode Name and WWPort Name. In a multipathing scenario, you would specify two different triplets in the boot order. (vHBA1, target-WWPN-1, LUN #) (vHBA2, target-WWPN-2, LUN #) The intention is that this would be two different paths, on two different RAID controllers, to the same LUN.

© Cisco Systems, Inc.

Server Profiles and Stateless Computing5-8

Moving a Service Profile to Different Blade 1. Shut server down nicely (if possible) 2. Disassociate from Current Blade 3. Associate SP to New Blade (can do right away) 4. Old blade has to disassociate before avail.

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 9

Moving a Service Profile to Different Blade If you are disassociating a service profile to do maintenance on a physical blade, it is likely you want to shut the server down nicely. If the server is still up when you disassociate, UCSM will try to shut the server OS down gracefully, but if it cannot do so within three minutes, the server will come crashing down. Note that true logical server mobility is likely to work flawlessly only when the old blade and new blade have the same type of mezzanine card (both Menlo, or both Palo). While the same profile can easily associate with blades with different types of adapters, the problem is that an existing OS for the profile will have storage and network adapters already associated with particular drivers. Unfortunately, the drivers are different for Menlo and Palo.

5-9 UCSI Boot Camp

© Cisco Systems, Inc.

Service Profiles and “Farm Model” Blade Failure

OS/data on blade local disk OR SAN

Profile A1 (choose blade and identites from pool)

Time A

Profile A2 (choose blade and identities from pool)

Time A

OS/data on blade local disk OR SAN

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 10

Service Profiles and “Farm Model” Every blade that is booted simultaneously must have its own service profile. The profile mechanism still facilitates multiple blade servers being treated like a farm in the following manner: • Pools of blades can be created manually or automatically. • A profile can be set to choose its assigned blade from a pool. • Identities (UUID, WWNN, WWPN’s, MAC’s) can be chosen from a pool (a unique identity taken from the pool will still belong to a particular service profile, and move from blade to blade if that service profile moves) • Templates or clones can easily be used for generation of similar service profiles simultaneous booting of multiple blades in the pool.

© Cisco Systems, Inc.

Server Profiles and Stateless Computing5-10

Server Pools Concepts Manually populated or Auto-populated Blade can be in multiple pools “Associate” service profile with pool – Will choose available (unassociated) blade or WAIT! – Will match needs of service profile

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 11

Server Pools Concepts Server pools lend themselves to the server farm model. Pools can be manually populated or auto-populated using Server Pool Policies, as demonstrated on an upcoming slide. A blade can be in multiple pools at the same time. Whichever profile “claims” a particular blade is its current “owner”, regardless of the number of pools it is in. To actually use a server pool, you associate a service profile with the pool. UCS Manager automatically selects an available blade from the pool. (An available blade is one currently discovered but not associated with any profile, and not in the process of being associated or disassociated).

5-11 UCSI Boot Camp

© Cisco Systems, Inc.

Creating and Manually Populating Server Pool

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 12

Creating and Manually Populating Server Pool Manually populating a server pool using the GUI is simple and natural, as shown on the screenshot. Remember, a blade can be in as many pools as you like. There is no distinction made between currently available (unassociated) blades and currently unavaiable (associated) blades when you manually populate a pool.

© Cisco Systems, Inc.

Server Profiles and Stateless Computing5-12

Auto-Populating a Server Pool 1. Create Empty Server Pool 2. Create Server Pool Qualification 1. Mix and match various criteria: 

Blade slots



CPU/RAM, etc.

3. Create a Server Pool Policy “blades matching this qualification will automatically be placed in this pool”

4. Auto-placement of blade in pool happens at blade discovery time © 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 13

Auto-Populating a Server Pool The auto-population feature lets you: • Specify qualifications that will be used for matching specific blades • Specify server pool policies --- these will put every blade that matches a particular qualification into a particular server pool Note that server pool auto-population only happens as individual blades are discovered. If you want it to apply to existing blades, you need to re-acknowledge them.

5-13 UCSI Boot Camp

© Cisco Systems, Inc.

Creating a Server Pool Qualification

0

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 14

Creating a Server Pool Qualification A qualification is a named object that can combine (within one named qualification) various types of parameters (adapter card, memory, cpu, storage). In this example, we are creating a qualification named “PlentyOMemoryl” which contains only a memory qualification. Blade Servers that have at least 75000 MB of RAM (72GB or so) will meet this qualification.

© Cisco Systems, Inc.

Server Profiles and Stateless Computing5-14

Creating a Server Pool Policy

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 15

Creating a Server Pool Policy A server pool policy ties together a named blade qualification, such as PlentyOMemory, as shown, with an existing server pool. As blades are discovered, those that meet the qualification will automatically be put in the pool.

5-15 UCSI Boot Camp

© Cisco Systems, Inc.

Identity Pools UUID Pools WW Node Name Pools WW Port Name Pools Mac Pools

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 16

Identity Pools There are identity pool types for all the different types of virtualized identity. A service profile or vNIC or vHBA, as appropriate, will specify that it wants to get the identity from a pool. Note that this is still a virtualized identity that belongs to the service profile until you delete the service profile or specify a different identity. It is the service profile that is the consumer of the pool, not the physical blade. Virtualized identities retrieved from pools still move from blade to blade along with the service profile. UUID – Allows you to set up a sequential pool of UUID suffixes, assigned as soon as service profile creation is complete WWNN Pool - Allows you to set up a sequential pool of WWN for assignment of the Adapter Node of the service profile’s HBA, assigned as soon as service profile creation is complete WWPN Pool - Allows you to set up a sequential pool of WWN for assignment of the Port Node of the service profile’s vHBA, assigned as soon as service profile creation is complete MAC Pools - Allows you to set up a sequential pool of MAC addresses to assign to the serivice profiles vNICs, assigned as soon as service profile creation is complete © Cisco Systems, Inc.

Server Profiles and Stateless Computing5-16

Creating Pools Always specify low value, how many values

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 17

Creating Pools The screenshot shows, as an example, creating a MAC address pool. You always specify the lowest identity and the number of values. Each value is tracked by the UCS manager, so it is not advisable to create gigantic identity pools, “just for the fun of it”. The GUI enforces specific ranges (Cisco OUI 00:25:B5, for example, for MAC addresses). The CLI does not enforce this.

5-17 UCSI Boot Camp

© Cisco Systems, Inc.

Using Pools Always Choose the Pool Name: “Change Association” (blade pool) WWNN specifier (WWNN pool) vHBA form (WWPN pool) vNIC form (MAC Pool)

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 18

Using Pools You reference pools from the appropriate places in service profile. A vNIC can choose on the GUI to get its identity from a particular pool, and so on. Note that service profile tries to consume an identity from a pool even before it is associated with a blade. remember, the service profile is the consumer. If the pool is empty or exhausted as you configure a service profile, the service profile creation or change still succeeds. The service profile will try one last time to get a virtualized identity from the pool at the time you associate with a blade. If at that time the pool is still exhausted, the association fails, waiting for a blade or an identity to become available. Once one becomes available, it will automatically be used by the SP (with no further human interruption required)

© Cisco Systems, Inc.

Server Profiles and Stateless Computing5-18

Advantage of Pools Challenge

With Pools

Without Pools

Managing Virtual Identiities

• Associating done if resources are available • Will wait until resources arrive

Must manually assign reassign server resources

Templates

• Allows greater speed and flexibility of server creation • Can update many profiles by updating template

Must make sure to make changes to created profiles to avoid resource conflicts

Cloning

Allows greater speed and flexibility in server creation

Must make sure to make changes to created profiles to avoid resource conflicts

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 19

Advantage of Pools Using pools can be a management advantage in all scenarios requiring virtualized identities. This is an alternative for making up a birthday for each individual virtualized identity, and hoping and praying you never have a conflict. With pools, you will be actively prevented from virtualized identity conflicts within the same UCS system. Pools have a particular advantage in a “server farm” model. Clones and templates can give you ways of easily creating many similar service profiles to be used simultaneously for different servers. This is detailed on the next few slides.

5-19 UCSI Boot Camp

© Cisco Systems, Inc.

Service Profile Templates Service Profile:ORC10gRAC-1 Service Profile Template: OracleServer • UUID: OracleUUPool • Boot Order: OrcBootPolicy • Host Firmware: OrcFWPolicy • SOL: SOLPolicy • WWNN: OrcWWNNPool

• UUID: 10:02:03:04:05 • Boot Order: VM / SAN Target 50:00:00:34:FA • Host Firmware: 9.0.03.146 • WWNN: 20:00:00:00:10 • WWPN: 20:00:00:00:20 • MAC: 20:03:CA:00:05

Adapter Policy: Eth0 VLAN: Oracle Fabric: A MAC: OrcMacPool

Adapter Policy: fc0 VSAN: Oracle Fabric: A WWPN: OrcWWPNPool

• UUID: 10:02:03:04:F3 • Boot Order: VM / SAN Target 50:00:00:34:FA • Host Firmware: 9.0.03.146 • WWNN: 20:00:00:00:50 • WWPN: 20:00:00:00:40 • MAC: 20:03:CA:00:FA

Service Profile:ORC10gRAC-4 © 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 20

Service Profile Templates Templates are an alternate way of creating large numbers of similar profiles. Templates come in two flavors. Initial templates are used to set the initial properties of an instance of the template, but modification of the template does not modify existing instances. An updating template is a slight variation. Modification of the template does modify existing instances of the template when you have an updating template.

© Cisco Systems, Inc.

Server Profiles and Stateless Computing5-20

Creating Template Looks just like creating profile Pick blade pool Pick identity pool Initial or Updating?

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 21

Creating Template The process of creating a template is nearly identical to that of creating a blade profile. You can associate the template to a server pool – in this case, a profile that is created as an instance of the template will have a blade selected from the pool immediately. You are likely to also choose MAC and WWN pools for the virtual adapters that you have in the template. Once again, as a profile as created as an instance of the template, actual values will immediately be selected from the pools.

5-21 UCSI Boot Camp

© Cisco Systems, Inc.

Creating Service Profile(s) from Template

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 22

Creating Service Profile(s) from Template Once a template is created, you can create as many instances of the template as you like. Each instance will be a separate logical service profile. The whole model really only works well when the template specifies that its virtualized identities are retrieved from pools. In the example, 44 instances of the service profile would be created (SPWebFarm1, SPWebFarm2, and so on) and each retrieve separate identities from the same pools. If pools are exhausted for some of the instantiated instances, those instances will fail to associate, but will wait for pool values to become available (and then immediately associate without any further intervention) They will be named using the specified prefix and a numerical suffix.

© Cisco Systems, Inc.

Server Profiles and Stateless Computing5-22

Unbinding and Rebinding From Template SP from updating or initial template is “bound

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 23

Unbinding and Rebinding From Template Service Profiles that are instantiated form templates are always “bound” to the template. Even if it came from an initial template (so it is safe from future modifications of the template), a service profile that came from a template cannot be individually modified until you unbind it. This is pretty much a “configuration safety” issue, “warning” the administrator that individual modifications to a service profile that came from a template are somewhat of a violation of the intention of the template.

5-23 UCSI Boot Camp

© Cisco Systems, Inc.

Service Profile and Template Cloning All pool associations retained on cloning Cloned SP gets new values/blade from pool “Specific virtual identities” not cloned

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 24

Service Profile and Template Cloning Note how cloning profiles lends itself well to the pool / server farm model. If a profile is associated with pools (associated with server pool, or its identity associated with MAC, WWN, and UUID suffix pools, then the clone will have these same pool associations. If the original is associated with server pool, the clone will also be, and a blade will be selected for association immediately on creation of the cloned profile. If the original is associated with (MAC, WWN) pools, specific values will be assigned out of the pool immediately as the clone is created.

© Cisco Systems, Inc.

Server Profiles and Stateless Computing5-24

Lesson 6

Multi-tenancy: Organizations and RBAC Overview In this lesson we will now examine how Organizations and the RBAC system allow for multi- tenancy. Objectives Upon completion students will be should be able to : •

Understand how orgs and RBAC can work separately and together

•

Define functionality of the hierarchical organization tree

•

Make use of objects higher in the tree in a service profile

•

Define locales for RBAC

•

Understand roles and define new roles

•

Define users that have roles in particular locales

Lesson Objectives Understand how orgs and RBAC can work separately and together Define functionality of the hierarchical organization tree Make use of objects higher in the tree in a service profile Define locales for RBAC Understand roles and define new roles Define users that have roles in particular locales

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 2

Upon completion students will: • • • • • •

Understand how orgs and RBAC can work separately and together Define functionality of the hierarchical organization tree Make use of objects higher in the tree in a service profile Define locales for RBAC Understand roles and define new roles Define users that have roles in particular locales

6-1 UCSI Boot Camp

© Cisco Systems, Inc.

Multitenancy Features Organizations (Organize Resources) – Pools – Policies – Profiles

RBAC – Locales – Roles – Priviledges

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 3

Multi-tenancy Features The organization structure (“orgs”) defines a management hierarchy within the UCS system. The hierarchy is strictly for management of resources and logical objects inside the system, it has no effect whatsoever on the actual operation of a blade and its operating system. The organization structure goes hand-in-hand with the Role-Based Access Control (RBAC) delegated management features. The RBAC features not only allow finegrained privileges to be assigned to different management users , but also allow restriction of a user’s privileges to a specific set of organizations.

© Cisco Systems, Inc.

Multitenancy: Organizations and RBAC 6-2

Organizations and RBAC

Locale

User

Roles Organization

Sub-Organization

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 4

Organizations and RBAC It is not absolutely required that you use RBAC if you are managing a hierarchical set of organizations, or vice versa. Orgs have advantages outside of RBAC: The structural management hierarchy allows you to better manage large amounts of equipment and large numbers of logical objects. In addition, some of the pool inheritance features that we will discuss can let you design preference logic that you could not achieve with a single flat org. Similarly, the RBAC style of delegated administration has advantages even if you do not have a hierarchical org structure. Using RBAC, you can still have users who have control over the logical server structure and other users that have control over the border (LAN and SAN) configuration, for example. Remember: • Orgs and RBAC could be used independently • Orgs without RBAC • Structural hierarchy for resources

6-3 UCSI Boot Camp

© Cisco Systems, Inc.

• RBAC without Orgs – More for defining job roles globally (e.g. SAN activities vs. LAN activities) • Everything in root org (as we have been doing so far)

© Cisco Systems, Inc.

Multitenancy: Organizations and RBAC 6-4

Organization Concepts Root is the parent to all Orgs Children can use Parent resources Do not have to create Orgs –Can use Root as your Org –RBAC will be based on Global UCS role

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 5

Organization Concepts We have actually been using orgs all along in all the discussions of configuration of logical servers. There is always an org named “root” (/), and if you do not create any other orgs, all logical server, pool, and policy administration is done within this single org.

6-5 UCSI Boot Camp

© Cisco Systems, Inc.

Example Organization Hierarchy root (/)

SWDev

SWgrpA

QA

SWgrpB

IntTest

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 6

Example Organization Hierarchy Multiple orgs are always configured in a hierarchical structure. Create a sub-org of a particular level by using the create org command, or equivalent GUI right-click gesture, in the scope of the parent org which will contain the new org. Understanding the hierarchy is important due to the pool and policy “inheritance” features that will be discussed on the upcoming pages.

© Cisco Systems, Inc.

Multitenancy: Organizations and RBAC 6-6

What’s in an Org? Orgs Organize Resources: • Parent Org has different resources than Child • Resources can have similar names • Management Hierarcy controls what resources an Org can access

• Resource has same name but in different Orgs • Child Can use Parents Pool

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 7

What’s in an Org? The most important element of an org. is the service profile. Each service profile lives in one and only one org. The profiles themselves (the entire service profile objects) are not inherited in any way. There is no direct way to move a logical service profile (or any other object, for that matter) from one org to another. The slide shows the profiles and policies that are also contained in an org. When you view an org in the GUI, links to configure all the profiles and policies within that particular org will automatically appear underneath the new org. The final element contained within an org are the server pools, MAC pools, and WWN pools. These appear in slightly different places in the GUI --- server pools are near the bottom of the Servers panel on the navigation pane, and MAC pools and WWN pools are on the LAN and SAN panes respectively. When you create a new org, appropriate links for the pools within the new org will appear.

6-7 UCSI Boot Camp

© Cisco Systems, Inc.

Blades and Organizational Concepts Blades are exclusive to Org Blades are not exclusive in pools Manually put different blades in specific pools Create Qualifiers to automate

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 8

Blades and Organizational Concepts Note that blade equipment itself is never part of an org. Any blade can appear in as many pools in as many different orgs as you like. Any blade can be associated with a logical server in any org. While blade pools (server pools) are part of an org, they are in a way advisory only. There is nothing that prevents you from associating any individual blade to any service profile in any org (bypassing the pools), regardless of how many pools the blade may be in. Note how the rule is basically “first come first served”. The first profile that successfully selects a blade for association (rather through pools, or directly without pools) is the owner of that blade until it is made available again through disassociation.

© Cisco Systems, Inc.

Multitenancy: Organizations and RBAC 6-8

Server Pools and Orgs

Second

When associating Profile with Server Pool “MyPool”:  Association if pool in local org or Parent – First try in local org – If does not exist :  Try pool with same name in next highest org

First

© 2008 Cisco Systems Inc. All rights reserved.

 Continue upward to root org

Course Title —Module Name 9

Server Pools and Orgs The hierarchical nature of server pools (and other pools, discussed on a subsequent slide) is one of the key features to understand about hierarchical organizations. There are two big rules: A logical service profile has access to names of server pools defined in its own local org, as well as server pools defined in orgs further up the tree. For example, imagine there is no server pool named MyPool in the same org as a logical server. However, imagine there is such a named pool in the org that is the grandparent. The logical server can still do its blade selection from that pool. Imagine there is a server pool named MyPool in the same org as a logical server. If that logical server specifies it wants to select a blade form MyPool, it will look first for a blade in that pool in the same org. If no blade is available from that pool, it will look for a pool with the same name (MyPool) in the parent org, and so on up to the root org until it successfully can allocate a blade for the profile.

6-9 UCSI Boot Camp

© Cisco Systems, Inc.

Server Pool Example Can Associate a service profile in Org “QA” (suborg of root) with any of these pools:

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 10

Server Pool Example This screenshot demonstrates a logical service profile in the org named CueAye. The profile has access to a server pool in its own organization (the pool BurgessPool in this example), as well as server pools in the parent pool (the pool YMCAPool in this example). If there were a parent of that parent (there is not, since the parent is the root pool), the service profile in question would have access to its pools as well.

© Cisco Systems, Inc.

Multitenancy: Organizations and RBAC 6-10

Server Pool Example: Pool in Org and Parent Org with same pool name

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 11

Server Pool Example: Pool in Org and Parent Org with same pool name This screenshot demonstrates what happens when there are pools with the same name in the organization in question (where the service profile in question lives) and in the parent (or grandparent, etc). When the logical service profile specifies selection from that pool name, it will search first in the pool in its own organization. If that pool has no blades available, then it will search in the pool with the same name in the parent organization, and so on. Note that this feature can be used as a strategy if you want to have a set of “preferred blades” for a logical server and “less preferred blades” for the same logical server. You can put the preferred blades in a pool in the same org as the logical service profile, and put the less preferred blades in a pool with the same name in a higher level organization.

6-11 UCSI Boot Camp

© Cisco Systems, Inc.

MAC Pools, WWNN Pools, WWPN Pools UUID Pools Analogous to Server Pools Same rules - local org or upward Find an available (MAC,WWN, UUID) in first pool with that name

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 12

MAC Pools, WWNN Pools, WWPN Pools UUID Pools MAC Pools, WWN Pools, and presumably UUID pools work identically to server pools. • Pools in parent/grandparent/etc org but with no such identical pool name in the “org in question” are still available to service profiles in the “org in question” • If there is a pool with the same name, then the service profile will look for an available entity starting from the same org as the service profile, and if none are available, then go upward using a pool with the same name.

© Cisco Systems, Inc.

Multitenancy: Organizations and RBAC 6-12

Other Server Parameters in Org (not Pools)

Example: Separate Policies Between Orgs

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 13

Other Server Parameters in Org (not Pools) A few of the other parameters for logical servers behave similarly to pools but slightly different in that they are just policies and there is nothing to allocate: •Names of these policies in the “org in question” and all parent/grandparent/etc orgs are available to a service profile in the “org in question” •If there are policies with same name in the “org in question” and a higher level org, a service profile in the “org in question” can only use the lower one. One way to look at this is: “a lower level policy with the same name as a higher level policy overrides the higher level policy for all logical servers at that lower level (or for those even lower than the lower level). Remember: •Similar but slightly different for following parameters of server profile: •Boot Policy •Local disk config policy •Host firmware policy 6-13 UCSI Boot Camp

© Cisco Systems, Inc.

•Can associate a profile with a policy in same or higher org •If policy in same org as logical server has same name as policy in higher org, higher one is not available for that logical server

© Cisco Systems, Inc.

Multitenancy: Organizations and RBAC 6-14

RBAC Role:myrole

Locale: myloc root (/) /SWDev /Eng/HWEng

priv1 priv2 priv3

User is assigned certain privs (one or more roles) over certain locales (one or more)

User: jim © 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 14

RBAC This slide shows a summary of role-based access control on the UCS system. A user is created and is given one or more roles to be applied in one or more locales. A locale is a collection of orgs – these orgs do not need to be related in the hierarchy in any special way. The only purpose of a locale is for application of a user’s roles (ie, they are used only with the RBAC scheme, and don’t have any other management meaning). A role is a set of privileges. There are several predefined roles, and you can create your own roles with custom sets of privileges. Note there is no way to have a user have some privileges in some orgs, and different privileges in a different orgs. All of those privileges that are org-related in the combination of all of a user’s roles will all be applied equally to all the orgs in all the locales specified for that user. There are privileges that are not org-related. If these privileges are present in the set of roles assigned to a user, they will apply regardless of the locales assigned to the user. 6-15 UCSI Boot Camp

© Cisco Systems, Inc.

Roles and Privileges

Roles

Privileges

Collection of Privileges

Specific, individual, permissions.

Can create custom roles

Cannot create privileges

Predefined Roles based on job roles

Special privileges: AAA, Admin

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 15

Roles and Privileges Some special privileges: • •

admin: can do everything on the whole system (including every org) aaa: Authentication, authorization, and accounting (administration of the RBAC feature itself)

Note that neither of the above privileges is org-specific. Some predefined roles: • admin: predefined role that has the admin privilege • aaa: predefined role that has the aaa privilege We will look at some more predefined roles on subsequent pages. The only predefined user is the user named “admin” who has the “admin role” and therefore the “admin privilege” which allows complete administration of the entire system.

© Cisco Systems, Inc.

Multitenancy: Organizations and RBAC 6-16

Predefined Roles and Privileges: Server Equipment

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 16

Predefined Roles and Privileges: Server Equipment The screenshot demonstrates the predefined server-equipment role, and its associated privileges. These privileges allow manipulation of the physical equipment (not service profiles).

6-17 UCSI Boot Camp

© Cisco Systems, Inc.

Predefined Roles and Privileges: Server Profile

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 17

Predefined Roles and Privileges: Server Profile Another predefined role is the “server-profile” role. You can see on the screenshot that the set of privileges for the this role are many of the privileges that allow administration of logical service profiles themselves and all the coordinate policies that go along with them. The privileges for the “Server” role are org-specific. That is, a user given this role will have these particular privileges only in the orgs that are part of the locales associated with that user. Remember again, it is the user (not the role) that is the glue between privileges and the specific orgs in which those privileges apply.

© Cisco Systems, Inc.

Multitenancy: Organizations and RBAC 6-18

Predefined Roles and Privileges: Network

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 18

Predefined Roles and Privileges: Network The predefined “Network” role defines a set of privileges which are not org-specific. They have to do with configuring the global LAN parameters (which VLAN”s are supported globally and per switch in the entire UCS, for example).

6-19 UCSI Boot Camp

© Cisco Systems, Inc.

Creating Locales

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 19

Creating Locales Recall locales are just sets of orgs, and the only reason for creating one is to assign it to a user so that that users org-specific privileges will apply only to those orgs. In the GUI you can drag sets of orgs into the locale creation pane using standard GUI gestures (CTRL-select and drag).

© Cisco Systems, Inc.

Multitenancy: Organizations and RBAC 6-20

Creating User

© 2008 Cisco Systems Inc. All rights reserved.

Course Title —Module Name 20

Creating User The screenshot shows the user creation screen. You can choose any combination of roles and any combination of locales. The union of all privileges for all the roles will be applied to the union of all orgs in the locales. There is no way to say “let a user have privilege set A in Org X but let the same user have privilege set B in Org Y”.

6-21 UCSI Boot Camp

© Cisco Systems, Inc.

Lesson 7

UCS System Maintenance Overview The primary learning objectives for this module are for you to be able to understand firmware upgrades, to manage backup and restores and to manage high availability. We also discuss running scripts of CLI commands, and configuring some of the external interfaces such as the call-home facility. Objectives Upon completion participants should be able to: •

Understand firmware upgrades

•

Backup and restore configuration

•

Manage high availability

•

Perform other maintenance tasks.

UCS System Maintenance Objectives

• Understand firmware upgrades • Backup and restore configuration • Manage high availability • Perform other maintenance tasks © 2008 Nuova, Inc. All rights reserved.

ICNX5 v1.0—2

The primary learning objectives for this module are for you to be able to understand firmware upgrades, to manage backup and restores and to manage high availability. We also discuss running scripts of CLI commands, and configuring some of the external interfaces such as the call-home facility.

• Understand firmware upgrades • Backup and restore configuration •

Manage high availability

•

Perform other maintenance tasks

7-1 UCSI Boot Camp

© Cisco Systems, Inc.

Upgrade Overview

Manage from equipment POV

Manage Using Service Profile

© 2008 Cisco Systems Inc. All rights reserved.

*Note overlap: All firmware must first exist in UCSM catalogue

UCS System Product Overview—data center and Server Trends 3

Upgrade Overview “Equipment point of view” firmware upgrades are often associated with a clean sweep firmware upgrade. This may be an activity you are likely to do as soon as you boot a brand new UCS system for the first time. Though much of the firmware that comes from the factory is likely to be up-to-date, you will still need to check all the firmware and apply updates as necessary. Managing firmware via service profile is a completely different way of looking at firmware. Using service profile, you can specify “by policy” the revisions of firmware that should be on a blade when it is associated with a particular service profile. You need not worry what firmware is on the blade before its association with your service profile; the association itself will check the firmware revisions, and, if they don’t match your service profile specifications, will update the appropriate firmware. No matter which point of view you are working with as you manage UCS firmware, all firmware must first be loaded into the USCM catalogue, or repository, before it can actually be applied to a component on the system.

© Cisco Systems, Inc.

UCS System Maintenance 7-2

UCS Repository (Catalogue) of Firmware  Firmware download to UCS  “Full bundle” or individual firmware  Can have multiple bundles or versions  Available then for all upgrades

© 2008 Cisco Systems Inc. All rights reserved.

UCS System Product Overview—data center and Server Trends 4

UCS Repository (Catalogue) of Firmware In order for firmware to be available for use in a UCS component it must first be downloaded into the UCSM catalogue, or firmware repository. Firmware’s presence in this repository does not mean that the firmware may yet be actually in use on a UCS component. Putting firmware in the catalogue is a separate operation from actually applying it to a component, whether that firmware installation will be using the equipment point-of-view or Service profile methods. Downloading firmware to the manager’s catalogue is driven by an action on the manager itself. Today, there is no built-in way of downloading firmware periodically (no “automatic updater”). Software retrieved from Cisco is placed on a local server accessible to the UCS manager. This software will either be a “single file bundle”, that incorporates a revision of firmware for all the components, or an individual firmware component. When UCS manager downloads a “bundle” it will automatically extract the individual components and catalogue them accordingly.

7-3 UCSI Boot Camp

© Cisco Systems, Inc.

A firmware download to the repository will never harm a running system (it affects none of the firmware actually running)

© Cisco Systems, Inc.

UCS System Maintenance 7-4

Download Bundle to Repository

© 2008 Cisco Systems Inc. All rights reserved.

UCS System Product Overview—data center and Server Trends 5

Download Bundle to Repository The firmware download action (to the UCS repository) on the GUI is accessed by highlighting the word “Equipment” on the equipment pane and accessing the firmware management pane. Not the firmware download cannot currently use http, or https; this implies you must have a server accessible to the UCS manager that uses ftp, tftp, scp, or sftp. You do not need to tell the downloader whether you are downloading an individual component or an entire bundle (as in the screenshot). The downloader will figure it out based on the contents of the download (each download is in a proprietary archive format that includes a table of contents).

7-5 UCSI Boot Camp

© Cisco Systems, Inc.

Repository Contents (“package view”)

© 2008 Cisco Systems Inc. All rights reserved.

UCS System Product Overview—data center and Server Trends 6

Repository Contents (“package view”) There are two views of the existing contents of the software repository. The “package view” categorizes the component software by the download file that it came in. You do not manage the individual components from this view, but if you delete components from the “Images” view (next slide) they will be marked as such in this package view. If you delete all the components that make up a package, the entire package will be deleted from this view.

© Cisco Systems, Inc.

UCS System Maintenance 7-6

Repository Contents (“Image view”)

© 2008 Cisco Systems Inc. All rights reserved.

UCS System Product Overview—data center and Server Trends 7

Repository Contents (“Image view”) This is the individual component view of the repository. You can delete components from this view. You can even delete firmware which is running on a “flashable component” (anything other than the fabric interconnect or manager), since deleting it from the catalogue does not affect the running component. You cannot delete running fabric interconnect or manager firmware (the manager maintains a “startup link” for these directly into the catalogue itself; the catalogue copy is the running copy). You can of course delete old unused (or new, never used) versions of fabric interconnect or manager software.

7-7 UCSI Boot Camp

© Cisco Systems, Inc.

“Equipment POV” Firmware Managemnt  Done “bottom up”  New install or “full sweep upgrade”  Adapter, BMC, IOM have:  Update (backup flash area)  Activate (swap backup / active flash next reboot)

 Fabric Interconnect (kernel, system) / Manager  Activate Only

© 2008 Cisco Systems Inc. All rights reserved.

UCS System Product Overview—data center and Server Trends 8

“Equipment POV” Firmware Managemnt When you upgrade firmware from the “equipment point of view” you will generally make sure upgrades are done “bottom up” (get new firmware running on adapters before BMC before IOM/fabric interconnect /manager). You are way more likely to get components that cannot communicate with new firmware at the higher (IOM, fabric interconnect) layers if you have not yet updated the firmware on the “leaves”. The adapter, BMC and IOM components are “special”. You always flash new firmware into an unused flash area. This operation (“update”) is non-interruptive. All adapters, BMC, and IOM’s in a large system could be “updated” simultaneously, since this flash operation only affects the backup flash. After the update the new firmware is “activated” by just setting a pointer so that when the component reboots the primary and backup flash areas just swap roles. The fabric interconnect components (kernel and system software) and the manager application itself have only an “Activate” operation and no “update”. This software basically just lives in a file system, and when you activate it, you set a pointer so that

© Cisco Systems, Inc.

UCS System Maintenance 7-8

on next reboot the component boots off the new firmware version by pointing straight into the catalogue.

7-9 UCSI Boot Camp

© Cisco Systems, Inc.

Upgrade (IOM, BMC, Adapter)

© 2008 Cisco Systems Inc. All rights reserved.

UCS System Product Overview—data center and Server Trends 9

“Equipment POV” Firmware Managemnt If you are doing a “clean sweep” firmware upgrade you can do the “update” portion for IOM/BMC/Adapter globally (all at the same time for all components); there is no danger in this since it is just flashing a backup flash area. From the “global form” available from the equipment view, or from various views of individual components, you could choose to also perform the update operation on a single or on chosen components.

© Cisco Systems, Inc.

UCS System Maintenance 7-10

Activate

© 2008 Cisco Systems Inc. All rights reserved.

UCS System Product Overview—data center and Server Trends 10

Activate The “Activate” operation will set a pointer so that the component boots the new flash version (“backup flash that was updated”, for IOM/BMC/adapter) on next boot. The default is that the UCSM operation will not only set the pointer but also reboot the component. It is possible to “Set Startup Version Only” and then the firmware will not actually be brought into use into you manually reboot a component.

7-11 UCSI Boot Camp

© Cisco Systems, Inc.

Typical “Full Sweep” Upgrade  Update all Adapter, BMC, IOM  Activate all Adapter (server reboot)  Activate all BMC  Activate all IOM, click button to “Set Startup”  Activate fabric-interconnect and MGR

© 2008 Cisco Systems Inc. All rights reserved.

UCS System Product Overview—data center and Server Trends 11

Typical “Full Sweep” Upgrade If there is a time when you will be upgrading all components to a specific firmware revision, you will typically:

• do all IOM/adapter/BMC “updates” (flash of backup area) simultaneously • Activate (and reboot) adapters first (yes, this is a blade server reboot) • Activate (and reboot) BMC (rebooting just BMC does not interrupt processing on the blade server) • Activate IOM, but tell it to just set the startup version, and not reboot • Activate fabric interconnect components and manager. If you have redundant fabric interconnects, they will reboot in a rolling fashion, and at the end, load your new manager.

© Cisco Systems, Inc.

UCS System Maintenance 7-12

Firmware Management Using Service Profile

© 2008 Cisco Systems Inc. All rights reserved.

UCS System Product Overview—data center and Server Trends 12

Firmware Management Using Service Profile Doing firmware upgrade from the service profile point of view is a completely different way of looking at things. This is a more “firmware by policy” point of view. A firmware package is a list of required firmware per component (a package can specify one (or zero) version of firmware per component: BIOS, Adapter firmware, Emulex FC firmware, LSI RAID firmware, Qlogic/Emulex option ROM firmware). If there is no version at all listed for a component, or no firmware package at all, it just means to keep the firmware as it is when service profile is associated. There are separate “management packages” for BMC firmware. A service profile or service profile template can point to one firmware package (having versions for one or more components) and one management (BMC) package. When the service profile is associated, it compares firmware versions on the blade being associated with those listed in the package, and “makes it so” (updates firmware as needed). Note for a component with the “update/activate” philosophy, the service profile association will do both the update and activate. 7-13 UCSI Boot Camp

© Cisco Systems, Inc.

Note that at this time there is no “firmware scrub” (no putting firmware back to some baseline when a service profile is disassociated. today, firmware just stays as it is on disassociation).

© Cisco Systems, Inc.

UCS System Maintenance 7-14

Firmware Packages (Host and Mgmt)

In SP/template

© 2008 Cisco Systems Inc. All rights reserved.

UCS System Product Overview—data center and Server Trends 13

Firmware Packages (Host and Mgmt) This is an example of a host firmware package (everything except BMC). The example has shown the package referring to versions of firmware for BIOS, RAID Controller, and Menlo-Q adapter. A service profile or SP template will then refer to the package by name. Just like all other policies, you can refer to a package name in the same org as an SP or in a higher org.

7-15 UCSI Boot Camp

© Cisco Systems, Inc.

Backup and Restore

© 2008 Nuova, Inc. All rights reserved.

© Cisco Systems, Inc.

ICNX5 v1.0—14

UCS System Maintenance 7-16

Backing up UCSM Configuration in the GUI

© 2008 Cisco Systems Inc. All rights reserved.

UCS System Product Overview—data center and Server Trends 15

Backing up UCSM Configuration in the GUI This screenshot shows where GUI access to the backup and import operations lie (highlighting the word “All” on the Admin Screen – it can be tricky. Details of the backup operation are on the next slide. Backup operations creates an “snapshot” of your system configuration and sends it to an external server. The “configuration” snapshots (everything except “full state”) can be restored “on the fly”. You can bring your configuration back to a known state (using “All configuration”) without having to reboot.

7-17 UCSI Boot Camp

© Cisco Systems, Inc.

Backup Operation

© 2008 Cisco Systems Inc. All rights reserved.

UCS System Product Overview—data center and Server Trends 16

Backup Operation Backup Task Backing up a UCS System starts with the creation of a “backup task”. A “backup task” is a managed object that defines all the criteria for backing up the configuration of the UCS. A “backup task” contains the following information:

•Type •Transmitting protocol •Host name where the back up is to be stored •Remote path and file name •User name to use on remote host •Password Administrative state Once you create the managed object that is the backup operation, to actually create the backup the administrative state of the backup must be changed to “enabled”. In the background, the state and/or configuration is collected (depending on the type of backup you perform), tarred, gzipped (for full state, the others are XML) and transferred to the backup server using the protocol you choose. You can follow the

© Cisco Systems, Inc.

UCS System Maintenance 7-18

workflow of the backup by examining the FSM associated with the backup. Once the backup is complete, its administrative state will automatically change to “disabled”. Backup types The types of backups you can create include the following:

•Full-state: this is a raw backup of the data. It must be imported on the exact same hardware (it restores all the hardware serial numbers and such), it is not for a hardware disaster but a configuration database disaster (this is a .tar.gz of the raw database; all the others are XML files)

•All configuration: this is the combination of “system configuration” and “logical configuration”

•Logical configuration: this is all configuration that is *not* associated with AAA (e.g. configured orgs, configured threshold policies, configured VLANs and VSANs in your LAN and SAN clouds respectively …etc, configured server, uplink and SAN ports).

•System Configuration: this is all configuration that specifically pertains to the AAA role (authentication, authorization, and accounting). (e.g. radius, ldap, tacacs, users, …etc). Protocols Your choice of protocols include FTP, TFTP, SCP, and SFTP. Note that you must choose a protocol that is running on the backup server and that you must provide user credentials that are valid on the backup server. Also note that the path name to remote file is *relative* to the directory where the protocol “puts” you upon “login” (despite the reference to a “fully qualified path name”). For example, if you select FTP and login anonymously, then you are not prompted for a password. The directory you “land in” depends on how the FTP server on that backup host is configured. If you select SCP, then you must provide valid user credentials and the directory you land in will be the home directory of that user. You can only create one backup configuration per backup server. Backups are identified in UCSM using the hostname or IP of the backup server in the configuration. Note that hostnames are resolved through the external DNS server defined as you set up the UCS manager (this can be changed on the Admin tab)

7-19 UCSI Boot Camp

© Cisco Systems, Inc.

Restore (import) Operation  Full State: imported at initial setup only  Other options:

© 2008 Cisco Systems Inc. All rights reserved.

UCS System Product Overview—data center and Server Trends 17

Restore (import) Operation Restore of a “full state” backup is done from the serial console only, as part of the “initial setup” dialogue (you can get back to this dialogue if you do (be careful) “erase configuration” from the local management shell.). The full raw database is just put back into place. Note if you really have different hardware (different chassis or blade serial numbers, for example), a full state restore is not what you want. Full state restore is for complete configuration data disaster only. For the other options, you can import the XML backup “on the fly”. If you use the “replace” option, all existing objects not in the saved file will be deleted (and blade servers will undergo a disassociation from any previous SP), and new data is inserted (and blade servers do undergo an association process). Note that often after import it looks like all your new associations are “failing”. In reality they are often just waiting for the disassociations from previous SP to complete, and then the new associations will succeed once blades become available.

© Cisco Systems, Inc.

UCS System Maintenance 7-20

Restoring Full State Backup at Switch Startup

© 2008 Cisco Systems Inc. All rights reserved.

UCS System Product Overview—data center and Server Trends 18

Restoring Full State Backup at Switch Startup This is the only way to restore the full state backup. Note that you must provide network configuration for the switch in order for it to go over the network to the backup server to retrieve the backup file. If you have a DHCP server, the process will try to use that as well. This IP is just for the restore itself, then the permanent static management IP is taken from the restore file itself.

7-21 UCSI Boot Camp

© Cisco Systems, Inc.

Restoring UCSM Configuration at Switch Startup (continued)

© 2008 Cisco Systems Inc. All rights reserved.

UCS System Product Overview—data center and Server Trends 19

Restoring UCSM Configuration at Switch Startup (continued) Note that when doing the restoration at switch startup (as opposed to at runtime), you must adhere to the following rules:

•The backup file you created is a full-state backup only. •The remote file path is an absolute path (from the root directory) to the backup file on the backup server (although this still could get interpreted in a “relative” fashion”, by a tftp or ftp server, for example)

© Cisco Systems, Inc.

UCS System Maintenance 7-22

Manage High Availability

© 2008 Nuova, Inc. All rights reserved.

7-23 UCSI Boot Camp

ICNX5 v1.0—20

© Cisco Systems, Inc.

Overview of HA in UCS System Management Node

UCSM Server Ports

IOM

Management Node

L1

L1

L2

L2

Mid-Plane

UCSM Server Ports

IOM

SEEPROM Chassis

© 2008 Cisco Systems Inc. All rights reserved.

UCS System Product Overview—data center and Server Trends 21

Overview of HA in UCS System Configuring two nodes in your UCS System can provide redundancy for both the UCSM functionality as well as the switching functionality that the management nodes provide. You are not required to run in HA mode, but if you do, you must follow certain rules. First, you must connect the L1 port of the “left” switch to that of the “right” switch and L2 of left to L2 of right. Second, you must connect the “left” FEX to the “left” switch (using between 1 and 4 server links) and connect the “right” FEX to the “right” switch. *Note that both fabric extenders share a serial EEPROM. This chip stores cluster state information and is used to resolve split brains (both partitions in time and in space). Each CMC maintains its own part of the shared chassis storage in the SEEPROM. Chassis storage contains a combination of “static” and “dynamic” information. For example, the static portion contains the nodeid for each node configured in the cluster. The dynamic portion contains a monotonically increasing integer which represents the version of the configuration as seen by that node. There is no need to replicate the contents of the SEEPROM. Each node maintains (writes to) its own portion while both nodes may read from both sections.

© Cisco Systems, Inc.

UCS System Maintenance 7-24

Split Brains  Partition in space – Total private network failure – Resolved by SEEPROM

 Partition in time – Node tries to start alone with old configuration – Resolved by SEEPROM

© 2008 Cisco Systems Inc. All rights reserved.

UCS System Product Overview—data center and Server Trends 22

Split Brains Partition in space A partition in space occurs when nodes fail to communicate with each other over the private network (L1 and L2 links both failed). To resolve this split brain (and assuming both switches are active at the time of the private network failure), each CMC will act on the behalf of their fabric’s UCSM instance to reach the serial eeprom first and write their node id in the primary field (known as a “quorum race”). The “winner” will remain in the cluster and the loser will abort. When the links are restored, the “losing” node can rejoin the cluster and act as subordinate. Partition in time A partition in time occurs when one of the nodes is down for a period of time during which changes to the configuration are made on the active primary node. These changes would have not been replicated to the down node (obviously). If the primary node were to shut down after having made configuration changes to the database but prior to being able to replicate them to the other (downed) node AND that downed node were to try to join the cluster alone, then that condition is referred to as a partition in

7-25 UCSI Boot Camp

© Cisco Systems, Inc.

time. To resolve this split brain, a version number that represents the configuration is written to the eeprom. On solo startup, a node compares its version to that of the other node (recall both nodes can read both parts of the EEPROM). If it is the same or higher, it can start the cluster. Otherwise, it will not, but it can be forced with the “cluster force” command.

© Cisco Systems, Inc.

UCS System Maintenance 7-26

Viewing and Changing Management HA  connect local-mgmt  show cluster extended-state myswitch-A# show cluster extended-state A: UP, PRIMARY B: UP, SUBORDINATE A: memb state UP, lead state PRIMARY, dme state: UP B: memb state UP, lead state SUBORDINATE, dme state: UP link state UP, heartbeat state PRIMARY_OKHA READY

 cluster lead  cluster force

© 2008 Cisco Systems Inc. All rights reserved.

UCS System Product Overview—data center and Server Trends 23

Viewing and Changing Management HA This is an example of viewing, and changing primary node assignment in the UCS system. myswitch-A# connect local-mgmt Cisco Nexus Operating System (NX-OS) Software TAC support: http://www.cisco.com/tac Copyright (c) 2002-2008, Cisco Systems, Inc. All rights reserved. The copyrights to certain works contained herein are owned by other third parties and are used and distributed under license. Some parts of this software may be covered under the GNU Public License or the GNU Lesser General Public License. A copy of each such license is available at http://www.gnu.org/licenses/gpl.html and http://www.gnu.org/licenses/lgpl.html myswitch-A# show cluster extended-state A: UP, PRIMARY B: UP, SUBORDINATE A: memb state UP, lead state PRIMARY, dme state: UP 7-27 UCSI Boot Camp

© Cisco Systems, Inc.

B: memb state UP, lead state SUBORDINATE, dme state: UP link state UP, heartbeat state PRIMARY_OK HA READY Chassis, serial: CHASSIS 81, ip: 127.5.1.254, state: active

The output of the ‘show cluster extended-state’ command displays the following information: •Running mode of each instance •Running mode of each DME •Status of private network •Status of UCSM HA •Chassis ID • IP address of primary IOM

Force or Lead? To suggest that this node be primary in the cluster, run the ‘cluster lead’ command. This command will attempt to promote this (subordinate) node to be the primary node. If the UCSM controller determines that this node is not “fit” to lead the cluster, then this command will abort its attempt. To force that this node be primary in the cluster, run the ‘cluster force’ command. This will override any information in the SEEPROM and UCSM logic that would otherwise contradict this node being elected primary.

© Cisco Systems, Inc.

UCS System Maintenance 7-28

Performing Other Maintenance Tasks

© 2008 Nuova, Inc. All rights reserved.

7-29 UCSI Boot Camp

ICNX5 v1.0—24

© Cisco Systems, Inc.

Setting the Time  Switch: clock set month day year hour min sec  CMC gets date from switch  BMC gets date from switch

© 2008 Cisco Systems Inc. All rights reserved.

UCS System Product Overview—data center and Server Trends 25

Setting the Time When installing a switch for the first time (or if you don’t run NTP on the switch), you may need to set the system clock. This clock is used as a reference clock for both the CMC and BMC so that all log file entries in a UCS System may be synchronized. You could also configure the switch as an NTP client if you wish to synchronize all the UCS System system clocks with an external time server. If you don’t configure NTP, then your system clocks will likely drift. You can also set the time and time zone as follows: ourswitch-A# scope system ourswitch-A# scope services ourswitch-A /services # set clock jan 5 2009 13 52 0 ourswitch-A /services # set timezone America/Los_Angeles ourswitch-A /services* # commit buffer You can also set the time zone from within the GUI.

© Cisco Systems, Inc.

UCS System Maintenance 7-30

Copying Scripts to UCS Manager

© 2008 Cisco Systems Inc. All rights reserved.

UCS System Product Overview—data center and Server Trends 26

Copying Scripts to UCS Manager You can write UCSM CLI scripts on another host, copy them to the /workspace directory of the management software using SCP, FTP, or TFTP, and then run them from the ‘connect local-mgmt’ interface of UCSM. Note that the ‘cp’ command will detect the existence of files by the same name and will prompt you if you wish to “clobber” it or not. You cannot copy scripts from within the GUI. *Note: you must be in the local-mgmt shell to perform these tasks.

7-31 UCSI Boot Camp

© Cisco Systems, Inc.

Example Script Usage  Save “show config” output for a Service Profile # scope org / # show config >scp://user1@mysrvr/home/user1/SP_config.out

1. Edit config file on server (fix, add scoping) 2. Copy script back to fabric interconnect 3. Delete service profile from original org 4. Use script to recreate SP in a different org

© 2008 Cisco Systems Inc. All rights reserved.

UCS System Product Overview—data center and Server Trends 27

Example Script Usage One idea for accomplishing something useful using the script interface is to effectively “move” as service profile. As shown, you could execute “show config” in the scope of the service profile to get nearly the set of command used to recreate the profile. You may have edit the script on an external server (often commands like “associate” are interspersed in “create” commands; whereas you can’t really associate a service profile until it is created. Note also, when you run the script it will always run from the top scope (you run it from the local-mgmt shell). You can put explicit scope commands in the script itself to make it “move” to the correct places)

© Cisco Systems, Inc.

UCS System Maintenance 7-32

Erasing Configuration  erase configuration – Removes the database and UCSM configuration – Automatically reboots switch – Prompts user to configure UCSM at startup

© 2008 Cisco Systems Inc. All rights reserved.

UCS System Product Overview—data center and Server Trends 28

Erasing the configuration Erasing the configuration (from the ‘local-mgmt’ interface) erases the database and removes the UCSM configuration. This command will automatically reboot the fabric and prompt the user to configure UCSM from scratch. Note this will not affect the other fabric interconnect in an HA configuration. If you want, you could run “erase configuration” on both.

7-33 UCSI Boot Camp

© Cisco Systems, Inc.

Using External Management Protocols

© 2008 Nuova, Inc. All rights reserved.

ICNX5 v1.0—29

In this section, you will identify each of the supported management protocols that can be used to gather information from outside a UCS system.

© Cisco Systems, Inc.

UCS System Maintenance 7-34

SNMP

© 2008 Cisco Systems Inc. All rights reserved.

UCS System Product Overview—data center and Server Trends 30

SNMP UCS supports SNMP v3 read-only monitoring of the UCS fabric interconnect as defined in the MIBs for the Nexus 5000 product running 4.0.(0)(N1)(1a). There are approximately 60 different MIBs defined for the NX5K (e.g. AAA, CallHome, STP, VLAN, and FC device aliases). You can have UCS send SNMP traps to define trap hosts. Once again, as of the current release, trap information is related only to the MIB’s defined for the switch component of the fabric interconnect.

7-35 UCSI Boot Camp

© Cisco Systems, Inc.

Syslog

© 2008 Cisco Systems Inc. All rights reserved.

UCS System Product Overview—data center and Server Trends 31

Syslog Syslog is a traditional UNIX logging facility that can be configured to send messages from different programs (or “facilities”) of varying severity (e.g. info, warn, crit, emerg) to the console, to a file, or to an external host (aka “loghost”). Syslog contains several built-in facilities such as kern (or kernel), print (i.e. print server), and mail (i.e. SMTP server). Users can also configure their own custom facilities for applications that don’t fall neatly into one of the predefined facilities. These facilities are named local0, local1, … etc.

In UCS, you can configure syslog to report to a local destination or a remote destination. Local destination configuration includes •Console •Monitor •file Remote destination configuration (for up to 3 log hosts) includes •Level

© Cisco Systems, Inc.

UCS System Maintenance 7-36

•Hostname •facility

7-37 UCSI Boot Camp

© Cisco Systems, Inc.

Serial Over LAN

© 2008 Cisco Systems Inc. All rights reserved.

UCS System Product Overview—data center and Server Trends 32

Serial Over LAN Serial Over LAN is a mechanism that enables the input and output of the serial port of a managed system to be redirected via an IPMI session over IP (i.e. using TELNET or SSH). In UCS, the serial ports on the blades are not connected to a traditional serial port interface. To allow users to access applications on the servers via the serial port, I/O of the serial port is redirected to the BMC and made available to external users on the management network. For example a user wishing to access a blade server that is running Linux via the serial port can ssh to the management network IP address associated with that blade and log in. On the blade server the login will be seen as coming through the serial port. In UCS, this is configured through a Serial Over LAN policy, which defines •Enable or disable the service •Baud rate for terminal sessions

© Cisco Systems, Inc.

UCS System Maintenance 7-38

IPMI

© 2008 Cisco Systems Inc. All rights reserved.

UCS System Product Overview—data center and Server Trends 33

IPMI IPMI operates independently of the operating system and allows administrators to manage a system remotely even in the absence of an operating system or even if the monitored system is powered off, but connected to a power source. IPMI can also function after the operating system has started. IPMI prescribes only the structure and format of the interfaces as a standard, while detailed implementations may vary. IPMI can send out alerts via a direct serial connection, a LAN or a SOL connection to a remote client. System administrators can then use IPMI messaging to query platform status, to review hardware logs, or to issue other requests from a remote console through the same connections. The IPMI consists of the BMC and other satellite controllers. The satellite controllers within the same chassis connect to the BMC via the system interface called IPMB (Intelligent Platform Management Bus/Bridge) — an enhanced implementation of I2C (Inter-Integrated Circuit). The BMC connects to satellite controllers or another BMC in another chassis via IPMC (Intelligent Platform Management Chassis) bus/bridge. It may be managed with the Remote Management Control Protocol (RMCP), a specialized wire protocol defined by this specification. A FRU holds the inventory (such as vendor id, manufacturer etc.) of potentially replaceable devices. A Sensor Data Records (SDR) repository provides the properties of 7-39 UCSI Boot Camp

© Cisco Systems, Inc.

the individual sensors present on the board. For example, the board may contain sensors for temperature, fan speed, and voltage.

© Cisco Systems, Inc.

UCS System Maintenance 7-40

SMASH-CLP

Hardware components of chassis 1 include 8 fan modules, 2 FEXs, 4 PSUs, and 4 servers

© 2008 Cisco Systems Inc. All rights reserved.

UCS System Product Overview—data center and Server Trends 34

SMASH-CLP The Systems Management Architecture for Server Hardware (SMASH) is a suite of specifications that deliver industry standard protocols to manage elements of a data center. The SMASH Command Line Protocol specification provides an interface to heterogeneous servers independent of server hardware, operating system, or network protocol method. It is a standard method for local and remote management of server hardware using out-of-band communication. SMASH is developed by the DMTF Server Management Working Group (SMWG). With SMASH CLP-enabled products, users can execute common operations (such as system power on and off, system log display, boot order configuration and text-based remote console) using the same commands across different vendor platforms. In UCS, you connect to the “clp” shell from the CLI to issue SMASH commands. Use ‘cd’ to traverse the information hierarchy and use ‘show’ to display values at different nodes in the information hierarchy.

7-41 UCSI Boot Camp

© Cisco Systems, Inc.

Call-Home

© 2008 Cisco Systems Inc. All rights reserved.

UCS System Product Overview—data center and Server Trends 35

Call-Home UCS Manager 1.0 uses the CallHome functionality as defined in the Nexus 5000. CallHome creates email messages in either XML or standard text format and sends the messages to predefined recipients when certain events occur. Configurable profiles define

•criticality levels (e.g. debug, warning, fatal, …) •email format (xml or txt) •email recipients In UCS, you can enable CallHome policies to generate notifications on thermal, voltage, power, identity, fru, and equipment events. You can send also full inventory data at the push of button or schedule it to be sent on a periodic basis. The “general” page of the call home is the setup for all your “From” information. You can provide fake information here if you are just testing. The exception is just the bottom part where you list the SMTP server to which you want to send information. Note this does not accept an SMTP server that authenticates.

© Cisco Systems, Inc.

UCS System Maintenance 7-42

Call Home Profiles

© 2008 Cisco Systems Inc. All rights reserved.

UCS System Product Overview—data center and Server Trends 36

Call Home Profiles Call home profiles allow you to establish a list of email addresses to which to send messages on events that match certain categories, or “alert groups”. CiscoTac is a special “alert group” that matches all critical events, as well as the sending of inventory. The other “alert group” that matches inventory setting is the “diagnostic” group (not the inventory group; this matches inventory events --- insertions and removals of hardware).

7-43 UCSI Boot Camp

© Cisco Systems, Inc.

Call Home Policy

© 2008 Cisco Systems Inc. All rights reserved.

UCS System Product Overview—data center and Server Trends 37

Call Home Policy The policies allow you to enable and disable call-home for certain categories of problems (fru-problem, equipment-inoperable, and the like)

© Cisco Systems, Inc.

UCS System Maintenance 7-44

Call Home – Send Inventory

© 2008 Cisco Systems Inc. All rights reserved.

UCS System Product Overview—data center and Server Trends 38

Call Home – Send Inventory You can periodically send inventory, or click to send on a “one time basis”. A full “smart call home” XML email is sent to email addresses in call-home profiles that match the “CiscoTac” or “diagnostic” alert groups at “normal” severity or below. The intention is that this email get digested by an artificial intelligence at Cisco TAC, who will file the customer’s entire inventory in a database. When a customer calls, Ciso TAC can pull up information and see exactly what the customer’s inventory is. The call-home functionality itself, however, is just an email.

7-45 UCSI Boot Camp

© Cisco Systems, Inc.