Ether Channel

Configuring Network Adapters for Redundancy Audience: System Administrators Date: October 15, 2001 You can protect against network adapter failure using AIX's Etherchannel support. Etherchannel allows you to create a "logical" ethernet adapter out of 2 physical adapters. If the primary network adapter fails, AIX seemlessly moves network traffic to the surviving adapter using the same IP address. The Etherchannel function is available in AIX 4.3.3+. Documentation is scarce, so here's a quick "how to". In this example, we assume we have two physical adapters (ent1 and ent2) that we want to group as a logical interface: smit => devices => communications => etherchannel =>"Add an Etherchannel" (fastpath "smit etherchannel") Add An Etherchannel Type or select values in entry fields. Press Enter AFTER making all desired changes. Etherchannel Adapters Enable ALTERNATE ETHERCHANNEL address ALTERNATE ETHERCHANNEL address Mode Internet Address to Ping Number of Retries Retry Timeout (sec)

[Entry Fields] ent1 ent2 no [] netif_backup [] [] []

This will create a "logical" interface using the next available adapter name ("ent3"). From here, configure an IP address on the logical interface ent3.

Comments: ALTERNATE ETHERCHANNEL = yes allows you to specify a "MAC" address. (Not necessary) Mode = round-robin provides higher aggregate bandwidth by distributing outgoing packets over multiple network adapters. However "round-robin" does not distribute incoming packets, nor does it provide adapter failover. Use "netif_backup" if you want failover. Internet Address to Ping: IP address for "keep alive". Optional as most failures detected at link layer Bruce Spencer, [email protected]

EtherChannel Readme File Etherchannel Introduction Etherchannel is an aggregation technology that allows you to produce a single large pipe by combining the bandwidth of multiple ethernet adapters. The aggregated pipe appears to be single ethernet interface to the upper layers (IP, etc.). Each of the ethernet adapters will share a single MAC address. There is a single ethernet interface associated with the aggregated pipe, which has a single IP address as well. Hardware The ethernet adapters inside which are members of the etherchannel can be any supported ethernet adapter. The ethernet adapters must be connected to a switch which supports Etherchannel . Configuration In AIX 4.3.3, a single etherchannel can combine either two or four ethernet adapters in the channel. You can configure multiple etherchannels on a single system. Be aware that each etherchannel constitutes an additional ethernet interface. The "no" option, ifsize (default 8), needs to be increased to include not only the ethernet interfaces for each adapter, but also any etherchannels that are configured. See the "no" command documentation in regards to modifying ifsize. The adapters in an etherchannel need to be configured to the same speed (10 or 100 Mbps) and mode (half or full duplex). I.Configuring Etherchannel

Etherchannel is configured through smit. I will use the following setup to show how etherchannel is configured: there will be 2 ethernet adapters, ent0 and ent1, in the etherchannel and there is no etherchannel alternate address specifed. There are 5 ethernet adapters to choose from. Execute: # smitty etherchannel

which produces a screen that looks like: Etherchannel Move cursor to desired item and press Enter.

List All Etherchannels Add An Etherchannel Change / Show Characteristics of an Etherchannel Remove An Etherchannel

Then choose "Add An Etherchannel" selection: Etherchannel Move cursor to desired item and press Enter. List All Etherchannels Add An Etherchannel Change / Show Characteristics of an Etherchannel Delete An Etherchannel -------------------------------------------------------------------------| Available Network Adapters | | | | Move cursor to desired item and press F7. | | ONE OR MORE items can be selected. | | Press Enter AFTER making all selections. | | | | > ent0 | | > ent1 | | ent2 | | ent3 | | ent4 | | | | F1=Help F2=Refresh F3=Cancel | | F7=Select F8=Image F10=Exit | F1| Enter=Do /=Find n=Find Next | F9 --------------------------------------------------------------------------

As instructed, select one more more of the listed adapters. The "Available Network Adapters" displays all of the ethernet adapters. If one selects an ethernet adapter that is already being used (i.e. has an interface defined), then one will get an error similar to:

NOTE Method error (/usr/lib/methods/cfgech):

0514-001 System error: Method error (/usr/lib/methods/chgent): 0514-062 Cannot perform the requested function because the specified device is busy. Add an Etherchannel Type or select values in entry fields. Press Enter AFTER making all desired changes. Etherchannel Adapters Enable ALTERNATE ETHERCHANNEL address ALTERNATE ETHERCHANNEL address Mode

[Entry Fields] ent0 ent1 no [] standard

+ + + +

Enable GIGABIT ETHERNET JUMBO frames Internet Address to Ping Number of Retries Retry Timeout (sec)

no [] [] []

After hitting the enter key: COMMAND STATUS Command: OK

stdout: yes

stderr: no

Before command completion, additional instructions may appear below. ent5 Available

Since an alternate address was not specified, the config method for the etherchannel changed the alternate address of ent1 to the NOTE hardware address of ent0. If an alternate address was specified, both ent0 and ent1 would have that alternate address. Then add an interface, en5 for standard ethernet or et5 for IEEE 802.3, through the normal means. The easiest way is to use the "Minimum Configuration & Startup" TCP/IP smit menu. II.Listing Etherchannels

Execute: # smitty etherchannel

and choose the "List All Etherchannels" selection: Etherchannel Move cursor to desired item and press Enter. List All Etherchannels Add An Etherchannel Change / Show Characteristics of an Etherchannel Remove An Etherchannel

COMMAND STATUS Command: OK

stdout: yes

stderr: no

Before command completion, additional instructions may appear below. ent5

Available

III.Changing the alternate address

Etherchannel

+ # #

Execute: # smitty etherchannel

Then choose "Change / Show Characteristics of an Etherchannel" selection: Etherchannel Move cursor to desired item and press Enter. List All Etherchannels Add An Etherchannel Change / Show Characteristics of an Etherchannel Remove An Etherchannel

-------------------------------------------------------------------------| Available Etherchannels | | | | Move cursor to desired item and press Enter. | | | | ent5 | | | | F1=Help F2=Refresh F3=Cancel | | F8=Image F10=Exit Enter=Do | F1| /=Find n=Find Next | F9 --------------------------------------------------------------------------

To change the Alternate etherchannel address to 1234deadbeef: Change / Show Characteristics of an Etherchannel Type or select values in entry fields. Press Enter AFTER making all desired changes. [Entry Fields] ent5 yes [0x1234deadbeef]

Etherchannel Enable ALTERNATE ETHERCHANNEL address ALTERNATE ETHERCHANNEL address

After hitting the enter key: COMMAND STATUS Command: OK

stdout: yes

stderr: no

Before command completion, additional instructions may appear below. ent5 changed

+ + +

If one wants to add or delete an ethernet adapter from the etherchannel, one must use the "Remove an Etherchannel" smit NOTE menu selection, then re-add the etherchannel with the "Add An Etherchannel" smit menu selection. Unconfiguring the Etherchannel Execute: # smitty etherchannel

Then choose "Remove an Etherchannel" selection: Etherchannel Move cursor to desired item and press Enter.

Choose 'ent5', then hit the enter key: COMMAND STATUS Command: OK

stdout: yes

stderr: no

Before command completion, additional instructions may appear below. ent5

deleted

ent0 changed ent1 changed

Troubleshooting tcpdump and trace can be used to troubleshoot the etherchannel. The trace hook id for the transmit packets is 2FA and for other events is 2FB. You cannot trace receive packets on the channel as a whole, but you can trace each of the adapter's receive trace hooks.

The entstat command will give the aggregate statistics of all the adapters in an etherchannel. If entstat is run with the "-d" option, the statistics of each of the individual adapters is printed. Restrictions Remote dump is not supported over an etherchannel. IV.Modes

There are three modes for Etherchannel. Each control the method the channel uses to send and receive packets.

Standard In this mode the channel will hash on the destination IP address and use this value to choose which adapter it will send the packet out on. This mode will guarantee packets are sent out over the channel in the order they were sent, but may not make full use of the bandwidth. Round Robin In this mode the channel will rotate through the adapters, giving each adapter one packet before repeating. In this mode packets may be sent out in a slightly different order than they were given to the channel, but the channel will make the best use of its bandwidth. Network Interface Backup In this mode, the channel will only activate one adapter at a time. The intention is that the adapters are plugged into different ethernet switches, each of which is capable of getting to any other machine on the subnet/network. When a problem is detected, either with the direct connection, or through inability to ping a machine, the channel will deactivate the current adapter, and activate a backup adapter. The channel will poll the adapter for Link Status in "Retry Timeout" intervals. If the Link Status is not up (either due to a cable being unplugged, switch down, or device driver problem) , the channel will switch to another adapter. This mode is the only one that makes use of the "Internet Address to Ping", "Number of Retries", and "Retry Timeout" fields. V.Internet Address to Ping The address will be ping'd if the address field has a non-zero address and the mode is set to netif_backup. If the channel is unable to ping the address for the "Number of Retries" times in "Retry Timeout" intervals. The channel will switch adapters. VI.Number of Retries The number of retries is the number of ping response failures before the channel switches adapters. The default is 3 times. VII.Retry Timeout The retry timeout is the interval in seconds between the times when the channel will send out a ping packet and poll the adapter's Link Status. The defauls is 1 sec intervals.

"Etherchannel" is a registered trademark of Cisco Systems, Inc.