Solaris Network Aggregations

First Alternative Aggregations

Solaris Network Aggregations Aggregations with dladm •

•

An aggregation is similar to a link created for failover, but has other advantages:*

An aggregation is treated as one device for configuration and management, and uses only one IP address.

*

In-bound as well as outbound load spreading is provided, so extra bandwidth is available.

*

Automatic failover is provided, to the remaining working links in the aggregation.

*

Policies can be applied to the traffic flows.

*

Parallel aggregations between servers can be created, without going through a switch.

To show the available network interfaces:-

First Alternative

# dladmin show-link eri0 type: legacy mtu: 1500 qfe0 type: legacy mtu: 1500 qfe1 type: legacy mtu: 1500 qfe2 type: legacy mtu: 1500 qfe3 type: legacy mtu: 1500 •

device: eri0 device: qfe0 device: qfe1 device: qfe2 device: qfe3

Early versions of Solaris 10 do not have this facility - you would need to download the Sun Trunking software to get it.

© First Alternative 2008

1 of 6

SMC

First Alternative Aggregations

IPMP (Multipathing) and Aggregation Aggregations - how to create •

•

There are conditions to consider:*

If you need to run an aggregation through a switch, that switch must support aggregation technology.

*

An interface that has been plumbed cannot become a member of an aggregation.

*

Interfaces must be of the GLDv3 type: xge, e1000g, and bge.

*

NOTE: Later releases of Solaris 10 will support aggregations on what are now considered legacy devices such as eri. qfe and hme.

*

All interfaces in the aggregation must run at the same speed and in full duplex mode.

To set up an aggregation using dladm, the basics steps are:-

# dladm create -aggr -d bge0 -d bge1 1 * Create an aggregation number 1 from interfaces bge0 and bge1.

First Alternative

# ifconfig aggr1 plumb 192.168.200.98 up * Brings the aggregation interface up. # dladm show-aggr key: 1 (0x0001) policy: L4 address: 0:3:ba:7:84:5e (auto) device address speed duplex link bge0 0:3:ba:7:84:5e 1000 Mbps full up bge1 0:3:ba:7:84:5e 1000 Mbps full up

© First Alternative 2008

2 of 6

state attached attached

SMC

First Alternative Aggregations

IPMP (Multipathing) and Aggregation Aggregations - Modifying •

To add another interface to the aggregation:-

# dladm add-aggr -d bge2 -d bge3 1 # dladm show-aggr 1 key: 1 (0x0001) policy: L4 address:0:3:ba:7:84:5e (auto) device address speed duplex link state bge0 0:14:4f:1:c8:b0 1000 Mbps full up attached bge1 0:14:4f:1:c8:b3 1000 Mbps full up attached bge2 0:14:4f:1:c8:b1 1000 Mbps full up attached bge3 0:14:4f:1:c8:b2 1000 Mbps full up attached

•

To remove a device from the aggregation:-

# dladm remove-aggr -d bge0 1 # dladm show-aggr 1

First Alternative

key: 1 (0x0001) policy: L4 address: 0:3:ba:7:84:5e (auto) device address speed duplex link state bge1 0:14:4f:1:c8:b3 1000 Mbps full up attached bge2 0:14:4f:1:c8:b1 1000 Mbps full up attached bge3 0:14:4f:1:c8:b2 1000 Mbps full up attached

© First Alternative 2008

3 of 6

SMC

First Alternative Aggregations

IPMP (Multipathing) and Aggregation Aggregations - Parallel (Back-to-Back) •

A parallel, or back-to-back, aggregation can be created between servers without the need for a switch. Local Network

bge0-3

bge0-3 Aggregation aggr1 honda

suzuki

Systems honda and suzuki are connected via an aggregation on interfaces bge0-3, and also to the local network through another interface. This is most commonly used between mirrored database servers to provide greater bandwidth between the servers themselves for database updates.

First Alternative

•

© First Alternative 2008

4 of 6

SMC

First Alternative Aggregations

IPMP (Multipathing) and Aggregation Aggregations - Policies and Zones •

It is possible to set policies for aggregations, such as whether to use LACP protocol or not.

•

The Sun documentation has some examples, but no explanations of what benefits changing policies may provide.

•

The most likely reason to change the policies would be the way the network switch handles aggregations, so we would recommend examination of the switch documentation.

•

Here is an example which sets LACP mode to active, and the timer interval to short.

First Alternative

# dladm modify-aggr -l active -t short 1 •

The default outbound load spreading is based on ports, but this can also be changed - see the dladm man page for full details.

•

Aggregations can be assigned to zones using IP instances (Update 4 onwards) - see zonecfg for details.

•

dladm can assign an aggregation temporarily to a zone, until the next reboot:-

# dladm set-linkprop -t -p zone=apache aggr1 * The -t option must be used. *

© First Alternative 2008

Use zonecfg to assign aggr1 permanently to the apache zone.

5 of 6

SMC

First Alternative Aggregations

IPMP (Multipathing) and Aggregation Aggregations - Interface Statistics •

dladm also provides functions to display statistics:-

# dladm show-dev -s eri0 eri0

ipackets 46712

rbytes ierrors opackets obytes oerrors 5010002 0 3208 219095 0

# dladm show-aggr -s key: 1 ipackets rbytes opackets obytes %ipkts %opkts Total 158030 31173663 12 504 qfe1 132878 26014941 12 504 84.1 100.0 qfe2 8818 1818890 0 0 5.6 0.0 qfe3 8174 1671697 0 0 5.2 0.0 qfe0 8160 1668135 0 0 5.2 0.0

Note the unbalanced loading on the aggregation - no doubt because it had been only lightly loaded - no other aggregations were present during the testing of the above link.

First Alternative

•

© First Alternative 2008

6 of 6

SMC