lag(7)lag(7)NAMElag - Link aggregation (also called trunking) introductory information
DESCRIPTION
Link aggregation, or trunking, enables administrators to combine two or
more physical Ethernet Network Interface Cards (NICs) and create a sin‐
gle virtual link. (Upper-layer software sees this link aggregation
group as a single virtual interface.) The single virtual link can carry
traffic at higher data rates than a single interface because the traf‐
fic is distributed across all of the physical ports that make up the
link aggregation group.
Using link aggregation provides the following capabilities: Increased
network bandwidth - The increase is incremental based on the number and
type of ports, or Network Interface Cards (NICs), added to the link
aggregation group. See the "Load Sharing" section for more information.
Fault tolerance - If a port in a link aggregation group fails, the
software detects the failure and reroutes traffic to the other avail‐
able ports. See the "Fault Tolerance" section for more information.
Load sharing - Traffic is distributed across all ports of a link aggre‐
gation group. See the "Load Sharing" section for more information.
You can use a link aggregation group virtual interface for the follow‐
ing point-to-point connections: server-to-server and server-to-switch.
For server-to-switch connections, the switch must support link aggrega‐
tion. See your switch documentation for information on configuring your
switch.
Link aggregation requires an optional kernel subsystem (lag.mod). You
can verify the presence of the link aggregation subsystem by issuing
the sysconfig -slag command. If the lag subsystem is not loaded, you
can load it using either of the following methods: Dynamically load it
using the sysconfig -clag command. This method does not persist
across system reboots. Edit the system configuration file, add an
options LAG entry to it, and build a new kernel by issuing the doconfig
command. Then, reboot the system. This method loads the subsystem each
time the system reboots.
After the subsystem is loaded, you can configure a link aggregation
group.
Link Aggregation Configuration
You can configure link aggregation groups either in multiuser mode or
at boot time with the lagconfig command. When you configure the group,
you can specify a virtual interface number, a key, a distribution algo‐
rithm, and a Media Access Control (MAC) address.
After you create a link aggregation group, you can then enable ports
(interfaces) for link aggregation. The enabled ports attach to the
link aggregation group with the corresponding key. If the port fails
in some way, the port detaches from the group and traffic is rerouted
to the remaining port or ports.
Any link aggregation configuration done in multiuser mode does not per‐
sist across system reboots. If you want link aggregation groups con‐
figured at boot time, you must include the appropriate lagconfig and
ifconfig commands in the /etc/inet.local file. See the Network Admin‐
istration: Connections manual for an example.
On platforms where I/O bandwidth may be a limiting factor, you might
increase link aggregation performance by distributing the NICs across
different portions of the I/O infrastructure (for example, different
PCI buses).
Fault Tolerance
The link aggregation subsystem monitors the link state of ports that
are enabled for link aggregation. When the link aggregation subsystem
detects that a port's link state is down, the subsystem detaches the
port from its link aggregation group and redistributes traffic among
the remaining ports.
When the link aggregation subsystem detects that the port's link state
is up, the subsystem reattaches the port to its link aggregation group.
The port then starts handling part of the traffic load again. The
amount of time it takes to detect a link state change and fail over
depends on the device and driver in use. For DE60x devices using the ee
driver, average failover times are typically 1 to 2 seconds. To achieve
faster failover, reduce the value of the ee subsystem link_check_inter‐
val attribute. A value of 20 (0.2 seconds) typically provides average
failover times of 0.1 to 0.2 seconds.
For DEGPA devices using the alt driver, average failover times are less
than 1 second.
Load Sharing
A link aggregation group performs load sharing of both inbound and out‐
bound traffic. Distribution of inbound packets is determined by the
server or switch to which the link aggregation group is connected.
When transmitting packets, the system uses a load distribution algo‐
rithm to determine on which attached port to transmit the packets. The
following load distribution algorithms are supported: For IP packets,
the port is selected based on a hash of the destination IP address.
For non-IP packets, the port is selected based on a hash of the desti‐
nation MAC address. All traffic addressed to a specific destination IP
address uses the same port in the link aggregation group.
This algorithm can utilize the combined bandwidth of a link
aggregation group in environments where traffic is destined to a
large number of different IP addresses (for example, a web
server).
However, this algorithm might not produce the expected bandwidth
utilization in environments where the majority of traffic is
destined to a single IP address (for example, a private server-
to-server interconnect). The port is selected based on a hash
of the destination MAC address. All traffic addressed to a spe‐
cific destination MAC address uses the same port in the link
aggregation group.
This algorithm can utilize the combined bandwidth of a link
aggregation group in environments where traffic is destined to a
large number of different MAC addresses (for example, a server
that sends most of its traffic to clients on the same LAN).
However, this algorithm might not produce the expected bandwidth
utilization in environments where the majority of traffic is
destined to a small number of MAC addresses (for example, a
server-to-server interconnect, or a server that sends most of
its traffic through a router). For TCP or UDP packets originat‐
ing on the system, the port is selected based on a hash of the
source and destination TCP or UDP port numbers. For all other
packets, including TCP and UDP packets being forwarded by the
system, the Destination IP address (dstip) algorithm is used.
All traffic addressed to a specific source+destination port pair
uses the same port in the link aggregation group.
This algorithm can utilize the combined bandwidth of a link
aggregation group in environments where traffic is destined to a
single IP or MAC address, but is exchanged between a number of
different TCP or UDP port number pairs (for example, a server-
to-server interconnect). The port is selected on a rotating
basis.
This algorithm can utilize the combined bandwidth of a link
aggregation group in most environments.
However, this algorithm may result in reordering of packets
belonging to the same flow (for example, a TCP connection),
which in turn may adversely affect performance.
RESTRICTIONS
The following restrictions apply: Supports only DEGPA (alt), DEGXA
(bcm), and DE60x (ee) network interface cards (NICs). Supports only
Ethernet (802.3 CSMA/CD) links. NetRAIN virtual interfaces cannot be
included in link aggregation groups. Ports must be operating in full
duplex mode. Ports in the same link aggregation group must operate at
the same data rate. Ports in a link aggregation group must be attached
to the same system, either server-to-server or server-to-switch.
SEE ALSO
Commands: lagconfig(8)
System Attributes: sys_attrs_ee(5), sys_attrs_lag(5)
Files: inet.local(4)
Technical Overview
Network Administration: Connections
lag(7)
