AOE(3)AOE(3)NAMEaoe - ATA-over-Ethernet (AoE) interface
bind -a #æ /dev
The AoE (ATA-over-Ethernet) interface serves a three-level directory
providing control and access to AoE targets. The interface provided is
primarily intended for low-level control of the AoE initiator. See
sdaoe(3) for the standard interface.
In order to access AoE targets, one or more Ethernet controllers need
to be bound to the AoE initiator. By default, the system starts with
no interfaces bound. For automatic binding of interfaces on boot, the
aoeif configuration variable is set in plan9.ini(8). Ethernet inter‐
faces are specified as ethern, not as #ln. To bind the first and sec‐
ond Ethernet devices on boot, add
To bind ether1 to a running system:
% echo bind '#l1/ether1' >/dev/aoe/ctl
And to unbind it
% echo unbind '#l1/ether1' >/dev/aoe/ctl
When an interface is unbound, targets depending on that interface are
Each local interface is called a netlink. The mapping of AoE targets
to netlinks is called a devlink. Each devlink may see multiple inter‐
faces per target. For example, if the local machine has one Ethernet
address bound and the target has two interfaces on the same Ethernet
segment, this will result in one netlink and one devlink with two Eth‐
ernet addresses. AoE frames are sent in round-robin fashion. Each
successive frame is sent on the next address available on the next
available devlink (local interface).
Normally the initiator automatically discovers and adds new device
directories on startup. New devices are not added except as new inter‐
faces are bound to the initiator. Several messages can be written to
/dev/aoe/ctl which alter this behavior:
If toggle is absent, the state of autodiscover is toggled. If
it is the string on, it is turned on. Any other string turns
autodisover off. This option is not useful after Ethernet
devices have been bound.
Attempt to find the named target on all bound interfaces.
The converse of discover: remove the named target if it exists.
Allow or disallow rediscovery. This allows for automatic dis‐
covery of new targets. Unfortunately, it also allows automatic
modification or loss of existing targets. This option is con‐
Reading /dev/aoe/ctl returns a list of colon-separated lines with key‐
words and their values:
Returns the current state of the variable named by the keyword.
Writing the variable's name to the control file toggles the
state of that variable.
Path to nth bound Ethernet device.
ifn ea Ethernet address of this device.
A flag of ``Up'' indicates that this interface is available.
Number of consecutive lost jumbograms.
Incorrect and unused.
Once configured, each AoE target is accessed via files in the directory
named for its shelf and slot. For example, shelf 42, slot 0 would be
accessed through the path The ident file contains the read-only, verba‐
tim result of the identify unit ATA command. The config file contains
the target's AoE configuration string. Writing to this file sets the
targets configuration string.
Reading a target's ctl file returns a list of colon-separated lines
with the following keywords and values:
state ``Up'' or ``down''.
nopen Number of clients using this target.
nout Number of outstanding AoE frames.
Maximum number of outstanding frames allowed.
Maximum number of outstanding frames. Nframes is greater than
nmaxout when the initiator is reducing the number of in-flight
frames due to packet loss. It is assumed that packet loss is
due to an overwhelmed target and not poor network conditions.
Maximum number of data bytes per AoE frame. Using standard
frames, maxbcount is 1024 or two sectors. AoE ATA headers are
The respective fields from the ATA identify unit command.
flag List of flags useful for debugging. The flag jumbo indicates
that jumbo frames are accepted, not that they are being used.
Maxbcount should be consulted for this purpose.
Writing to the ctl file, the following commands may be issued:
failio fail outstanding i/o.
send an ata command to the target.
set the maximum number of block sent per packet.
mtu n set the maximum number of bytes (including header) sent per
nofail never fail this target. This is useful if your root device is
on this target.
with no arguments, reset the device size to the size claimed.
Otherwise, assume the device is the given size.
The data file may be read or written like a normal file except that
reads and writes to this file are converted to AoE commands to the tar‐
get, so transfers should be 512 or 1024 bytes long (or a larger multi‐
ple of 512 iff jumbo packets are in use). The size of this file is the
usable size of the target.
The devlink directory contains one file for each interface the target
was discovered on. The files are numbers from 0 to n and contain a
list of colon-separated lines with keywords and their values:
addr A space-separated list of the target's Ethernet addresses visi‐
ble from this interface.
npkt The number of frames sent on this interface.
resent The number of frames re-sent. Frames are re-sent when they have
been outstanding twice the RTT average.
flag ``Up'' when the netlink is up.
Minimum timer and RTT average as per Congestion Avoidance and
Path of the Ethernet device.
nl ea Ethernet address of the local Ethernet device.
``Up'' if the local interface is up.
Number of consecutive jumbograms lost.
SEE ALSOsd(3), sdaoe(3), aoesrv(8), snoopy(8)
Van Jacobson and Michael J. Karels, ``Congestion Avoidance and Con‐
trol'', ACM Computer Communication Review; Proceedings of the Sigcomm
'88 Symposium in Stanford, CA, August, 1988.
There is no raw file for executing arbitrary commands.
This is a fairly primitive interface; sdaoe(3) is usually more suit‐