AOE(3)AOE(3)NAMEaoe - ATA-over-Ethernet (AoE) interface
SYNOPSIS
bind -a #æ /dev
/dev/aoe/ctl
/dev/aoe/log
/dev/aoe/shelf.slot/config
/dev/aoe/shelf.slot/ctl
/dev/aoe/shelf.slot/devlink/0
...
/dev/aoe/shelf.slot/devlink/i
/dev/aoe/shelf.slot/ident
...
DESCRIPTION
The AoE (ATA-over-Ethernet) interface serves a three-level directory
providing control and access to AoE targets. The interface provided is
primarily intended for low-level control of the AoE initiator. See
sdaoe(3) for the standard interface.
Top-level files
In order to access AoE targets, one or more Ethernet controllers need
to be bound to the AoE initiator. By default, the system starts with
no interfaces bound. For automatic binding of interfaces on boot, the
aoeif configuration variable is set in plan9.ini(8). Ethernet inter‐
faces are specified as ethern, not as #ln. To bind the first and sec‐
ond Ethernet devices on boot, add
aoeif=ether0 ether1
To bind ether1 to a running system:
% echo bind '#l1/ether1' >/dev/aoe/ctl
And to unbind it
% echo unbind '#l1/ether1' >/dev/aoe/ctl
When an interface is unbound, targets depending on that interface are
removed.
Each local interface is called a netlink. The mapping of AoE targets
to netlinks is called a devlink. Each devlink may see multiple inter‐
faces per target. For example, if the local machine has one Ethernet
address bound and the target has two interfaces on the same Ethernet
segment, this will result in one netlink and one devlink with two Eth‐
ernet addresses. AoE frames are sent in round-robin fashion. Each
successive frame is sent on the next address available on the next
available devlink (local interface).
Normally the initiator automatically discovers and adds new device
directories on startup. New devices are not added except as new inter‐
faces are bound to the initiator. Several messages can be written to
/dev/aoe/ctl which alter this behavior:
autodiscover toggle
If toggle is absent, the state of autodiscover is toggled. If
it is the string on, it is turned on. Any other string turns
autodisover off. This option is not useful after Ethernet
devices have been bound.
discover shelf.slot
Attempt to find the named target on all bound interfaces.
remove shelf.slot
The converse of discover: remove the named target if it exists.
rediscover toggle
Allow or disallow rediscovery. This allows for automatic dis‐
covery of new targets. Unfortunately, it also allows automatic
modification or loss of existing targets. This option is con‐
sidered dangerous.
Reading /dev/aoe/ctl returns a list of colon-separated lines with key‐
words and their values:
debug
autodiscover
rediscover
Returns the current state of the variable named by the keyword.
Writing the variable's name to the control file toggles the
state of that variable.
ifn path
Path to nth bound Ethernet device.
ifn ea Ethernet address of this device.
ifn flag
A flag of ``Up'' indicates that this interface is available.
ifn lostjumbo
Number of consecutive lost jumbograms.
ifn datamtu
Incorrect and unused.
Target subdirectories
Once configured, each AoE target is accessed via files in the directory
named for its shelf and slot. For example, shelf 42, slot 0 would be
accessed through the path The ident file contains the read-only, verba‐
tim result of the identify unit ATA command. The config file contains
the target's AoE configuration string. Writing to this file sets the
targets configuration string.
Reading a target's ctl file returns a list of colon-separated lines
with the following keywords and values:
state ``Up'' or ``down''.
nopen Number of clients using this target.
nout Number of outstanding AoE frames.
nmaxout
Maximum number of outstanding frames allowed.
nframes
Maximum number of outstanding frames. Nframes is greater than
nmaxout when the initiator is reducing the number of in-flight
frames due to packet loss. It is assumed that packet loss is
due to an overwhelmed target and not poor network conditions.
maxbcount
Maximum number of data bytes per AoE frame. Using standard
frames, maxbcount is 1024 or two sectors. AoE ATA headers are
36 bytes.
model
serial
firmware
The respective fields from the ATA identify unit command.
flag List of flags useful for debugging. The flag jumbo indicates
that jumbo frames are accepted, not that they are being used.
Maxbcount should be consulted for this purpose.
Writing to the ctl file, the following commands may be issued:
failio fail outstanding i/o.
identify
send an ata command to the target.
maxbno n
set the maximum number of block sent per packet.
mtu n set the maximum number of bytes (including header) sent per
packet.
nofail never fail this target. This is useful if your root device is
on this target.
setsize n
with no arguments, reset the device size to the size claimed.
Otherwise, assume the device is the given size.
The data file may be read or written like a normal file except that
reads and writes to this file are converted to AoE commands to the tar‐
get, so transfers should be 512 or 1024 bytes long (or a larger multi‐
ple of 512 iff jumbo packets are in use). The size of this file is the
usable size of the target.
The devlink directory contains one file for each interface the target
was discovered on. The files are numbers from 0 to n and contain a
list of colon-separated lines with keywords and their values:
addr A space-separated list of the target's Ethernet addresses visi‐
ble from this interface.
npkt The number of frames sent on this interface.
resent The number of frames re-sent. Frames are re-sent when they have
been outstanding twice the RTT average.
flag ``Up'' when the netlink is up.
rttavg
mintimer
Minimum timer and RTT average as per Congestion Avoidance and
Control.
nl path
Path of the Ethernet device.
nl ea Ethernet address of the local Ethernet device.
nl flag
``Up'' if the local interface is up.
nl lostjumbo
Number of consecutive jumbograms lost.
nl datamtu
Unused.
SOURCE
/sys/src/9/port/devaoe.c
SEE ALSOsd(3), sdaoe(3), aoesrv(8), snoopy(8)
http://support.coraid.com/documents/AoEr11.txt
Van Jacobson and Michael J. Karels, ``Congestion Avoidance and Con‐
trol'', ACM Computer Communication Review; Proceedings of the Sigcomm
'88 Symposium in Stanford, CA, August, 1988.
BUGS
There is no raw file for executing arbitrary commands.
This is a fairly primitive interface; sdaoe(3) is usually more suit‐
able.
AOE(3)
