LMC(4) BSD Kernel Interfaces Manual LMC(4)NAME
lmc — device driver for LMC (now SBE) wide-area network interface cards
SYNOPSIS
To wire this driver into your kernel, add the following line to your ker‐
nel configuration file:
device lmc
Alternatively, to load this module at boot time, add
if_lmc_load="YES"
to /boot/loader.conf; see loader.conf(5).
To wire a line protocol into your kernel, add:
options NETGRAPH
device sppp
It is not necessary to wire line protocols into your kernel, they can be
loaded later with kldload(8). The driver can send and receive raw IP
packets even if neither SPPP nor Netgraph are configured into the kernel.
Netgraph and SPPP can both be enabled; Netgraph will be used if the
rawdata hook is connected.
DESCRIPTION
This is an open-source UNIX device driver for PCI-bus WAN interface
cards. It sends and receives packets in HDLC frames over synchronous
circuits. A generic PC plus UNIX plus some LMC / SBE cards makes an open
router. This driver works with FreeBSD, NetBSD, OpenBSD, BSD/OS and
Linux OSs. It has been tested on i386 (SMP 32-bit little-endian) and
Sparc (64-bit big-endian) architectures.
The lmc driver works with the following cards:
· SBE wanADAPT-HSSI (LMC5200)
High Speed Serial Interface, EIA612/613, 50-pin connector, 0 to 52
Mb/s, DTE only.
· SBE wanADAPT-T3 (LMC5245)
T3: two 75-ohm BNC connectors, C-Parity or M13 Framing, 44.736 Mb/s,
up to 950 ft.
· SBE wanADAPT-SSI (LMC1000)
Synchronous Serial Interface, V.35, X.21, EIA449, EIA530(A), EIA232,
0 to 10 Mb/s, DTE or DCE.
· SBE wanADAPT-T1E1 (LMC1200)
T1 or E1: RJ45 conn, 100 or 120 ohms, T1-ESF-B8ZS, T1-SF-AMI,
E1-(many)-HDB3, 1.544 Mb/s or 2.048 Mb/s, up to 6 Kft.
Cards contain a high-performance PCI interface, an HDLC function and
either integrated modems (T1, T3) or modem interfaces (HSSI and SSI).
PCI The PCI interface is a DEC 21140A "Tulip" Fast Ethernet chip.
This chip has an efficient PCI implementation with scatter/gather
DMA, and can run at 100 Mb/s full duplex (twice as fast as needed
here).
HDLC The HDLC functions (ISO-3309: flags, bit-stuffing, CRC) are imple‐
mented in a Field Programmable Gate Array (FPGA) which talks to
the Ethernet chip through a Media Independent Interface (MII).
The hardware in the FPGA translates between Ethernet packets and
HDLC frames on-the-fly; think it as a WAN PHY chip for Ethernet.
Modem The modem chips are the main differences between cards. HSSI
cards use ECL10K chips to implement the EIA-612/613 interface. T3
cards use a TranSwitch TXC-03401 framer chip. SSI cards use Lin‐
ear Technology LTC1343 modem interface chips. T1 cards use a
BrookTree/Conexant/Mindspeed Bt8370 framer and line interface
chip.
Line protocols exist above device drivers and below internet protocols.
They typically encapsulate packets in HDLC frames and deal with higher-
level issues like protocol multiplexing and security. This driver is
compatible with several line protocol packages:
Netgraph Netgraph(4) implements many basic packet-handling functions
as kernel loadable modules. They can be interconnected in
a graph to implement many protocols. Configuration is done
from userland without rebuilding the kernel. Packets are
sent and received through this interface if the driver's
rawdata hook is connected, otherwise the ifnet interface
(SPPP and RawIP) is used. ASCII configuration control mes‐
sages are not currently supported.
SPPP sppp(4) implements Synchronous-PPP, Frame-Relay and Cisco-
HDLC in the kernel.
RawIP This null line protocol, built into the driver, sends and
receives raw IPv4 and IPv6 packets in HDLC frames (aka IP-
in-HDLC) with no extra bytes of overhead and no state at
the end points.
EXAMPLES
ifconfig and lmcconfig
The program lmcconfig(8) manipulates interface parameters beyond the
scope of ifconfig(8). In normal operation only a few arguments are
needed:
-X selects the external SPPP line protocol package.
-x selects the built-in RawIP line protocol package.
-Z selects PPP line protocol.
-z selects Cisco-HDLC line protocol.
-F selects Frame-Relay line protocol.
lmcconfig lmc0
displays interface configuration and status.
lmcconfig lmc0 -D
enables debugging output from the device driver only.
ifconfig lmc0 debug
enables debugging output from the device driver and from the line
protocol module above it. Debugging messages that appear on the
console are also written to file /var/log/messages. Caution:
when things go very wrong, a torrent of debugging messages can
swamp the console and bring a machine to its knees.
Operation
Activate a PPP link using SPPP and Netgraph with:
ngctl mkpeer lmc0: sppp rawdata downstream
ifconfig sppp0 10.0.0.1 10.0.0.2
Activate a PPP link using only SPPP with:
lmcconfig lmc0 -XYZ
ifconfig lmc0 10.0.0.1 10.0.0.2
Activate a Cisco-HDLC link using SPPP and Netgraph with:
ngctl mkpeer lmc0: sppp rawdata downstream
ifconfig sppp0 10.0.0.1 10.0.0.2 link2
Activate a Cisco-HDLC link using only SPPP with:
lmcconfig lmc0 -XYz
ifconfig lmc0 10.0.0.1 10.0.0.2
Activate a Cisco-HDLC link using only Netgraph with:
ngctl mkpeer lmc0: cisco rawdata downstream
ngctl mkpeer lmc0:rawdata iface inet inet
ifconfig ng0 10.0.0.1 10.0.0.2
Activate a Frame-Relay DTE link using SPPP with:
lmcconfig lmc0 -XYF
ifconfig lmc0 10.0.0.1 10.0.0.2
(SPPP implements the ANSI T1.617 annex D LMI.)
Activate a Frame-Relay DTE link using Netgraph with:
ngctl mkpeer lmc0: frame_relay rawdata downstream
ngctl mkpeer lmc0:rawdata lmi dlci0 auto0
ngctl connect lmc0:rawdata dlci0 dlci1023 auto1023
ngctl mkpeer lmc0:rawdata rfc1490 dlci500 downstream
ngctl mkpeer lmc0:rawdata.dlci500 iface inet inet
ifconfig ng0 10.0.0.1 10.0.0.2
This is ONE possible Frame Relay configuration; there are many.
Activate a RAWIP link using only the driver with:
lmcconfig lmc0 -x
ifconfig lmc0 10.0.0.1 10.0.0.2
Activate a RAWIP link using Netgraph with:
ngctl mkpeer lmc0: iface rawdata inet
ifconfig ng0 10.0.0.1 10.0.0.2
If the driver is unloaded and then loaded, reconnect hooks by:
ngctl connect lmc0: ng0: rawdata inet
TESTING
Testing with Loopbacks
Testing with loopbacks requires only one card. Packets can be looped
back at many points: in the PCI chip, in the modem chips, through a loop‐
back plug, in the local external equipment, or at the far end of a cir‐
cuit.
Activate the card with ifconfig(8):
ifconfig lmc0 10.0.0.1 10.0.0.1
All cards can be looped through the PCI chip. Cards with internal modems
can be looped through the modem framer and the modem line interface.
Cards for external modems can be looped through the driver/receiver
chips. See lmcconfig(8) for details.
Loopback plugs test everything on the card.
HSSI Loopback plugs can be ordered from SBE (and others). Transmit
clock is normally supplied by the external modem. When an HSSI
card is operated with a loopback plug, the PCI bus clock must be
used as the transmit clock, typically 33 MHz. When testing an
HSSI card with a loopback plug, configure it with lmcconfig(8):
lmcconfig lmc0 -a 2
“-a 2” selects the PCI bus clock as the transmit clock.
T3 Connect the two BNC jacks with a short coax cable.
SSI Loopback plugs can be ordered from SBE (only). Transmit clock is
normally supplied by the external modem. When an SSI card is
operated with a loopback plug, the on-board clock synthesizer must
be used. When testing an SSI card with a loopback plug, configure
it with lmcconfig(8):
lmcconfig lmc0 -E -f 10000000
-E puts the card in DCE mode to source a transmit clock. “-f
10000000” sets the internal clock source to 10 Mb/s.
T1/E1 A loopback plug is a modular plug with two wires connecting pin 1
to pin 4 and pin 2 to pin 5.
One can also test by connecting to a local modem (HSSI and SSI) or NI (T1
and T3) configured to loop back. Cards can generate signals to loopback
remote equipment so that complete circuits can be tested; see
lmcconfig(8) for details.
Testing with a Modem
Testing with a modem requires two cards of different types.
T3/HSSI If you have a T3 modem with an HSSI interface (made by Digital
Link, Larscom, Kentrox etc.) then use an HSSI card in one
machine and a T3 card in the other machine. The T3 coax cables
must use the null modem configuration (see below).
T1/V.35 If you have a T1 (or E1) modem with a V.35, X.21 or EIA530
interface, then use an SSI card in one machine and a T1 card in
the other machine. Use a T1 null modem cable (see below).
Testing with a Null Modem Cable
Testing with a null modem cable requires two cards of the same type.
HSSI Three-meter HSSI null-modem cables can be ordered from SBE. In a
pinch, a 50-pin SCSI-II cable up to a few meters will work as a
straight HSSI cable (not a null modem cable). Longer cables
should be purpose-built HSSI cables because the cable impedance is
different. Transmit clock is normally supplied by the external
modem. When an HSSI card is connected by a null modem cable, the
PCI bus clock can be used as the transmit clock, typically 33 MHz.
When testing an HSSI card with a null modem cable, configure it
with lmcconfig(8):
lmcconfig lmc0 -a 2
“-a 2” selects the PCI bus clock as the transmit clock.
T3 T3 null modem cables are just 75-ohm coax cables with BNC connec‐
tors. TX OUT on one card should be connected to RX IN on the
other card. In a pinch, 50-ohm thin Ethernet cables usually work
up to a few meters, but they will not work for longer runs —
75-ohm coax is required.
SSI Three-meter SSI null modem cables can be ordered from SBE. An SSI
null modem cable reports a cable type of V.36/EIA449. Transmit
clock is normally supplied by the external modem. When an SSI
card is connected by a null modem cable, an on-board clock synthe‐
sizer is used. When testing an SSI card with a null modem cable,
configure it with lmcconfig(8):
lmcconfig lmc0 -E -f 10000000
-E puts the card in DCE mode to source a transmit clock. “-f
10000000” sets the internal clock source to 10 Mb/s.
T1/E1 A T1 null modem cable has two twisted pairs that connect pins 1
and 2 on one plug to pins 4 and 5 on the other plug. Looking into
the cable entry hole of a plug, with the locking tab oriented
down, pin 1 is on the left. A twisted pair Ethernet cable makes
an excellent straight T1 cable. Alas, Ethernet cross-over cables
do not work as T1 null modem cables.
OPERATION NOTES
Packet Lengths
Maximum transmit and receive packet length is unlimited. Minimum trans‐
mit and receive packet length is one byte.
Cleaning up after one packet and setting up for the next packet involves
making several DMA references. This can take longer than the duration of
a short packet, causing the adapter to fall behind. For typical PCI bus
traffic levels and memory system latencies, back-to-back packets longer
than about 20 bytes will always work (53 byte cells work), but a burst of
several hundred back-to-back packets shorter than 20 bytes will cause
packets to be dropped. This usually is not a problem since an IPv4
packet header is at least 20 bytes long.
This device driver imposes no constraints on packet size. Most operating
systems set the default Maximum Transmission Unit (MTU) to 1500 bytes;
the legal range is usually (72..65535). This can be changed with
ifconfig lmc0 mtu 2000
SPPP enforces an MTU of (128..far-end-MRU) for PPP and 1500 bytes for
Cisco-HDLC. RAWIP sets the default MTU to 4032 bytes, but it can be
changed to anything.
BPF - Berkeley Packet Filter
This driver has hooks for bpf(4), the Berkeley Packet Filter. The line
protocol header length reported to BPF is four bytes for SPPP and P2P
line protocols and zero bytes for RawIP.
To include BPF support into your kernel, add the following line to
conf/YOURKERNEL:
device bpf
To test the BPF kernel interface, bring up a link between two machines,
then run ping(8) and tcpdump(1):
ping 10.0.0.1
and in a different window:
tcpdump -i lmc0
The output from tcpdump(1) should look like this:
03:54:35.979965 10.0.0.2 > 10.0.0.1: icmp: echo request
03:54:35.981423 10.0.0.1 > 10.0.0.2: icmp: echo reply
Line protocol control packets will appear among the ping(8) packets occa‐
sionally.
Device Polling
A T3 receiver can generate over 100K interrupts per second, This can
cause a system to “live-lock”: spend all of its time servicing inter‐
rupts. FreeBSD has a polling mechanism to prevent live-lock.
FreeBSD's mechanism permanently disables interrupts from the card and
instead the card's interrupt service routine is called each time the ker‐
nel is entered (syscall, timer interrupt, etc.) and from the kernel idle
loop; this adds some latency. The driver is permitted to process a lim‐
ited number of packets. The percentage of the CPU that can be consumed
this way is settable.
See the polling(4) manpage for details on how to enable the polling mode.
SNMP: Simple Network Management Protocol
This driver is aware of what is required to be a Network Interface Object
managed by an Agent of the Simple Network Management Protocol. The
driver exports SNMP-formatted configuration and status information suffi‐
cient for an SNMP Agent to create MIBs for:
RFC-2233: Interfaces group,
RFC-2496: DS3 interfaces,
RFC-2495: DS1/E1 interfaces,
RFC-1659: RS232-like interfaces.
An SNMP Agent is a user program, not a kernel function. Agents can
retrieve configuration and status information by using Netgraph control
messages or ioctl(2) system calls. User programs should poll
sc->cfg.ticks which increments once per second after the SNMP state has
been updated.
HSSI and SSI LEDs
The card should be operational if all three green LEDs are on (the upper-
left one should be blinking) and the red LED is off. All four LEDs turn
on at power-on and module unload.
RED upper-right No Transmit clock
GREEN upper-left Device driver is alive if blinking
GREEN lower-right Modem signals are good
GREEN lower-left Cable is plugged in (SSI only)
T1E1 and T3 LEDs
The card should be operational if the upper-left green LED is blinking
and all other LEDs are off. For the T3 card, if other LEDs are on or
blinking, try swapping the coax cables! All four LEDs turn on at power-
on and module unload.
RED upper-right Received signal is wrong
GREEN upper-left Device driver is alive if blinking
BLUE lower-right Alarm Information Signal (AIS)
YELLOW lower-left Remote Alarm Indication (RAI)
The green LED blinks if the device driver is alive.
The red LED blinks if an outward loopback is active.
The blue LED blinks if sending AIS, on solid if receiving AIS.
The yellow LED blinks if sending RAI, on solid if receiving RAI.
E1 Framing
Phone companies usually insist that customers put a Frame Alignment
Signal (FAS) in time slot 0. A Cyclic Redundancy Checksum (CRC) can also
ride in time slot 0. Channel Associated Signalling (CAS) uses Time Slot
16. In telco-speak signalling is on/off hook, ringing, busy, etc. Sig‐
nalling is not needed here and consumes 64 Kb/s. Only use E1-CAS formats
if the other end insists on it! Use E1-FAS+CRC framing format on a pub‐
lic circuit. Depending on the equipment installed in a private circuit,
it may be possible to use all 32 time slots for data (E1-NONE).
T3 Framing
M13 is a technique for multiplexing 28 T1s into a T3. Muxes use the C-
bits for speed-matching the tributaries. Muxing is not needed here and
usurps the FEBE and FEAC bits. Only use T3-M13 format if the other end
insists on it! Use T3-CParity framing format if possible. Loop Timing,
Fractional T3, and HDLC packets in the Facility Data Link are not sup‐
ported.
T1 & T3 Frame Overhead Functions
Performance Report Messages (PRMs) are enabled in T1-ESF.
Bit Oriented Protocol (BOP) messages are enabled in T1-ESF.
In-band loopback control (framed or not) is enabled in T1-SF.
Far End Alarm and Control (FEAC) msgs are enabled in T3-CPar.
Far End Block Error (FEBE) reports are enabled in T3-CPar.
Remote Alarm Indication (RAI) is enabled in T3-Any.
Loopbacks initiated remotely time out after 300 seconds.
T1/E1 'Fractional' 64 kb/s Time Slots
T1 uses time slots 24..1; E1 uses time slots 31..0. E1 uses TS0 for FAS
overhead and TS16 for CAS overhead. E1-NONE has no overhead, so all 32
TSs are available for data. Enable/disable time slots by setting 32
1s/0s in a config param. Enabling an E1 overhead time slot, or enabling
TS0 or TS25-TS31 for T1, is ignored by the driver, which knows better.
The default TS param, 0xFFFFFFFF, enables the maximum number of time
slots for whatever frame format is selected. 56 Kb/s time slots are not
supported.
T1 Raw Mode
Special gate array microcode exists for the T1/E1 card. Each T1 frame of
24 bytes is treated as a packet. A raw T1 byte stream can be delivered
to main memory and transmitted from main memory. The T1 card adds or
deletes framing bits but does not touch the data. ATM cells can be
transmitted and received this way, with the software doing all the work.
But that is not hard; after all it is only 1.5 Mb/s second!
T3 Circuit Emulation Mode
Special gate array microcode exists for the T3 card. Each T3 frame of
595 bytes is treated as a packet. A raw T3 signal can be packetized,
transported through a packet network (using some protocol) and then
reconstituted as a T3 signal at the far end. The output transmitter's
bit rate can be controlled from software so that it can be frequency
locked to the distant input signal.
HSSI and SSI Transmit Clocks
Synchronous interfaces use two transmit clocks to eliminate skew caused
by speed-of-light delays in the modem cable. DCEs (modems) drive ST,
Send Timing, the first transmit clock. DTEs (hosts) receive ST and use
it to clock transmit data, TD, onto the modem cable. DTEs also drive a
copy of ST back towards the DCE and call it TT, Transmit Timing, the sec‐
ond transmit clock. DCEs receive TT and TD and use TT to clock TD into a
flip flop. TT experiences the same delay as (and has no skew relative
to) TD. Thus, cable length does not affect data/clock timing.
SEE ALSOtcpdump(1), ioctl(2), bpf(4), kld(4), netgraph(4), polling(4), sppp(4),
loader.conf(5), ifconfig(8), lmcconfig(8), mpd(8) (ports/net/mpd),
ngctl(8), ping(8), ifnet(9)
http://www.sbei.com/
HISTORY
Ron Crane had the idea to use a Fast Ethernet chip as a PCI interface and
add an Ethernet-to-HDLC gate array to make a WAN card. David Boggs
designed the Ethernet-to-HDLC gate array and PC cards. We did this at
our company, LAN Media Corporation (LMC). SBE Corp. acquired LMC and
continues to make the cards.
Since the cards use Tulip Ethernet chips, we started with Matt Thomas'
ubiquitous de(4) driver. Michael Graff stripped out the Ethernet stuff
and added HSSI stuff. Basil Gunn ported it to Solaris (lost) and Rob
Braun ported it to Linux. Andrew Stanley-Jones added support for three
more cards and wrote the first version of lmcconfig(8). David Boggs
rewrote everything and now feels responsible for it.
AUTHORS
David Boggs ⟨boggs@boggs.palo-alto.ca.us⟩
BSD May 20, 2006 BSD