KDEBUG(4) BSD Programmer's Manual KDEBUG(4)NAMEkdebugSYNOPSIS
options KDEBUG
DESCRIPTION
Kdebug is a simple command line based debugging facility that can be
built into the kernel. It is completely self contained, a second machine
is not needed. The debugger does not provide source level, or even sym-
bolic debugging capabilities; it is a very simple monitor.
General capabilities include:
o Decoding and displaying system trace buffer
o Display various kernel and hardware variables
o Decode and display certain kernel data structures
o Display and modify 32 bit words or 8 bit bytes in physical or virtual
memory
o Issue inb and outb instructions to I/O devices
o Use hardware breakpoint registers of CPU to implement four kernel
breakpoints or watchpoints.
o Start cross system GDB
o Reboot the system immediately
This facility is not intended to replace kgdb (which is source level sym-
bolic debugger). It is useful to gain control of the machine when it is
otherwise hung, allows simple debugging without a second machine, and
provides a way to use the built in hardware debugging registers for
watchpointing and breakpointing.
This facility is also useful to poke and prod at a device to determine
its behavior while writing or debugging a device driver.
ENABLING THE DEBUGGER
Enabling kdebug requires a kernel source distribution. The kernel must
be built with options KDEBUG.
Kdebug will use either a serial port or the VGA screen as its console.
The default is to use whatever type of device the system console is, and
if serial to assume that it is /dev/tty00 (serial port 0x3f8).
The KDEBUG_CONTYPE option allows the console type to be fixed (to use a
serial port for debugging with a VGA console for instance). If set to 1
the serial I/O routines are used, if set to 2 the VGA routines are used.
When a serial port is being used the option KDEBUG_SERBASE determines the
I/O address of the serial port. This defaults to 0x3f8. Note that with
no options the debugger will automatically do the 'right thing' if you
have a VGA console or serial console on the first serial port. If you are
using the second serial port as the console, then the KDEBUG_SERBASE
variable must be used to give the alternate I/O address (0x2f8 in the
case of /dev/tty01).
The serial I/O routines are very simple and are built into , the VGA I/O
routines are part of the standard VGA driver. For a very clean debugging
environment (which doesn't depend on other kernel facilities) use a seri-
al debug console. The serial routines assume the serial port has already
been set up by the kernel (baud rate, parity, etc.).
ENTERING THE DEBUGGER
Entering the debugger disables all interrupts (except NMI). If the bpnmi
option is set and the debugger is entered due to an NMI or breakpoint ex-
ception then a broadcast NMI is sent. If the thalt option is set, the
current kernel trace mask is set to 0. The system should remain static
while kdebug is running; however in a multi processor system, other pro-
cessors may still be executing user or kernel code.
A number of mechanisms can be used to enter the debugger:
Ctl-Alt-Del or Break
If the kernel global cntaltdelcore is set (either via sysctl(8) or
via the CNTALTDEL=1 option) then a console break (serial) or the
Ctl-Alt-Del key sequence will cause the debugger to run.
SysReq key
If a PC keyboard is attached the debugger will be entered when the
SysReq (PrintScrn) key is pressed; note that if the console is seri-
al (and the debugger console selection has not been overridden) the
debugger will start on the serial console, not on the VGA screen.
NMI For cases where interrupts are disabled and the console is unrespon-
sive, it is sometimes possible to regain control by toggling the I/O
channel check line on the ISA bus.
I/O channel check will generate an NMI on most motherboards (you may
have to enable RAM parity checking to enable the channel check NMI).
Generating an I/O channel check requires some hardware support. The
two opposing pins in an ISA slot that are closest to the back edge
of the motherboard must be shorted; this can be done (very careful-
ly!) with a paper clip or small screwdriver, if done often it is
suggested that a debugging card with a pushbutton on it be built or
purchased for this purpose. Do NOT attempt to do this with a PCI
slot.
It is also possible to generate an NMI by grounding the LINT0 pin on
the CPU, this may sometimes be necessary if the chipset does not
pass an I/O channel check indication as an NMI (details of imple-
menting this are left to the reader with a current Pentium pinout).
Finally, it is possible to generate an NMI via a message on the APIC
bus (from one CPU to another). The nmi command can be used to bring
another CPU into the debugger. The set bpnmi command will force a
CPU entering the debugger as the result of a breakpoint or NMI to
send a broadcast NMI message on the APIC bus, sending all other
CPU's into the debugger.
When kdebug is compiled into the kernel, the IDT entry for NMI is
set up as a task gate (vs. and interrupt gate). When an interrupt is
delivered via a task gate the CPU saves its registers into a special
area (call the TSS, or task state segment) and then loads new state
(including the P counter and virtual memory page directory pointer)
from a different TSS. When the interrupt has been serviced, the CPU
state is reloaded from the first TSS and execution resumes.
The advantage of using a TSS to service NMI requests for kdebug is
that it provides a fresh stack and virtual address space in which to
run. This allows recovery from conditions where the kernel has over-
run (or otherwise molested) its stack and or virtual address space
pointer.
The registers at the time of the NMI can be displayed with the
display tss command, the registers of interest are in the "Main O/S
tss".
Resuming after an NMI is not always possible, there are race condi-
tions at the hardware level that sometimes cause the machine to hang
when resuming. This is fairly rare.
Hardware breakpoint
When the break command has been used to set up the CPU's hardware
debug registers, a debug breakpoint will enter kdebug (via a task
gate).
Resuming after a breakpoint works as one would expect.
When entered from a breakpoint, displaying the various TSS's (via
the display tss command) will show the registers at the time of the
breakpoint, in the "Main O/S tss".
Panic
When the system panics (before it syncs the disks) it will enter
kdebug just after printing the panic message. This is done as a nor-
mal subroutine call from the panic() routine. This runs kdebug on
the caller's stack.
Subroutine call
A call to kdebug() can be placed anywhere it is appropriate to enter
the debugger (such as in error cases while debugging a device driv-
er). This runs kdebug on the caller's stack.
Boot option
If the option -kdebug 2 is included in the /etc/boot.default or en-
tered at the Boot: prompt, kdebug will be entered very early in the
system startup (just before printing the first console message con-
taining the copyright notice). This is done on the init processes
stack.
Double fault
When the CPU detects a double fault condition (a fault while trying
to dispatch a fault handler, the details of exactly when this is
done are fairly complex), kdebug is entered via a TSS gate (which
gives it a new stack). The registers at the time of the double fault
should be in the "Main O/S tss", however for many double faults the
previous state of the CPU is not saved (in this case the best indi-
cator of what was happening is the trace buffer).
USING BREAKPOINTS
A detailed description of the debugging facilities provided by the CPU
can be found the Pentium Family Users Manual; this is only a brief
overview.
There are four debug address registers (dr0 through dr3) that can have
any linear address placed in them. Debug register dr7 is used to deter-
mine what type of breakpoint each address register represents and if it
is enabled or not. At reset time the enables are all cleared.
When a matching condition (address plus type plus length) is detected
kdebug is entered via a TSS gate. When the debugger is exited, it sets a
special bit in the task segment that causes breakpoint to be disabled for
one instruction only (this prevents the breakpoint from firing again im-
mediately and allows the instruction to execute unmolested).
The four breakpoint definitions are global, but each CPU in a system has
its own set of breakpoint registers. The hardware breakpoint registers
are set when a CPU leaves the debugger, thus steps must be taken to in-
sure that all CPU's have exited the debugger after a breakpoint has been
set. The simplest way is to enter the debugger on one CPU (via the SysReq
key or a serial break), then bring the additional CPUs into the debugger
via the 'nmi ff' (broadcast NMI) command. Once all CPU's are in the de-
bugger, set the breakpoint then use the 'cont' command so they all exit
at the same time with the debug registers set correctly.
The breakpoint conditions (set individually for each breakpoint) are:
o Memory read or write
o Memory write
o Instruction execution
o I/O read or write (address is an I/O port address)
In addition to the breakpoint type, the length of the data item must be
specified for all but the instruction breakpoint (which must always be a
byte length breakpoint).
The syntax of the break command is described in detail in a later sec-
tion, it provides a way to set up each of the four hardware breakpoints.
Unfortunately kdebug does not have access to kernel symbols, so linear
addresses (in hex) must be entered for the breakpoints (usually nm(1) or
gdb(1) is run separately to get the address of a global or line of source
code).
USER INTERFACE
The interface consist of a prompt at which a command and optional argu-
ments are entered, terminated by a <CR>. The line may be edited with Ctl-
U (to erase the entire line), or Ctl-H (backspace) to back up one charac-
ter.
The prompt consists of the list of CPU's currently running in the debug-
ger, a ':' and the current cpu number (in hex) followed by a greater than
sign (>). If running on a UP system without local APIC's the prompt will
usually be only a greater than sign (although some motherboards may pre-
sent garbage data at the local APIC address which will result in a prompt
with a random current CPU number).
All numeric input is in hex, with alpha characters in lower case. A
leading 0x may optionally be supplied, but is not necessary.
The output may be paginated (mostly useful on VGA consoles) by setting
the screen size via set lines.
When running on a VGA console the debugger will flash the keyboard LED's
while it is waiting for input; this is so that one can tell they are in
the debugger even if X was running on the console. There is currently no
way to reset the VGA display to text mode after entering the debugger,
although commands can by typed blind (usually 'boot' or 'sysdump').
MULTIPROCESSOR SUPPORT
If two (or more) processors are in kdebug simultaneously, they share ac-
cess to the command line via a mutex lock. Commands entered at the
prompt are given to the CPU which generated the prompt (the number to the
left of the prompt determines which CPU is requesting input).
To switch to another processor a <CR> or <ESC> may be issued, to force
the debugger to run on a particular cpu, use the set cpu command.
A heartbeat is generated by the CPU in control of the command line, if
that heartbeat ceases then one of the other CPU's waiting for the command
line mutex will break the mutex lock, allowing it (and any other CPUs
waiting) access to the kernel.
There are no tracing macros in the mainline debugger code (although the
entry and exit code contains some tracing). When a given CPU enters the
debugger the other CPU(s) may still be running kernel code and generating
trace entries (usually clock interrupts), which will rapidly clear the
other CPUs trace buffer of useful information. There are several strate-
gies that can be used to deal with this situation: the set tcpu command
can be used to display only trace entries from the CPU that is in the de-
bugger (it will usually be the one with the kernel lock and the one with
the data of interest), the set bpnmi command will cause a broadcast NMI
to be generated when one CPU enters the debugger, or the set thalt com-
mand can be used to cause any debugger entry to clear the trace mask (the
trace mask must be manually restored via the set ktr_mask command before
exiting the debugger).
It is sometimes desirable to bring all the CPUs into the debugger (to set
a breakpoint for instance) and then have them all leave the debugger
somewhat simultaneously, the cont command will produce this result. With
cont each CPU will wait until the last CPU leaves the debugger before re-
suming normal execution. The quit command does not wait for other CPU's
to leave the debugger (this could be used to keep one CPU in the debugger
at all times to monitor execution of the other CPU).
COMMAND REFERENCE
Command may be abbreviated to their shortest unique representation.
+, =
Displays the next item when appropriate. The definition of 'next'
depends on what display command was most recently issued (for exam-
ple, when viewing memory with view, + will display the next 16 bytes
of memory). This command executes immediately (a <CR> is not re-
quired).
- This displays the previous item as appropriate to the last command
that was issued. This command executes immediately (a <CR> is not
required).
boot
Immediately reboots the system, with no attempt to flush disk
buffers.
break bpno [keyword ...]
Allows modification of one of the four hardware breakpoints. When
given with no arguments, the state of the breakpoint registers are
displayed.
When bpno is given it refers to one of the hardware breakpoints (in
the range of 0 through 3). The keywords given after the breakpoint
number modify a particular attribute of the breakpoint. Breakpoint
attributes are:
enable
Enable the breakpoint.
disable
Disable the breakpoint.
addr address
Set breakpoint address, address is a 32 bit hex linear address
(use care when setting a breakpoint in user space as the linear
address mappings can differ per process).
inst
Set the breakpoint type to 'instruction fetch'. This breakpoint
will trigger when an instruction is about to be executed at the
breakpoint address. The length attribute of the breakpoint is
automatically set to 'byte', forcing it to anything other than
'byte' may have unexpected consequences.
write
Set the breakpoint to watch for data writes at the breakpoint
address with the length specified by the length attribute.
Note that this does not snoop DMA activity at the given ad-
dress, nor from other processors.
rw Set the breakpoint to watch for data reads or writes at the
breakpoint address. Note that this does not snoop DMA activity
at the given address, nor from other processors.
io Set the breakpoint to watch for read or write accesses to the
given I/O port.
byte
Set the breakpoint watch length to 1 byte; the address may be
aligned on any boundary.
short
Set the breakpoint watch length to 2 bytes; for memory watch-
points the address must be on a 2 byte boundary (even).
word
Set the breakpoint watch length to 4 bytes; for memory watch-
points the address must be on a 4 byte boundary (addr & 0x3 ==
0).
For example, setting an instruction breakpoint at 0xf0142232:
0> br 0 addr 0xf0142232 inst en
This would result in the following setup:
0> br
0: f0142232 byte inst fetch enabled
1: 0 byte inst fetch disabled
2: 0 byte inst fetch disabled
3: 0 byte inst fetch disabled
Setting breakpoints in the debugger is not recommended.
cont
Wait for all other CPUs to leave the debugger and then exit.
display item
Display various system details.
spl Display information related to the blocking of interrupts. Ex-
ample:
Cpl : 7ffffffa
Hwcpl : 7ffffffa
Ipending : 0
DoPicChk : 0
Nonemask : 7fffa7a0
OldNonemask: 0
Highmask : bfffffff
Ttymask : 7fffb7ba
Biomask : 7fffe7e0
Impmask : 7fffafa0
OrigImpmask: 7fffafa0
Memmask : 7fffefe0
Protomask : ffffa7a0
pic Display internal status of the two 8259 programmable interrupt
controllers (PICs). Example:
0> d pic
Mask : fffa
Request : 811
In service : 0
Stray Master: 1
Stray Slave : 0
lock
Show status of various kernel locks. Example:
0> d lock
Main kernel lock (klock)
Gate : 1
Owner : 0x0
Count : 2
Held at : Clkintr
Spincnt : 0x0
[...]
io apicno
Display internal status of an I/O APIC. Example:
1> d io 2
Base VA : ffc23000
Id : 2000000
Version : 170011
ArbId : 0
Pin Vec Dest DestMode DelMode Mask Trigger RemIrr Polarity
0 f0 f Physical LowPri Ena Edge NoPend High
1 d0 f Physical LowPri Ena Edge NoPend High
2 0 f Physical LowPri Dis Edge NoPend High
[...]
aerr
Display CPU local apic error counters. Example:
1> d aerr
Cpu 0x1 error counters
Total : 0
Stray : 0
Ill Reg Addrs : 0
Send
Checksum : 0
Accept : 0
Illegal Vec : 0
Receive
Checksum : 0
Accept : 0
Illegal Vec : 0
[...]
inin
Display interrupt information table (used to program and route
interrupts): Example:
1> d inin
Interrupt sources (not including free ones)
Entry 0x0 (f01864bc) IRQ0 (via PIC):
Mask : bfffffff Flags : 0 Cnt : d5
Type : 4 Irq : 0 Aux : 0
Apic : 2 Pin : 0 IDTVec: f0
Base : ffc23000 Regsel : 10 RdHigh: f000000
Disa : 101f0 Ena : 1f0
[...]
pcpu
Display information on per-cpu area mappings and selected per-
cpu area fields. Example:
1> d pcpu
f0a18000 -- Cpu-1:
map : ffc00000 --> 13af023
[...]
map : ffc33000 --> 1c8061
state : 1
curproc : 0
curpcb : f11b8000
idleproc : f0a0d700
npxproc : 0
prevproc : 0
astpend : 25
want_rsch: 4
emergency: f0a1f290 -> f0a1f68f
[...]
The address given on the first line is the low core address of
the per CPU area for the listed processor (always mapped at
that address).
idt Display the contents of the interrupt descriptor table. Exam-
ple:
1> d idt
CPU Idt=300007ff f016ffc3
0: 300007ff f016ffc3 f0001af4 esi= 143 edi= 6a0076
dest=??? (f1580411)
1: f016ffc3 f11b9c68 f0001b00 esi= 157 edi= 6a0076
dest=??? (f12c0426)
2: f11b9c68 f013c29a 0
[...]
This somewhat cryptic display gives: The actual IDT registers
(returned by lidt), for each IDT entry the following data are
displayed:
1. The IDT entry number.
2. The raw contents of the low order IDT entry word.
3. The raw contents of the high order IDT entry word.
4. The address of the handler.
5. The value to be loaded into esi before calling the generic
handler (only for interrupt gates).
6. The value loaded into edi on the way to the generic han-
dler.
7. The address and type of the generic handler for this in-
terrupt.
Note that not all decoded fields make sense for all IDT entry
types.
stat
Display current debugger settings and state.
tss Display the various task state segment save areas. The one that
is usually of interest is the 'Main O/S TSS', this is where
registers are saved on an NMI or breakpoint. Each CPU has a set
of TSS's associated with it.
Example:
Cpu-0 Main O/S tss
Cpu-0 Eip : f0133f40
Cpu-0 Esp : ffc07678
Cpu-0 Eax : 0
Cpu-0 Ebx : 0
Cpu-0 Ecx : 0
Cpu-0 Edx : 0
Cpu-0 Ebp : 0
Cpu-0 Esi : 0
Cpu-0 Edi : 0
Cpu-0 CR3 : 12f2000
Cpu-0 Flags : 2
ktr_mask
Display the current trace mask in hex.
gdb Call the cross system gdb support stub, this has the effect of caus-
ing the machine to wait for a remote connection on the gdb port.
This command only exists if kgdb is compiled into the kernel.
When kgdb is detached, kdebug resumes.
help, ?
Display a brief summary of all available commands.
ib [port]
iw [port]
il [port]
Read a byte, word, or long from an I/O port and display it. This
simply issues the appropriate in instruction. If the port number
isn't given, the last port read is used.
mod addr byte1 [byte2 ...]
pmod addr byte1 [byte2 ...]
Modify memory bytes. The addr gives the first byte to modify, each
subsequent argument is a byte to be stored (starting at addr and go-
ing up). The mod form modifies virtual memory, while the pmod form
modifies physical memory.
nmi target
Sends and NMI IPI to target. The target is an APIC bus address. If
'ff' is given then a broadcast NMI message is sent (this sends NMI
to all CPUs except the one sending the message). This command is
used to force another CPU to enter the debugger.
ob [port [val]]
ow [port [val]]
ol [port [val]]
Writes val (a byte, word, or long) to the I/O port at port. If val
isn't given, the last value written is used. If port isn't given,
the last port read or written is used.
pciread addr [agent [bus [func]]]
pciwrite addr data [agent [bus [func]]]
Reads or writes 32 bit value from or to PCI configuration space (as-
suming configuration mechanism 1). The addr must be aligned on a 4
byte boundary. Agent, bus, and func can be used to specify the
agent, bus and function. If not given the previous value is used, 0
is the default.
print structure addr
Prints portions of selected structures, given a pointer to the
struct in addr. The structures that may be printed are:
trapframe
Display a struct trapframe.
frame
Display some elements of a C stack frame.
quit
Leave this instance of the debugger. If running on multiple CPU's,
another CPU may still be in the debugger. This does not wait for
other CPUs to leave the debugger.
read [addr]
pread [addr]
Read a 32 bit value from memory (little endian), the address is a
byte address in linear virtual memory (for read) or in physical mem-
ory (for pread ).
set keyword [val]
Set various debugger parameters.
cpu cpuno
Set a preferred CPU to be used by the debugger. If the listed
CPU isn't running (or is ff) then the debugger prompt will ro-
tate between the CPUs running in the debugger.
tcpu cpuno
Restrict display of trace buffer entries to a particular CPU's
trace buffer, use 0xff to unrestrict and display from all trace
buffers.
ktr_mask value
Set kernel trace mask to value. This takes effect immediately.
verbose
Display certain items (such as the trace display) in more de-
tail.
noverbose
Display certain items in normal (non-verbose) mode.
frame addr
Set the current stack frame address and display details about
the frame (as print frame does). After this command, the - com-
mand may be used to walk backwards through the stack one frame
at a time.
lines length
Set the screen length, this enables paging of output. If set to
0, paging is disabled (the default is 0 for a serial console
and 24 for a VGA console).
cons vga | serial [port [speed]]
Switch the debugger console to the specified device. If port is
given, it must the be the base hex address for the serial port
to be used (Ex: 0x3f8 for tty00, 0x2f8 for tty01).
If the speed option is given the serial port will be initial-
ized for 8 bits, no parity, and the selected speed. Exiting the
debugger will not restore the prior speed; use this option only
if the port is otherwise unused and must be initialized. The
only supported speeds are 9600 and 115200.
stb [addr]
Perform a stack traceback given the starting frame. If the starting
frame isn't given then a default is chosen based on how the debugger
was entered.
The output gives the address of the stack frame, the EIP that is to
be returned to in that frame (all f's for the first frame since we
don't know the value of the P counter for it), followed by the first
8 arguments that the frame was called with.
If the frame is in the trap() function additional decoding of the
arguments is performed (the struct trapframe is printed in detail).
sync
Runs the normal system and file system sync and time-of-day clock
sync routines.
sysdump
Runs the system dumper, this will normally write an image of physi-
cal memory to the dump device. Depending on the device driver ser-
vicing the dump device, it may not be possible to resume system op-
eration after writing the dump image.
trace
Display the most recent entry from the set of active trace buffers
(there is one trace buffer per CPU). Additional entries (going back-
ward in time) are displayed with the - command.
view [addr]
pview [addr]
View 16 bytes of memory (in hex and ASCII). The + and - commands may
be used to scroll through memory after this command is issued.
If the addr isn't given, the last address given to view is used.
The view form of the command displays virtual memory, and the pview
form of the command display physical memory.
write [addr [val]]
pwrite [addr [val]]
Write a 32 bit value to memory (little endian). If val isn't given,
the last value written is used. If addr isn't given, the last ad-
dress written is used. The write form writes to virtual memory, and
the pwrite form writes to physical memory.
xlat addr [pdir]
Shows detailed translation information for a given virtual address (
addr ). The page directory for the current instance of the debugger
is used unless pdir is supplied, once given this page directory ad-
dress is used regardless of the current debugger virtual memory con-
text; automatic directory selection may be restored by specifying a
page directory address of 0. The page directory pointer is the page
aligned physical address of the page directory page (this is the
same value that is loaded into the Cr3 register to set up an address
space).
The output breaks up the components of the virtual addresses and
walks the page directory and page table entries. Example:
01:0> xlat f0010123
Translating f0010123:
Page directory (CR3) : 5b1000
Directory Offset : f00 (idx 3c0)
PT Offset : 40 (idx 10)
Directory Entry : 204063
PT Entry : 10023
Physical Address : 10123
01:1>
SEE ALSOgdb(1), ktr(4)HISTORY
The kdebug facility appeared in BSD/OS 4.0.
