kcusage(1M)kcusage(1M)NAMEkcusage - query the usage of kernel resources
SYNOPSISkcusage [( [tunable...
DESCRIPTIONkcusage is used to query the usage of kernel resources controlled by
various kernel tunables. The full list of tunables are given below,
along with some indication on how to interpret the data on each tun‐
able.
When none of the options h, d, m, or y are given, data is displayed
that reflects the currently running system. When any of these options
are given, then historical data is displayed.
When tunables are specified, then information is displayed for those
tunables. When no tunables are listed, then data is displayed for all
kernel tunables for which kcusage has information.
kcusage displays data recorded in the following 4 files:
· /var/adm/kcmond/hour
· /var/adm/kcmond/day
· /var/adm/kcmond/month
· /var/adm/kcmond/year
Each of these files contains the usage data for each monitored parame‐
ter for different time periods, in the form of records.
Every time the history is updated, the current usage value is compared
with the usage value recorded for the last 5 minute interval in the
hour file, for the last one hour interval in the day file, for the last
one day interval in the month file and for the last one week interval
in the year file.
If the current value exceeds the recorded value then the recorded value
is replaced with the current value. Thus in the historical output the
maximum usage value will be recorded, when the history was updated in
that time interval.
These records are updated when kcusage is executed with super user
privileges. These records are also updated periodically if there is an
ems request to monitor the resource /system/kernel_parameters/history.
Operands
recognizes the following operands
tunable Name of a monitorable tunable (see Monitored Tunables,
below).
Options
recognizes the following options
Print five minute data for the past hour.
Print hourly data for the past 24 hours.
Print daily data for the past 31 days.
Print weekly data for the past 52 weeks.
Print the listing in a long format that is easier to parse by another
program.
Print a listing that include the top 5 processes or users of the given
resource.
By default kcusage lists each of the monitorable tunables with the cur‐
rent usage of each of those tunables, no information on the top 5
users, and not in an easily machine parseable form.
The start of an hour, day or week is determined by the timezone kernel
tunable and is independent of the TZ environment variable.
There is no need to trim the data files as they are of fixed size.
MONITORED TUNABLES
The following kernel tunables are monitorable.
· nflocks
· npty
· nstrpty
· nkthread
· nproc
· nfile
· ninode
· nstrtel
· max_thread_proc
· maxfiles_lim
· maxdsiz
· maxdsiz_64bit
· maxssiz
· maxssiz_64bit
· maxtsiz
· maxtsiz_64bit
· maxuprc
· shmmax
· shmmni
· semmns
· semmni
· msgtql
· msgmni
· filecache_max
· nswapdev
· nswapfs
· shmseg
KERNEL TUNABLES
Kernel tunables allow the system administrator to control the size of
various resources within the kernel. Some kernel tunables control the
size of tables that can only be set when the kernel is built or booted.
Other tunables put limits on individual processes to prevent a rogue
process from taking over the system and causing other processes to
fail. Some tunables can be resized to change the performance of the
system. Still other tunables control the behavior of kernel.
Allocated Tables
Many kernel tunables set system wide limits on the amount of some
resource that can be allocated. Some of these limits exist because the
size of a table needs to be known at boot time. Others exist because
allocating one of these resources increases the memory usage of the
kernel.
· msgmni
· msgtql
· nfile
· nflocks
· ninode
· nkthread
· nproc
· npty
· nstrpty
· nstrtel
· semmni
· semmns
· shmmax
· shmmni
In general, system calls will fail if these values are set too low. If
the values are set too high memory will be wasted. Wasted memory slows
the performance of the system.
How to set these tunables depends on the importance of speed versus the
need for reliability. If your system can stand occasional failures you
might set these tunables so peak demand is 80% of the set limit. This
will give you efficient use of your memory and increased performance. A
small change in the use of the system could cause processes to fail.
If high availability is your goal, these tunables might be set so that
peak usage is only 20% of the set limits. More memory will be wasted,
and the system will be slightly slower, but the processes will keep
running even when the demands on the system change dramatically.
Many of these system limits can be set quite high without increasing
the size of the kernel. Previously, nproc and nkthread allocated static
tables in the kernel. Since HP-UX 11.22, both are limits on how big the
dynamic tables are allowed to grow.
See the man page for each of these kernel tunables to learn what memory
penalty, if any, each has when it is raised.
Inter Process Protection
Many of the tunables control the amount of any resource allocated to an
individual process. If all of the processes were allowed to grow with‐
out bound, the finite size of the system itself would cause system
calls allocating resource to fail.
The tunables that kcusage reports on that are used to protect processes
from one another are:
· maxdsiz/maxdsiz_64bit
· maxfiles_lim
· maxssiz/maxssiz_64bit
· maxtsiz/maxtsiz_64bit
· maxuprc
· max_thread_proc
The "-t" option in kcusage will show the top five users of each of
these resources.
In general, setting the size of these tunables depends on how much you
trust the processes on your system. Systems running well tested and/or
mission critical software can have these limits set quite high. This
will reduce administration costs in diagnosing problems with tunables
set too low.
On systems where software is being developed, or where large numbers of
users have access, these limits should be kept as low as is practical.
Controlling System Performance
kcusage monitors the sizes of the dynamic buffer cache and the inode
cache. Sizing these too small slows performance by increasing cache
miss rates. Sizing these too large slows performance by increasing the
swapping of processes.
Controlling kernel behavior
kcusage does not monitor any tunables that control system behavior.
kcusage gives data useful in setting 25 kernel tunables. The way the
usage information for each kernel tunable is interpreted is slightly
different for each tunable. The following is a description for how to
interpret each kernel tunables kcusage output.
filecache_max
filecache_max and filecache_min control the size of the dynamic buffer
cache for the system. These two tunables give the maximum and minimum
amount of memory, in percent of system memory, that is used to buffer
disk IO.
Because the kernel tunable is in percent of system memory, the usage is
also reported as a percentage of system memory. The usage value for
filecache_max is the percent of system memory used for the disk buffer
cache. The system resizes the disk buffer cache based on how much swap‐
ping the system is doing as well as the buffer cache miss rate.
For most kernel tunables, having the usage near the limit usually means
that you want to raise the limit. The usage data for filecache_max may
mean the opposite. If your system usage reads lots of disk data, but
does so in a way that disk records are not visited repeatedly, then a
large buffer cache may not speed up performance. The kernel, seeing a
high buffer cache miss rate, may dynamically enlarge the buffer cache
when doing so will not do much good. As the buffer cache gets larger
there is less room for running programs and swapping increases.
If you see that the size of the buffer cache is equal to filecache_max,
you may want to LOWER filecache_max. Some people recommend lowering
filecache_max to insure the dynamic buffer cache is at most 400
Megabytes. Others recommend raising filecache_max until the buffer
cache miss rate is under 5%.
Many data base applications manage their own buffering. Lowering the
limit on the buffer cache size can make these programs run faster.
The kcusage data for filecache_max and filecache_min can be somewhat
confusing because it is a percent, and kcusage reports usage as a per‐
cent of the limit. The percent of a percent is kind of confusing. If
filecache_max is set to 50% and 40% of the system memory is used as the
buffer cache, then the buffer cache will be reported as 80% of its
limit.
Since the usage data for filecache_max is simply the size of the buffer
cache, this data can also be useful when setting filecache_min.
maxdsiz/maxdsiz_64bit
maxdsiz sets a limit on the memory used by each process's data segment.
This is the memory allocated with calls to sbrk(2), malloc(3C) and new.
When a processes memory use reaches maxdsiz, calls to acquire more mem‐
ory will fail. When this happens, many programs dereference a null
pointer and core dump with a segmentation violation.
The cause can be one of two problems. The value for maxdsiz could be
set too low, or the program could have a defect commonly called a "mem‐
ory leak". Programs with a memory leak tend to slowly grow in size.
kcusage not only shows the size of the largest data segment, but it
also lists the IDs and executable names of the five process with the
largest data segments on the system. Look for trends in the size of
individual processes and decide if there is a memory leak or not. If
there is no memory leak, raise maxdsiz.
If there is a memory leak, this should be considered a defect in the
application, but can be worked around by restarting the application
during off peak use times. kcalarm can be used to signal when it is
time to restart such applications.
maxfiles_lim
The usage data shown for maxfiles_lim is the maximum number of files
opened by a process, across all processes. The "-t" option will show
details for the top 5 processes.
Trends in the maxfiles_lim data can reveal defects in programs where
files are opened and never closed.
This data is also useful when setting the tunable maxfiles_lim.
maxssiz/maxssiz_64bit
The usage data shown for maxssiz is the size of the largest stack seg‐
ment owned by any process on the system.
When this limit is reached the program that hits the limit is generally
allocating stack space for the call stack. Hitting this limit is usu‐
ally fixed by increasing maxssiz although recursive functions with
errors in their terminating logic can also be the cause.
Stack memory does not generally "leak" like data memory.
maxtsiz/maxtsiz_64bit
maxtsiz controls the size of the text segment of the processes on the
system. The text size of a process does not vary as it runs. This size
is based on the size of the executable.
The usage data shown here is the size of the largest text segment on
the system. Not all of the data in an executable is loaded into memory
when it is executed. This means the usage will not be the same as the
size shown by "ls -l executable_path".
If some processes are close to the limit set by maxtsiz, it may be a
good idea to increase maxtsiz when upgrading to a new version of the
software.
maxuprc
The usage value for maxuprc is the maximum number of processes owned by
a single user on the system. This limit prevents any individual user
from shutting out other users by allocating all processes up to the
limit set by nproc.
maxuprc should be set high enough to allow each user to get their job
done, but lower than the typical number of unused processes on the sys‐
tem. It may be necessary to raise maxuprc to meet both of these con‐
straints.
Privileged users are exempt from the limit set by maxuprc. Processes
with a user id of 0 are not counted in usage of this tunable.
kcusage shows the top five users of processes on the system. In this
case the "id" is the user id and the "name" is the user name.
max_thread_proc
kcusage shows the maximum number of kernel threads in use by any
process on the system.
max_thread_proc should be set high enough that well behaved applica‐
tions can get their work done, but low enough that a new process that
allocates its limit of threads cannot take all of the threads on the
system. Check the usage data for nkthread to know how many threads are
usually free.
kcusage shows the number of threads allocated by the top five processes
on the system.
msgmni
The usage data for msgmni is the number of message queues in use system
wide.
msgtql
The usage data for msgtql is the number of messages in queues system
wide.
Because data is found by polling, this will often be zero unless a mes‐
sage queue is busy enough that some messages stay in the queue for a
non-zero length of time.
nfile
nfile sets a limit on the total number of files that can be open on the
system as any given moment. kcusage shows the number of files that are
currently open system wide.
Look at the top five data for maxfiles_lim to learn which processes are
the largest users of open files.
nflocks
nflocks sets a limit on the total number of file locks that can be cre‐
ated on the system as any given moment. kcusage shows the number of
file locks that are in use, system wide.
ninode
kcusage data for ninode gives the number of inode records (HFS + VxFS)
in the inode cache.
nkthread
The usage data for nkthread is the number of kernel threads in use sys‐
tem wide. The processes that are the top five users of threads can be
found by looking at the data for max_thread_proc.
When the nkthread limit is reached the fork(2) and pthread_create(3T)
system calls will fail.
nproc
The usage data for nproc is the number of processes in use system wide.
The top fives users of process can be found by looking at the data for
maxuprc.
When the nproc limit is reached, the fork(2) system call will fail.
npty
The usage data for npty is the number of pseudo terminals in use system
wide. Locally running terminal emulators, such as dtterm(1), tend to
use pseudo terminals.
nstrpty
The usage data for nstrpty is the number of stream pseudo terminals in
use system wide. Running rlogin(1) from another system to the local
system is a common way that a stream pseudo terminal is used.
nstrtel
The usage data for nstrtel is the number of stream telnet(1) sessions
in use system wide. Running telnet from another system to the local
system is a common way to allocate a stream telnet.
semmni
The usage data for semmni is the number of semaphore identifiers in use
system wide. Each semaphore identifier contains 1 to semmsl semaphores.
semmns
The usage data for semmns is the number of semaphores in use system
wide.
shmmax
The usage data for shmmax shows the size of the largest shared memory
segment on the system. Shared memory is usually used to map shared
libraries but can also be allocated programmatically to allow data to
be shared between two processes.
shmmni
The usage data for shmmni shows the number of shared memory segments in
use. A shared memory segment is used to map each shared library in use
on the system, but a segment can also be allocated programmatically to
allow data to be shared between two processes.
nswapdev
The usage data for nswapdev in the number of devices that have been
enabled on the system for swap.
nswapfs
The usage data for nswapfs is the number of filesystems that have been
enabled on the system for swap.
shmseg
The usage data for shmseg is the number of System V shared memory seg‐
ments in use system wide.
EXAMPLES
Command with no options
kcusage
Tunable Usage / Setting
==================================
nflocks 67 / 200
npty 4 / 60
nstrpty 0 / 60
. .
. .
. .
. .
One day of hourly readings
kcusage-d nproc
Tunable: nproc
Setting: 276
Time Usage %
=============================================
Sun 02/24/02 18:00 MST 124 44.9
Sun 02/24/02 19:00 MST 118 42.8
Sun 02/24/02 20:00 MST 125 45.3
Sun 02/24/02 21:00 MST 125 45.3
Sun 02/24/02 22:00 MST 118 42.8
Sun 02/24/02 23:00 MST 119 43.1
Mon 02/25/02 00:00 MST 118 42.8
Displays the old tunable name with top five users of a kernel resource in human readable format
kcusage-t max_thread_proc|more
Tunable Usage / Setting Usage Id Name
==========================================================================
max_thread_proc 21 / 256
21 1405 dced
17 39 vxfsd
12 1611 swagentd
8 357 utmpd
8 366 wtmpd
DEPENDENCIESkcusage displays data read from files in /var/adm/kcmond. These files
are updated by the kcmond(1M) daemon. This daemon is started using the
kcalarm(1M) command. The data files are also updated whenever kcusage
is run by a privileged user.
AUTHORS
was developed by Hewlett-Packard.
SEE ALSOkcalarm(1M), kcmond(1M), kcweb(1M)kcusage(1M)
