JSCAN(8) BSD System Manager's Manual JSCAN(8)NAMEjscan — journal file processing program
SYNOPSISjscan [-2dfuvF] [-c count[k,m,g,t]] [-D directory]
[-m mirror_transid_file/none] [-o/O output_transid_file/none]
[-s size[k,m,g,t]] [-w/W journal_prefix] [journal_prefix/file]
DESCRIPTION
The jscan utility scans journal file or input stream for the purposes of
debugging dumps, restoration, undo, mirroring, and other journaling fea‐
tures.
-2 Implement the full-duplex acknowledgement protocol on the input
descriptor. Note that shell pipes are full-duplex and can be
used with this option.
-c count
Specify the number of transaction records which should be
scanned, then exit. This option is typically used along with -m
to limit the amount of work that jscan does, giving you the abil‐
ity to incrementally run a mirror forwards or backwards. It is
not usually used when piping in a live journal, but it can be.
-d Display the contents of the journaling file or stream in a human
readable format on stderr. Note that stdout is used only for -o.
-fjscan will sleep for 5 seconds and loop when it hits EOF on file
or prefix set input rather than exit. This option is typically
used when running on an input file or prefix set which is live
(being written to by another jscan instance).
-D directory
Specify the base directory for the mirroring option.
-m mirror_transid_file/none
Generate a mirror in the directory specified by -D or, if not
specified, the current directory. The mirror_transid_file will
be used to track the transaction id representing the current syn‐
chronization point for the mirror. The keyword none may be spec‐
ified if no tracking file is desired. However, if no tracking
file is specified it will not be possible to roll the mirror for‐
wards or backwards or restart the journaling stream being used to
generate the mirror.
It is important to note that journaling streams can contain meta-
transactions representing huge, multi-gigabyte operations. If
the journaling data is not being recorded to regular files via
-w/W it is possible that jscan could run itself out of memory
trying to record the meta-transactions. In addition, the mirror
would not be restartable. If the journaling data is being
recorded via -w/W and a mirroring transaction id file is being
kept, the mirror can be restarted.
While it is possible to run a journaling stream directly into a
mirror, it is more typical to file the jornaling stream with -w
and catch the mirror up as a batch job with the journaling file
set prefix specified as the input every so often. This way the
system operator can use other jscan commands to, for example, run
a mirror backwards and forwards in time.
-o/O output_transid_file/none
Generate a journaling stream on stdout using the specified file
to track the transaction id to help with restarts. The -o option
indicates a half-duplex output stream while the -O option indi‐
cates a full-duplex (ACK protocol) output stream.
This option is not really designed to output to regular files
because it does NOT necessarily weed out duplicate records. When
both the input stream and output stream are full-duplex and -w/W
is not specified, jscan acts as a stateless transceiver and the
input stream is not acked until an ack is received from the out‐
put stream.
This option is most typically used in conjunction with -w/W. In
this case the ACK protocol is handled independently for the input
side and the output side uses the journaling data recorded by
-w/W as a buffer.
In half-duplex output mode the output transaction id file is
updated after a raw transaction record has been successfully
written to stdout. In full-duplex output mode the file is only
updated with ACK data returned on the stdout descriptor.
As with the -m option, you can combine -o in a journaling pipe
with other options, but if you are trying to use it as a buffer
it may be better to have it separately pull its data off of a
journaling file set generated via -w.
-s size
Change the size limit for rotating files created via -w. The
default is 100M. Values are in bytes or may be suffixed with k,
m, or g. If a raw transaction causes the file's size limit to be
exceeded, a new file will be created. If a raw transaction is,
in-whole, larger than the file's size limit, the raw transaction
will still be fully written to the file before a new file is cre‐
ated. Raw transactions are typically limited to the size of the
source system's memory FIFO. This option is typically used to
size journaling files to fit onto the appropriate backup media or
to provide bite-sized chunks for other programs to ingest.
When restarting a journal, a new sequence number will always be
chosen for the resumption of data recording. No existing file
will be appended to when jscan is reinvoked.
-u Will cause the journal to be scanned backwards (requires seekable
media). Transactions will be dumped in reverse order. If mir‐
roring, the UNDO data will be executed. If not specified, 1
hour's worth of data will be undone. Can only be used with a
journaling file or journaling prefix as the input.
-v Increase verbosity on stderr. This option is primarily used for
debugging.
-w prefix
The received journaling stream is recorded in journaling files
named <prefix>.<seq> and the current transaction id is tracked in
a file named <prefix>.transid. A journaling file is closed out
and a new file with the next sequence number is created once the
file surpasses 100MB.
This option is robust across restarts. The current transaction
id will be read and the input stream will be skipped until it is
reached. If the input is a journaling file or prefix set, jscan
will be able to quickly seek to the restart point.
NOTE: If you are generating a mirror with the same command via
-m, and the journaling data input is a stream rather than a file
or prefix set, you must use -w/W if you want the mirror to be
restartable. This is because while we can pick up the transac‐
tion id where we left off, that raw transaction id may have cut a
larger meta-transaction in half and the mirroring code will not
be able to access the whole of the transaction unless it has a
file or prefix set to work with.
-W prefix
Similar to -w except that the journaling files created are
strictly temporary and will be deleted once they exceed the size
limit AND the related meta-transactions have been completed.
If combined with -m, the meta-transactions are considered to be
completed only when the mirror finishes executing them. It is
possible for several sequence number files to build if a particu‐
larly large meta-transaction is coming down the pipe.
If combined with -o/O, the meta-transactions are considered to be
completed when the data has been successfully written out to the
pipe in half duplex mode, or when the ACK has been received in
full-duplex mode.
If both -m and -w/W is used, the journaling data files are only
deleted when both actions no longer need the data.
-F Forces jscan to fsync() after updating a journaling file prior to
acknowledging the data or updating a transaction-id-tracking
file. If specified twice, jscan will also fsync() after updating
the transaction-id-tracking file.
journal_prefix/file
Specify the input to jscan. This can be a journaling file set
prefix or it can be a plain file. If no input file is specified,
stdin is assumed. Note that when generating a mirror from a
stdin stream, the mirror will not be restartable unless -w/W is
also used.
OPERATIONAL NOTES
It is often important to be able to quickly stage journaled data through
a dedicated backup machine on a LAN. There are several places where data
can be buffered and staged out.
The machine generating the journal typically buffers several megabytes of
journal data in the kernel. This local machine can pipe that data to
jscan or some other locally run program to add another buffering stage,
or you can directly attach a TCP connection to the kernel's journaling
output.
The LAN backup box typically buffers gigabytes worth of data by running
multiple jscans. The jscan on the receiving end of the TCP or pipe (for
example, via ssh) typically records the data via the -w option, and then
runs other jscan programs from scripts or cron to take that data and copy
it to your off-site backup machine. Other jscan programs may use the
same data set to generate mirrors or other backup streams.
It should be noted that if -w/W is specified, both mirroring mode and
output mode will internally fork the program once the appropriate syn‐
chronization point has been reached, effectively decoupling their opera‐
tion, and read all of their data via the journaling files written out by
the master program. In particular, blockages in the mirroring and output
code will not affect our ability to buffer the journaling input data via
-w/W. If -w/W is not specified then neither the mirroring or output
modes will fork. Under these conditions, if the input is a stream rather
than a file jscan will be forced to buffer meta-transactions (for mirror‐
ing) entirely in memory, which could present a serious problem since a
single meta-transaction can exceed a gigabyte (e.g. if someone were to do
a single write() system call writing a gigabyte all in one go).
SEE ALSOmountctl(8)CAVEATS
This utility is currently under construction and not all features have
been implemented yet. In fact, most have not.
HISTORY
The jscan utility first appeared in DragonFly 1.3.
BSD February 8, 2009 BSD
