A number of utility scripts and programs are supplied with Exim. Most of them are built as part of the normal building process, but the log file analyser is entirely free-standing.
The shell script called exiwhat first of all empties the file exim-process.info in Exim's spool directory. It then uses the ps command to find all processes running exim, and sends each one the SIGUSR1 signal. This causes each process to write a single line describing its current activity to the file. The script waits for one second to allow the Exim processes to react, then copies the file to the standard output.
Unfortunately, the ps command varies between different versions of Unix. Not only are different options used, but the format of the output is different. For this reason, there are some system configuration options that configure exactly how exiwhat works. If it doesn't seem to be working for you, check the following compile-time options:
EXIWHAT_PS_CMD the command for running ps EXIWHAT_PS_ARG the argument for ps EXIWHAT_EGREP_ARG the argument for egrep to select from ps output EXIWHAT_KILL_ARG the argument for the kill command
This facility is available only in operating systems where a signal handler can
be set up such that an interrupted system call is resumed when the signal
handler has finished. An example of typical output from exiwhat is
The first number in the output line is the process number.
164 daemon: -q1h, listening on port 25
10483 running queue: waiting for 0tAycK-0002ij-00 (10492)
10492 delivering 0tAycK-0002ij-00 to mail.ref.book [42.42.42.42]
(editor@ref.book)
10592 handling incoming call from [245.211.243.242]
10628 accepting a local non-SMTP message
The exiqsumm utility is a Perl script, provided in the util directory, which reads the output of exim -bp and produces a summary of the messages by outputting a line like the following for each domain:
3 2322 74m 66m msn.com
This contains the number of messages for that domain, their total volume, and the length of time the oldest and the newest have been waiting. By default the output is sorted on the domain name, but exiqsumm has the options -a and -c, which cause it to be sorted by oldest message and by count of messages, respectively.
The output of exim -bp is based on the original addresses in the message, so no addresses generated by aliasing or forwarding are included. Consequently this applies also to the output from exiqsumm.
The exigrep utility is a Perl script, provided in the util directory, that extracts from one or more log files all entries relevant to any message whose log entries contain at least one that matches a given pattern. The pattern match is case-insensitive. Thus one can search for all mail for a given user or a given host, for example. The usage is:
exigrep [-l] <pattern> [<log file>] ...
where the -l flag means `literal', that is, treat all characters in the pattern as standing for themselves. Otherwise the pattern must be a Perl regular expression. If no file names are given on the command line, the standard input is read.
If the location of a zcat command is known from the definition of ZCAT_COMMAND in `Local/Makefile', exigrep automatically passes any file whose name ends in COMPRESS_SUFFIX through zcat as it searches it.
The exicyclog script can be used to cycle mainlog and rejectlog files that have been written to local disc. This is not necessary if only syslog is being used. Some operating systems have their own standard scripts for log cycling, and these can be used instead of exicyclog if preferred.
Each time exicyclog is run the files get `shuffled down' by one. If the main log file name is mainlog (the default) then when exicyclog is run mainlog becomes mainlog.01, the previous mainlog.01 becomes mainlog.02 and so on, up to a limit which is set in the script, and which defaults to 10.
In versions of Exim prior to 1.90, exicyclog used single-digits for numbers less than ten. This was changed to make the files list in a more natural order. The script contains conversion code. If it finds a file called mainlog.1 it attempts to rename all files in the old form to the new form.
If no mainlog file exists, the script does nothing. Reject logs are handled similarly. Files that `drop off' the end are deleted. All files with numbers greater than 01 are compressed, using a compression command which is configured by the COMPRESS_COMMAND setting in `Local/Makefile'.
It is usual to run exicyclog daily from a crontab entry of the form
1 0 * * * /opt/exim/bin/exicyclog
In this way, each day's log is (mostly) in a separate file. There will be some overlap from processes that have the log open at the time of renaming.
The exicyclog script can be run as the Exim user when one is defined, because the log files will be owned by that user in that case. Otherwise it has to be run as root.
The exim_dbmbuild program reads an input file in the format of an alias file (see chapter 23) and writes a DBM database using the lower-cased alias names as keys and the remainder of the information as data. The lower-casing can be prevented by calling the program with the -nolc option.
A terminating zero is included as part of the key string. This is expected by the dbm lookup type. However, if the option -nozero is given, exim_dbmbuild creates files without terminating zeroes in either the key strings or the data strings. The dbmnz lookup type can be used with such files.
The program requires two arguments: the name of the input file (which can be a single hyphen to indicate the standard input), and the name of the output database. It creates the database under a temporary name, and then renames the file(s) if all went well. If the native DB interface is in use (USE_DB is set in a compile-time configuration file -- this is common in free versions of Unix) the two file names must be different, because in this mode the Berkeley DB functions create a single output file using exactly the name given. For example:
exim_dbmbuild /etc/aliases /etc/aliases.db
reads the system alias file and creates a DBM version of it in /etc/aliases.db.
In systems that use the ndbm routines (mostly proprietary versions of Unix), DBM databases consist of two files with suffixes .dir and .pag. In this environment, the suffixes are added to the second argument of exim_dbmbuild, so it can be the same as the first. This is also the case when the Berkeley functions are used in compatibility mode (though this is not recommended), because in that case it adds a .db suffix to the file name.
The program outputs a warning if it encounters a duplicate key, and when it finishes, its return code is 1 rather than zero, unless the -noduperr option is used. By default, only the first of a set of duplicates is used -- this makes it compatible with lsearch lookups. There is an option -lastdup which causes it to use the data for the last duplicate instead. There is also an option -nowarn, which stops it listing duplicate keys to stderr. For other errors, where it doesn't actually make a new file, the return code is 2.
A utility called exinext (mostly a Perl script) provides the ability to fish specific information out of the retry database. Given a mail domain (or a complete address), it looks up the hosts for that domain, and outputs any retry information. At present, the retry information is obtained by running exim_dumpdb (see below) and post-processing the output. For example:
exinext piglet@milne.fict.book kanga.milne.fict.book:100.100.8.1 error 146: Connection refused first failed: 21-Feb-1996 14:57:34 last tried: 21-Feb-1996 14:57:34 next try at: 21-Feb-1996 15:02:34 roo.milne.fict.book:100.100.8.3 error 146: Connection refused first failed: 20-Jan-1996 13:12:08 last tried: 21-Feb-1996 11:42:03 next try at: 21-Feb-1996 19:42:03 past final cutoff time
You can also give exinext a local local_part, without a domain, and it will give any retry information for it. Also, a message id can be given to obtain retry information pertaining to a specific message. This exists only when an attempt to deliver a message to a remote host suffers a message-specific error (see section 48.2). Exinext is not particularly efficient, but then it isn't expected to be run very often.
Three utility programs are provided for maintaining the DBM files that Exim uses to contain its delivery hint information. Each program requires two arguments. The first specifies the name of Exim's spool directory, and the second is the name of the database it is to operate on. These are as follows:
The entire contents of a database are written to the standard output by the exim_dumpdb program, which has no options or arguments other than the spool and database names. For example, to dump the retry database:
exim_dumpdb /var/spool/exim retry
Two lines of output are produced for each entry:
T:mail.ref.book:242.242.242.242 146 77 Connection refused 31-Oct-1995 12:00:12 02-Nov-1995 12:21:39 02-Nov-1995 20:21:39 *
The first item on the first line is the key of the record. It starts with one of the letters D, R, or T, depending on whether it refers to a directing, routing, or transport retry. For a local delivery, the next part is the local address; for a remote delivery it is the name of the remote host, followed by its failing IP address. Then there follows an error code, an additional error code, and a textual description of the error.
The three times on the second line are the time of first failure, the time of the last delivery attempt, and the computed time for the next attempt. The line ends with an asterisk if the cutoff time for the last retry rule has been exceeded.
Each output line from exim_dumpdb for the reject database consists of a date and time, followed by the letter T or F and a fixed point number, followed by the address that was rejected, followed either by the name of the host that sent the bad address, if this has been verified, or otherwise the IP address. The letter is F if only one previous rejection of this address (from this host) has been done recently, and T if a second has occurred, causing rejection of the MAIL command, and subsequently rejection of the RCPT commands. The fixed point number is zero when the last rejection was a permanent one. Otherwise it records the rate of temporary rejections for the same address from the same host, per hour.
Each output line from exim_dumpdb for the wait-xxx databases consists of a host name followed by a list of ids for messages that are or were waiting to be delivered to that host. If there are a very large number for any one host, continuation records, with a sequence number added to the host name, may be seen. The data in these records is often out of date, because a message may be routed to several alternative hosts, and Exim makes no effort to keep cross-references.
Each output line from exim_dumpdb for the serialize-smtp database consists of a host name preceded by the time that Exim made a connection to that host. Exim keeps track of connections only for those hosts or networks that have been configured for serialization.
The exim_tidydb utility program is used to tidy up the contents of the hints databases. If run with no options, it removes all records from a database that are more than 30 days old. The cutoff date can be altered by means of the -t option, which must be followed by a time. For example, to remove all records older than a week from the retry database:
exim_tidydb -t 7d /var/spool/exim retry
For the wait-xxx and retry databases, the -f option can also be used. Both these databases contain items that involve message ids. In the former these appear as data in records keyed by host -- they were messages that were waiting for that host -- and in the latter they are the keys for retry information for messages that have suffered certain types of error. When -f is used, a check is made to ensure that message ids in database records are those of messages that are still on the queue. Message ids for messages that no longer exist are removed from wait-xxx records, and if this leaves any records empty, they are deleted. For the retry database, -f causes the removal of records whose keys are non-existent message ids. For other types of database, -f has no effect.
The exim_tidydb utility outputs comments on the standard output whenever it removes information from the database. It is suggested that it be run periodically on all the hints databases, but at a quiet time of day, since it requires a database to be locked (and therefore inaccessible to Exim) while it does its work.
The exim_fixdb program is a utility for interactively modifying databases. Its main use is for testing Exim, but it might also be occasionally useful for getting round problems in a live system. It has no options, and its interface is somewhat crude. On entry, it prompts for input with a right angle-bracket. A key of a database record can then be entered, and the data for that record is displayed.
If `d' is typed at the next prompt, the entire record is deleted. For all except the retry database, that is the only operation that can be carried out. For the retry database, each field is output preceded by a number, and data for individual fields can be changed by typing the field number followed by new data, for example:
> 4 951102:1000
resets the time of the next delivery attempt. Time values are given as a sequence of digit pairs for year, month, day, hour, and minute. Colons can be used as optional separators.
A Perl script called eximstats is supplied in the util directory. This has been hacked about quite a bit over time. It now gives quite a lot of information by default, but there are options for suppressing various parts of it. Following any options, the arguments to the script are a list of files, which should be main log files.
Eximstats extracts information about the number and volume of messages received from or delivered to various hosts. The information is sorted both by message count and by volume, and the top fifty hosts in each category are listed on the standard output. For messages delivered and received locally, similar statistics are produced per user.
The output also includes total counts and statistics about delivery errors, and histograms showing the number of messages received and deliveries made in each hour of the day. A delivery with more than one address in its `envelope' (for example, an SMTP transaction with more than one RCPT command) is counted as a single delivery by eximstats.
Though normally more deliveries than receipts are reported (as messages may have multiple recipients), it is possible for eximstats to report more messages received than delivered, even though the spool is empty at the start and end of the period in question. If an incoming message contains no valid recipients, no deliveries are recorded for it. An error report is handled as an entirely separate message.
Eximstats always outputs a grand total summary giving the volume and number of messages received and deliveries made, and the number of hosts involved in each case. It also outputs the number of messages that were delayed (that is, not completely delivered at the first attempt), and the number that had at least one address that failed.
The remainder of the output is in sections that can be independently disabled or modified by various options. It consists of a summary of deliveries by transport, histograms of messages received and delivered per time interval (default per hour), information about the time messages spent on the queue, a list of relayed messages, lists of the top fifty sending hosts, local senders, destination hosts, and destination local users by count and by volume, and a list of delivery errors that occurred.
The relay information lists messages that were actually relayed, that is, they came from a remote host and were directly delivered to some other remote host. A delivery that is considered as a relay by the checking features described in section 46.4, because its domain is not in local_domains, might still end up being delivered locally under some configurations, and if this happens it doesn't show up as a relay in the eximstats output.
The options for eximstats are as follows:
Suppress the statistics about delivery by transport.
This option controls the histograms of messages received and deliveries per time interval. By default the time interval is one hour. If -h0 is given, the histograms are suppressed; otherwise the value of <n> gives the number of divisions per hour, so -h2 sets an interval of 30 minutes, and the default is equivalent to -h1.
Suppress information about times messages spend on the queue.
This option sets an alternative list of time intervals for the queueing information. The values are separated by commas and are in seconds, but can involve arithmetic multipliers, so for example you can set 3*60 to specify 3 minutes. A setting such as
-q60,5*60,10*60
causes eximstats to give counts of messages that stayed on the queue for less than one minute, less than five minutes, less than ten minutes, and over ten minutes.
Suppress information about messages relayed through this host.
Suppress information about relayed messages that match the pattern, which is matched against a string of the following form (split over two lines here in order to fit it on the page):
H=<host> [<ip address>] A=<sender address> => H=<host> A=<recipient address>
for example
H=in.host [1.2.3.4] A=from@some.where => H=out.host A=to@else.where
The sending host name appears in parentheses if it has not been verified as matching the IP address. The mail addresses are taken from the envelope, not the headers. This option allows you to screen out hosts whom you are happy to have using your host as a relay.
Sets the `top' count to <n>. This controls the listings of the `top <n>' hosts and users by count and volume. The default is 50, and setting 0 suppresses the output altogether.
Omit local information from the `top' listings.
Suppress the list of delivery errors.
The exim_lock utility locks a mailbox file using the same algorithm as Exim. This can be used to prevent any modification of a mailbox by Exim or a user agent while investigating a problem. The utility requires the name of the file as its first argument. If the locking is successful, the second argument is run as a command (using C's system() function); if there is no second argument, the value of the SHELL environment variable is used; if this is unset or empty, /bin/sh is run. When the command finishes, the mailbox is unlocked and the utility ends. The following options are available:
Use fcntl() locking on the open mailbox.
This must be followed by a number, which is a number of seconds; it sets the
interval to sleep between retries (default 3).
-interval (exim_lock)
Create a lock file before opening the mailbox.
Lock the mailbox using MBX rules.
This must be followed by a number; it sets the number of times to try to get
the lock (default 10).
-retries (exim_lock)
This must be followed by a number, which is a number of seconds; it sets a
timeout to be used with a blocking fcntl() lock. If it is not set (the
default), a non-blocking call is used.
-timeout (exim_lock)
Generate verbose output.
Suppress verification output.
If none of -fcntl, -lockfile or -mbx are given, the default is to create a lock file and also use fcntl() locking on the mailbox, which is the same as Exim's default. The use of -fcntl requires that the file be writeable; the use of -lockfile requires that the directory containing the file be writeable. Locking by lock file does not last for ever; Exim assumes that a lock file is expired if it is more than 30 minutes old.
The -mbx option is mutually exclusive with -fcntl. It causes a shared lock to be taken out on the open mailbox, and an exclusive lock on the file /tmp/.n.m where n and m are the device number and inode number of the mailbox file. When the locking is released, if an exclusive lock can be obtained for the mailbox, the file in /tmp is deleted.
The default output contains verification of the locking that takes place. The -v option causes some additional information to be given. The -q option suppresses all output except error messages.
A command such as
exim_lock /var/spool/mail/spqr
runs an interactive shell while the file is locked, whereas
exim_lock -q /var/spool/mail/spqr <<End <some commands> End
runs a specific non-interactive sequence of commands while the file is locked, suppressing all verification output. A single command can be run by a command such as
exim_lock -q /var/spool/mail/spqr \ "cp /var/spool/mail/spqr /some/where"
Note that if a command is supplied, it must be entirely contained within the second argument -- hence the quotes.
Go to the first, previous, next, last section, table of contents.