A number of utility scripts and programs are supplied with Exim and are described in this chapter. There is also the Exim Monitor, which is covered in the next chapter. The utilities described here are:
| 46.1 exiwhat | list what Exim processes are doing |
| 46.2 exiqgrep | grep the queue |
| 46.3 exiqsumm | summarize the queue |
| 46.4 exigrep | search the main log |
| 46.5 exicyclog | cycle (rotate) log files |
| 46.6 eximstats | extract statistics from the log |
| 46.7 exim_checkaccess | check address acceptance from given IP |
| 46.8 exim_dbmbuild | build a DBM file |
| 46.9 exinext | extract retry information |
| 46.10 exim_dumpdb | dump a hints database |
| 46.10 exim_tidydb | clean up a hints database |
| 46.10 exim_fixdb | patch a hints database |
| 46.11 exim_lock | lock a mailbox file |
On operating systems that can restart a system call after receiving a signal (most modern OS), an Exim process responds to the SIGUSR1 signal by writing a line describing what it is doing to the file exim-process.info in the Exim spool directory. The exiwhat script sends the signal to all Exim processes it can find, having first emptied the file. It then waits for one second to allow the Exim processes to react before displaying the results. In order to run exiwhat successfully you have to have sufficient privilege to send the signal to the Exim processes, so it is normally run as root.
Unfortunately, the ps command which exiwhat uses to find Exim processes varies in different operating systems. Not only are different options used, but the format of the output is different. For this reason, there are some system configuration options that configure exactly how exiwhat works. If it doesn't seem to be working for you, check the following compile-time options:
EXIWHAT_PS_CMD the command for running ps
EXIWHAT_PS_ARG the argument for ps
EXIWHAT_EGREP_ARG the argument for egrep to select from ps output
EXIWHAT_KILL_ARG the argument for the kill command
An example of typical output from exiwhat is
164 daemon: -q1h, listening on port 25
10483 running queue: waiting for 0tAycK-0002ij-00 (10492)
10492 delivering 0tAycK-0002ij-00 to mail.ref.example [10.19.42.42]
(editor@ref.example)
10592 handling incoming call from [192.168.243.242]
10628 accepting a local non-SMTP message
The first number in the output line is the process number. The third line has been split here, in order to fit it on the page.
This utility is a Perl script contributed by Matt Hubbard. It runs
exim -bpu
to obtain a queue listing with undelivered recipients only, and then greps the output to select messages that match given criteria. The following selection options are available:
Match the sender address. The field that is tested is enclosed in angle brackets, so you can test for bounce messages with
exiqgrep -f '^<>$'
Match a recipient address. The field that is tested is not enclosed in angle brackets.
Match against the size field.
Match messages that are younger than the given time.
Match messages that are older than the given time.
Match only frozen messages.
Match only non-frozen messages.
The following options control the format of the output:
Display only the count of matching messages.
Long format – display the full message information as output by Exim. This is the default.
Display message ids only.
Brief format – one line per message.
Display messages in reverse order.
There is one more option, -h, which outputs a list of options.
The exiqsumm utility is a Perl script which reads the output of exim -bp and produces a summary of the messages on the queue. Thus, you use it by running a command such as
exim -bp | exiqsumm
The output consists of one line for each domain that has messages waiting for it, as in the following example:
3 2322 74m 66m msn.com.example
Each line lists the number of pending deliveries for a domain, their total volume, and the length of time that the oldest and the newest messages have been waiting. Note that the number of pending deliveries is greater than the number of messages when messages have more than one recipient.
A summary line is output at the end. By default the output is sorted on the domain name, but exiqsumm has the options -a and -c, which cause the output to be sorted by oldest message and by count of messages, respectively.
The output of exim -bp contains the original addresses in the message, so this also applies to the output from exiqsumm. No domains from addresses generated by aliasing or forwarding are included (unless the one_time option of the redirect router has been used to convert them into “top level” addresses).
The exigrep utility is a Perl script that searches one or more main log files for entries that match a given pattern. When it finds a match, it extracts all the log entries for the relevant message, not just those that match the pattern. Thus, exigrep can extract complete log entries for a given message, or all mail for a given user, or for a given host, for example. The usage is:
exigrep [-l] [-t<n>] <pattern> [<log file>] ...
The -t argument specifies a number of seconds. It adds an additional condition for message selection. Messages that are complete are shown only if they spent more than <n> seconds on the queue.
The -l flag means “literal”, that is, treat all characters in the pattern as standing for themselves. Otherwise the pattern must be a Perl regular expression. The pattern match is case-insensitive. If no file names are given on the command line, the standard input is read.
If the location of a zcat command is known from the definition of ZCAT_COMMAND in Local/Makefile, exigrep automatically passes any file whose name ends in COMPRESS_SUFFIX through zcat as it searches it.
The exicyclog script can be used to cycle (rotate) mainlog and rejectlog files. This is not necessary if only syslog is being used, or if you are using log files with datestamps in their names (see section 45.3). Some operating systems have their own standard mechanisms for log cycling, and these can be used instead of exicyclog if preferred.
Each time exicyclog is run the file names get “shuffled down” by one. If the main log file name is mainlog (the default) then when exicyclog is run mainlog becomes mainlog.01, the previous mainlog.01 becomes mainlog.02 and so on, up to a limit which is set in the script, and which defaults to 10. Reject logs are handled similarly.
If no mainlog file exists, the script does nothing. Files that “drop off” the end are deleted. All files with numbers greater than 01 are compressed, using a compression command which is configured by the COMPRESS_COMMAND setting in Local/Makefile. It is usual to run exicyclog daily from a root crontab entry of the form
1 0 * * * su exim -c /usr/exim/bin/exicyclog
assuming you have used the name “exim” for the Exim user. You can run exicyclog as root if you wish, but there is no need.
A Perl script called eximstats is provided for extracting statistical information from log files. The output is either plain text, or HTML. Exim log files are also suported by the Lire system produced by the LogReport Foundation (http://www.logreport.org).
The eximstats script has been hacked about quite a bit over time. The latest version is the result of some extensive revision by Steve Campbell. A lot of information is given by default, but there are options for suppressing various parts of it. Following any options, the arguments to the script are a list of files, which should be main log files. For example:
eximstats -nr /var/spool/exim/log/mainlog.01
By default, eximstats extracts information about the number and volume of messages received from or delivered to various hosts. The information is sorted both by message count and by volume, and the top fifty hosts in each category are listed on the standard output. Similar information, based on email addresses or domains instead of hosts can be requested by means of various options. For messages delivered and received locally, similar statistics are also produced per user.
The output also includes total counts and statistics about delivery errors, and histograms showing the number of messages received and deliveries made in each hour of the day. A delivery with more than one address in its envelope (for example, an SMTP transaction with more than one RCPT command) is counted as a single delivery by eximstats.
Though normally more deliveries than receipts are reported (as messages may have multiple recipients), it is possible for eximstats to report more messages received than delivered, even though the queue is empty at the start and end of the period in question. If an incoming message contains no valid recipients, no deliveries are recorded for it. A bounce message is handled as an entirely separate message.
eximstats always outputs a grand total summary giving the volume and number of messages received and deliveries made, and the number of hosts involved in each case. It also outputs the number of messages that were delayed (that is, not completely delivered at the first attempt), and the number that had at least one address that failed.
The remainder of the output is in sections that can be independently disabled or modified by various options. It consists of a summary of deliveries by transport, histograms of messages received and delivered per time interval (default per hour), information about the time messages spent on the queue, a list of relayed messages, lists of the top fifty sending hosts, local senders, destination hosts, and destination local users by count and by volume, and a list of delivery errors that occurred.
The relay information lists messages that were actually relayed, that is, they came from a remote host and were directly delivered to some other remote host, without being processed (for example, for aliasing or forwarding) locally.
The options for eximstats are as follows:
The “league tables” are computed on the basis of the superior domains of the sending hosts instead of the sending and receiving hosts. This option may be combined with -byhost and/or -byemail.
This is a synonym for -byemaildomain.
The “league tables” are computed on the basis of complete email addresses, instead of sending and receiving hosts. This option may be combined with -byhost and/or -bydomain.
The “league tables” are computed on the basis of the sender's email domain instead of the sending and receiving hosts. This option may be combined with -byhost, -bydomain, or -byemail.
The “league tables” are computed on the basis of sending and receiving hosts. This is the default option. It may be combined with -bydomain and/or -byemail.
Cache results of timegm() lookups. This results in a significant speedup when processing hundreds of thousands of messages, at a cost of increasing the memory utilisation.
When -charts is specified, create the charts in the directory <dir>.
When -charts is specified, this option specifies the relative directory for the img src= tags from where to include the charts.
Create graphical charts to be displayed in HTML output. This requires the GD, GDTextUtil, and GDGraph Perl modules, which can be obtained from http://www.cpan.org/modules/01modules.index.html.
To install these, download and unpack them, then use the normal Perl installation procedure:
perl Makefile.PL make make test make install
This is a debug flag. It causes eximstats to output the eval()'d parser to the standard output, which makes it easier to trap errors in the eval section. Remember to add one to the line numbers to allow for the title.
Show help information about eximstats' options.
This option controls the histograms of messages received and deliveries per time interval. By default the time interval is one hour. If -h0 is given, the histograms are suppressed; otherwise the value of <n> gives the number of divisions per hour. Valid values are 0, 1, 2, 3, 5, 10, 15, 20, 30 or 60, so -h2 sets an interval of 30 minutes, and the default is equivalent to -h1.
Output the results in HTML instead of plain text.
This option causes eximstats to merge old reports into a combined report. When this option is used, the input files must be outputs from previous calls to eximstats, not raw log files. For example, you could produce a set of daily reports and a weekly report by commands such as
eximstats mainlog.sun > report.sun.txt eximstats mainlog.mon > report.mon.txt eximstats mainlog.tue > report.tue.txt eximstats mainlog.wed > report.wed.txt eximstats mainlog.thu > report.thu.txt eximstats mainlog.fri > report.fri.txt eximstats mainlog.sat > report.sat.txt eximstats -merge -html report.*.txt > weekly_report.html
You can merge text or html reports and output the results as text or html. You can use all the normal eximstats output options, but only data included in the original reports can be shown. When merging reports, some loss of accuracy may occur in the “league tables”, towards the ends of the lists. The order of items in the “league tables” may vary when the data volumes round to the same value.
Suppress the display of information about failed deliveries (errors).
Suppress information about messages relayed through this host.
Suppress information about relayed messages that match the pattern, which is matched against a string of the following form (split over two lines here in order to fit it on the page):
H=<host> [<ip address>] A=<sender address> => H=<host> A=<recipient address>
for example
H=in.host [1.2.3.4] A=from@some.where.example => H=out.host A=to@else.where.example
The sending host name appears in parentheses if it has not been verified as matching the IP address. The mail addresses are taken from the envelope, not the headers. This option allows you to screen out hosts whom you are happy to have using your host as a relay.
Suppress the statistics about delivery by transport.
Suppress the statistics about delivery by any transport whose name matches the pattern. If you are using one transport to send all messages to a scanning mechanism before doing the real delivery, this feature can be used to omit that transport from your normal statistics (on the grounds that it is of no interest).
Count lines matching specified patterns and show them in the results. For example:
-pattern 'Refused connections' '/refused connection/'
This option can be specified multiple times.
Suppress information about times messages spend on the queue.
This option sets an alternative list of time intervals for the queueing information. The values are separated by commas and are in seconds, but can involve arithmetic multipliers, so for example you can set 3*60 to specify 3 minutes. A setting such as
-q60,5*60,10*60
causes eximstats to give counts of messages that stayed on the queue for less than one minute, less than five minutes, less than ten minutes, and over ten minutes.
Sets the “top” count to <n>. This controls the listings of the “top <n>” hosts and users by count and volume. The default is 50, and setting 0 suppresses the output altogether.
Omit local information from the “top” listings.
Include remote users in the “top” listings.
The -bh command line argument allows you to run a fake SMTP session with debugging output, in order to check what Exim is doing when it is applying policy controls to incoming SMTP mail. However, not everybody is sufficiently familiar with the SMTP protocol to be able to make full use of -bh, and sometimes you just want to answer the question Does this address have access? without bothering with any further details.
The exim_checkaccess utility is a “packaged” version of -bh. It takes two arguments, an IP address and an email address:
exim_checkaccess 10.9.8.7 A.User@a.domain.example
The utility runs a call to Exim with the -bh option, to test whether the given email address would be accepted in a RCPT command in a TCP/IP connection from the host with the given IP address. The output of the utility is either the word “accepted”, or the SMTP error response, for example:
Rejected: 550 Relay not permitted
When running this test, the utility uses <> as the envelope sender address for the MAIL command, but you can change this by providing additional options. These are passed directly to the Exim command. For example, to specify that the test is to be run with the sender address himself@there.example you can use:
exim_checkaccess 10.9.8.7 A.User@a.domain.example \ -f himself@there.example
Note that these additional Exim command line items must be given after the two mandatory arguments.
Because the exim_checkaccess uses -bh, it does not perform callouts while running its checks. You can run checks that include callouts by using -bhc, but this is not yet available in a “packaged” form.
The exim_dbmbuild program reads an input file containing keys and data in the format used by the lsearch lookup (see section 9.2). It writes a DBM file using the lower-cased alias names as keys and the remainder of the information as data. The lower-casing can be prevented by calling the program with the -nolc option.
A terminating zero is included as part of the key string. This is expected by the dbm lookup type. However, if the option -nozero is given, exim_dbmbuild creates files without terminating zeroes in either the key strings or the data strings. The dbmnz lookup type can be used with such files.
The program requires two arguments: the name of the input file (which can be a single hyphen to indicate the standard input), and the name of the output file. It creates the output under a temporary name, and then renames it if all went well. If the native DB interface is in use (USE_DB is set in a compile-time configuration file – this is common in free versions of Unix) the two file names must be different, because in this mode the Berkeley DB functions create a single output file using exactly the name given. For example,
exim_dbmbuild /etc/aliases /etc/aliases.db
reads the system alias file and creates a DBM version of it in /etc/aliases.db.
In systems that use the ndbm routines (mostly proprietary versions of Unix), two files are used, with the suffixes .dir and .pag. In this environment, the suffixes are added to the second argument of exim_dbmbuild, so it can be the same as the first. This is also the case when the Berkeley functions are used in compatibility mode (though this is not recommended), because in that case it adds a .db suffix to the file name.
If a duplicate key is encountered, the program outputs a warning, and when it finishes, its return code is 1 rather than zero, unless the -noduperr option is used. By default, only the first of a set of duplicates is used – this makes it compatible with lsearch lookups. There is an option -lastdup which causes it to use the data for the last duplicate instead. There is also an option -nowarn, which stops it listing duplicate keys to stderr. For other errors, where it doesn't actually make a new file, the return code is 2.
A utility called exinext (mostly a Perl script) provides the ability to fish specific information out of the retry database. Given a mail domain (or a complete address), it looks up the hosts for that domain, and outputs any retry information for the hosts or for the domain. At present, the retry information is obtained by running exim_dumpdb (see below) and post-processing the output. For example:
exinext piglet@milne.fict.example kanga.milne.fict.example:192.168.8.1 error 146: Connection refused first failed: 21-Feb-1996 14:57:34 last tried: 21-Feb-1996 14:57:34 next try at: 21-Feb-1996 15:02:34 roo.milne.fict.example:192.168.8.3 error 146: Connection refused first failed: 20-Jan-1996 13:12:08 last tried: 21-Feb-1996 11:42:03 next try at: 21-Feb-1996 19:42:03 past final cutoff time
You can also give exinext a local part, without a domain, and it will give any retry information for that local part in your default domain. A message id can be used to obtain retry information pertaining to a specific message. This exists only when an attempt to deliver a message to a remote host suffers a message-specific error (see section 43.2). exinext is not particularly efficient, but then it isn't expected to be run very often.
Three utility programs are provided for maintaining the DBM files that Exim uses to contain its delivery hint information. Each program requires two arguments. The first specifies the name of Exim's spool directory, and the second is the name of the database it is to operate on. These are as follows:
retry: the database of retry information
wait-<transport name>: databases of information about messages waiting for remote hosts
misc: other hints data (for example, for serializing ETRN runs)
The entire contents of a database are written to the standard output by the exim_dumpdb program, which has no options or arguments other than the spool and database names. For example, to dump the retry database:
exim_dumpdb /var/spool/exim retry
Two lines of output are produced for each entry:
T:mail.ref.example:192.168.242.242 146 77 Connection refused
31-Oct-1995 12:00:12 02-Nov-1995 12:21:39 02-Nov-1995 20:21:39 *
The first item on the first line is the key of the record. It starts with one of the letters R, or T, depending on whether it refers to a routing or transport retry. For a local delivery, the next part is the local address; for a remote delivery it is the name of the remote host, followed by its failing IP address (unless no_retry_include_ip_address is set on the smtp transport). Then there follows an error code, an additional error code, and a textual description of the error.
The three times on the second line are the time of first failure, the time of the last delivery attempt, and the computed time for the next attempt. The line ends with an asterisk if the cutoff time for the last retry rule has been exceeded.
Each output line from exim_dumpdb for the wait-xxx databases consists of a host name followed by a list of ids for messages that are or were waiting to be delivered to that host. If there are a very large number for any one host, continuation records, with a sequence number added to the host name, may be seen. The data in these records is often out of date, because a message may be routed to several alternative hosts, and Exim makes no effort to keep cross-references.
The exim_tidydb utility program is used to tidy up the contents of the hints databases. If run with no options, it removes all records from a database that are more than 30 days old. The cutoff date can be altered by means of the -t option, which must be followed by a time. For example, to remove all records older than a week from the retry database:
exim_tidydb -t 7d /var/spool/exim retry
Both the wait-xxx and retry databases contain items that involve message ids. In the former these appear as data in records keyed by host – they were messages that were waiting for that host – and in the latter they are the keys for retry information for messages that have suffered certain types of error. When exim_tidydb is run, a check is made to ensure that message ids in database records are those of messages that are still on the queue. Message ids for messages that no longer exist are removed from wait-xxx records, and if this leaves any records empty, they are deleted. For the retry database, records whose keys are non-existent message ids are removed. The exim_tidydb utility outputs comments on the standard output whenever it removes information from the database.
Removing records from a DBM file does not normally make the file smaller, but all the common DBM libraries are able to re-use the space that is released. It is therefore suggested that exim_tidydb be run periodically on all the hints databases, but at a quiet time of day, because it requires a database to be locked (and therefore inaccessible to Exim) while it does its work.
The exim_fixdb program is a utility for interactively modifying databases. Its main use is for testing Exim, but it might also be occasionally useful for getting round problems in a live system. It has no options, and its interface is somewhat crude. On entry, it prompts for input with a right angle-bracket. A key of a database record can then be entered, and the data for that record is displayed.
If “d” is typed at the next prompt, the entire record is deleted. For all except the retry database, that is the only operation that can be carried out. For the retry database, each field is output preceded by a number, and data for individual fields can be changed by typing the field number followed by new data, for example:
> 4 951102:1000
resets the time of the next delivery attempt. Time values are given as a sequence of digit pairs for year, month, day, hour, and minute. Colons can be used as optional separators.
The exim_lock utility locks a mailbox file using the same algorithm as Exim. For a discussion of locking issues, see section 26.3. Exim_lock can be used to prevent any modification of a mailbox by Exim or a user agent while investigating a problem. The utility requires the name of the file as its first argument. If the locking is successful, the second argument is run as a command (using C's system() function); if there is no second argument, the value of the SHELL environment variable is used; if this is unset or empty, /bin/sh is run. When the command finishes, the mailbox is unlocked and the utility ends. The following options are available:
Use fcntl() locking on the open mailbox.
Use flock() locking on the open mailbox, provided the operating system supports it.
This must be followed by a number, which is a number of seconds; it sets the interval to sleep between retries (default 3).
Create a lock file before opening the mailbox.
Lock the mailbox using MBX rules.
Suppress verification output.
This must be followed by a number; it sets the number of times to try to get the lock (default 10).
This option causes exim_lock to restore the modified and read times to the the locked file before exiting. This allows you to access a locked mailbox (for example, to take a backup copy) without disturbing the times that the user subsequently sees.
This must be followed by a number, which is a number of seconds; it sets a timeout to be used with a blocking fcntl() lock. If it is not set (the default), a non-blocking call is used.
Generate verbose output.
If none of -fcntl, -flock, -lockfile or -mbx are given, the default is to create a lock file and also to use fcntl() locking on the mailbox, which is the same as Exim's default. The use of -flock or -fcntl requires that the file be writeable; the use of -lockfile requires that the directory containing the file be writeable. Locking by lock file does not last for ever; Exim assumes that a lock file is expired if it is more than 30 minutes old.
The -mbx option can be used with either or both of -fcntl or -flock. It assumes -fcntl by default. MBX locking causes a shared lock to be taken out on the open mailbox, and an exclusive lock on the file /tmp/.n.m where n and m are the device number and inode number of the mailbox file. When the locking is released, if an exclusive lock can be obtained for the mailbox, the file in /tmp is deleted.
The default output contains verification of the locking that takes place. The -v option causes some additional information to be given. The -q option suppresses all output except error messages.
A command such as
exim_lock /var/spool/mail/spqr
runs an interactive shell while the file is locked, whereas
exim_lock -q /var/spool/mail/spqr <<End
<some commands>
End
runs a specific non-interactive sequence of commands while the file is locked, suppressing all verification output. A single command can be run by a command such as
exim_lock -q /var/spool/mail/spqr \ "cp /var/spool/mail/spqr /some/where"
Note that if a command is supplied, it must be entirely contained within the second argument – hence the quotes.