Exim writes three different logs, referred to as the main log, the reject log, and the panic log:
The main log records the arrival of each message and each delivery in a single line in each case. The format is as compact as possible, in an attempt to keep down the size of log files. Two-character flag sequences make it easy to pick out these lines. A number of other events are recorded in the main log. Some of them are optional, in which case the log_selector option controls whether they are included or not. A Perl script called eximstats which does simple analysis of main log files is provided in the Exim distribution (see section 45.5).
The reject log records information from messages that are rejected as a result of a configuration option (that is, for policy reasons). If the message's header has been read at the time the log is written, its contents are written to this log, following a copy of the one-line message that is written to the main log. You can use the reject log to check that your policy controls are working correctly.
When certain serious errors occur, Exim writes entries to its panic log. If the error is sufficiently disastrous, Exim bombs out afterwards. Panic log entries are usually written to the main log as well, but can get lost amid the mass of other entries. The panic log should be empty under normal circumstances. It is therefore a good idea to check it (or to have a cron script check it) regularly, in order to become aware of any problems. When Exim cannot open its panic log, it tries as a last resort to write to the system log (syslog). This is opened with LOG_PID+LOG_CONS and the facility code of LOG_MAIL. The message itself is written at priority LOG_CRIT.
Every log line starts with a timestamp, in the format shown in this example:
2001-09-16 16:09:47 SMTP connection from [127.0.0.1] closed by QUIT
The logs may be written to local files, or to syslog, or both. However, it should be noted that many syslog implementations use UDP as a transport, and are therefore unreliable in the sense that messages are not guaranteed to arrive at the loghost, nor is the ordering of messages necessarily maintained. It has also been reported that on large log files (tens of megabytes) you may need to tweak syslog to prevent it syncing the file with each write - on Linux this has been seen to make syslog take 90% plus of CPU time.
The destination for Exim's logs is configured by setting LOG_FILE_PATH in Local/Makefile or by setting log_file_path in the run time configuration. This latter string is expanded, so it can contain, for example, references to the host name:
log_file_path = /var/log/$primary_hostname/exim_%slog
It is generally advisable, however, to set the string in Local/Makefile rather than at run time, because then the setting is available right from the start of Exim's execution. Otherwise, if there's something it wants to log before it has read the configuration file (for example, an error in the configuration file) it will not use the path you want, and may not be able to log at all.
The value of LOG_FILE_PATH or log_file_path is a colon-separated list, currently limited to at most two items. This is one option where the facility for changing a list separator may not be used. The list must always be colon-separated. If an item in the list is ``syslog'' then syslog is used; otherwise the item must either be an absolute path, containing %s at the point where ``main'', ``reject'', or ``panic'' is to be inserted, or be empty, implying the use of the default path, which is log/%slog in Exim's spool directory. The default path is used if nothing is specified. Here are some examples of possible settings:
| LOG_FILE_PATH=syslog | syslog only |
| LOG_FILE_PATH=:syslog | syslog and default path |
| LOG_FILE_PATH=syslog : /usr/log/exim_%s | syslog and specified path |
| LOG_FILE_PATH=/usr/log/exim_%s | specified path only |
If there are more than two paths in the list, the first is used and a panic error is logged.
Some operating systems provide centralized and standardised methods for cycling log files. For those that do not, a utility script called exicyclog is provided (see section 45.4). This renames and compresses the main and reject logs each time it is called. The maximum number of old logs to keep can be set. It is suggested this script is run as a daily cron job.
An Exim delivery process opens the main log when it first needs to write to it, and it keeps the file open in case subsequent entries are required - for example, if a number of different deliveries are being done for the same message. However, remote SMTP deliveries can take a long time, and this means that the file may be kept open long after it is renamed if exicyclog or something similar is being used to rename log files on a regular basis. To ensure that a switch of log files is noticed as soon as possible, Exim calls stat() on the main log's name before reusing an open file, and if the file does not exist, or its inode has changed, the old file is closed and Exim tries to open the main log from scratch. Thus, an old log file may remain open for quite some time, but no Exim processes should write to it once it has been renamed.
The use of syslog does not change what Exim logs or the format of its messages, except in one respect. If syslog_timestamp is set false, the timestamps on Exim's log lines are omitted when these lines are sent to syslog. Apart from that, the same strings are written to syslog as to log files. The syslog ``facility'' is set to LOG_MAIL, and the program name to ``exim''. On systems that permit it (all except ULTRIX) the LOG_PID flag is set so that the syslog() call adds the pid as well as the time and host name to each line. The three log streams are mapped onto syslog priorities as follows:
mainlog is mapped to LOG_INFO
rejectlog is mapped to LOG_NOTICE
paniclog is mapped to LOG_ALERT
Many log lines are written to both mainlog and rejectlog, so there will be duplicates if these are routed by syslog to the same place.
Exim's log lines can sometimes be very long, and some of its rejectlog entries contain multiple lines when headers are included. To cope with both these cases, entries written to syslog are split into separate syslog() calls at each internal newline, and also after a maximum of 1000 characters. To make it easy to re-assemble them later, each component of a split entry starts with a string of the form ``[<n>/<m>]'' or ``[<n>\<m>]'' where <n> is the component number and <m> is the total number of components in the entry. The / delimiter is used when the line was split because it was too long; if it was split because of an internal newline, the \ delimiter is used. For example, supposing the length limit to be 70 instead of 1000, the following would be the result of a typical rejection message to mainlog (LOG_INFO), each line in addition being preceded by the time, host name, and pid as added by syslog:
[1/3] 2002-09-16 16:09:43 16RdAL-0006pc-00 rejected from [127.0.0.1] (ph10):
[2/3] syntax error in 'From' header when scanning for sender: missing or ma
[3/3] lformed local part in "<>" (envelope sender is <ph10@cam.example>)
The same error might cause the following lines to be written to ``rejectlog'' (LOG_NOTICE):
[1/14] 2002-09-16 16:09:43 16RdAL-0006pc-00 rejected from [127.0.0.1] (ph10):
[2/14] syntax error in 'From' header when scanning for sender: missing or ma
[3\14] lformed local part in "<>" (envelope sender is <ph10@cam.example>)
[4\14] Recipients: ph10@some.domain.cam.example
[5\14] P Received: from [127.0.0.1] (ident=ph10)
[6\14] by xxxxx.cam.example with smtp (Exim 4.00)
[7\14] id 16RdAL-0006pc-00
[8\14] for ph10@cam.example; Mon, 16 Sep 2002 16:09:43 +0100
[9\14] F From: <>
[10\14] Subject: this is a test header
[11\14] X-something: this is another header
[12\14] I Message-Id: <E16RdAL-0006pc-00@xxxxx.cam.example>
[13\14] B Bcc:
[14/14] Date: Mon, 16 Sep 2002 16:09:43 +0100
Log lines that are neither too long nor contain newlines are written to syslog without modification.
If only syslog is being used, the Exim monitor is unable to provide a log tail display, unless syslog is routing mainlog to a file on the local host and the environment variable EXIMON_LOG_FILE_PATH is set to tell the monitor where it is.
One line is written to the main log for each message received, and for each successful, unsuccessful, and delayed delivery. These lines can readily be picked out by the distinctive two-character flags that immediately follow the timestamp. The flags are:
| <= | message arrival |
| => | normal message delivery |
| -> | additional address in same delivery |
| *> | delivery suppressed by -N |
| ** | delivery failed; address bounced |
| == | delivery deferred; temporary problem |
The format of the single-line entry in the main log that is written for every message received is shown in the basic example below, which is split over several lines in order to fit it on the page:
2002-10-31 08:57:53 16ZCW1-0005MB-00 <= kryten@dwarf.fict.example
H=mailer.fict.example [192.168.123.123] U=exim
P=smtp S=5678 id=<incoming message id>
The address immediately following ``<='' is the envelope sender address. A bounce message is shown with the sender address ``<>'', and if it is locally generated, this is followed by an item of the form
R=<message id>
which is a reference to the message that caused the bounce to be sent.
For messages from other hosts, the H and U fields identify the remote host and record the RFC 1413 identity of the user that sent the message, if one was received. The number given in square brackets is the IP address of the sending host. If there is just a single host name in the H field, as above, it has been verified to correspond to the IP address (see the host_lookup option). If the name is in parentheses, it was the name quoted by the remote host in the SMTP HELO or EHLO command, and has not been verified. If verification yields a different name to that given for HELO or EHLO, the verified name appears first, followed by the HELO or EHLO name in parentheses.
Misconfigured hosts (and mail forgers) sometimes put an IP address, with or without brackets, in the HELO or EHLO command, leading to entries in the log containing text like these examples:
H=(10.21.32.43) [192.168.8.34]
H=([10.21.32.43]) [192.168.8.34]
This can be confusing. Only the final address in square brackets can be relied on.
For locally generated messages (that is, messages not received over TCP/IP), the H field is omitted, and the U field contains the login name of the caller of Exim.
For all messages, the P field specifies the protocol used to receive the message. This is set to ``asmtp'' for messages received from hosts which have authenticated themselves using the SMTP AUTH command. In this case there is an additional item A= followed by the name of the authenticator that was used. If an authenticated identification was set up by the authenticator's server_set_id option, this is logged too, separated by a colon from the authenticator name.
The id field records the existing message id, if present. The size of the received message is given by the S field. When the message is delivered, headers may get removed or added, so that the size of delivered copies of the message may not correspond with this value (and indeed may be different to each other).
The log_selector option can be used to request the logging of additional data when a message is received. See section 44.13 below.
The format of the single-line entry in the main log that is written for every delivery is shown in one of the examples below, for local and remote deliveries, respectively. Each example has been split into two lines in order to fit it on the page:
2002-10-31 08:59:13 16ZCW1-0005MB-00 => marv <marv@hitch.fict.example>
R=localuser T=local_delivery
2002-10-31 09:00:10 16ZCW1-0005MB-00 => monk@holistic.fict.example
R=dnslookup T=remote_smtp H=holistic.fict.example [192.168.234.234]
For ordinary local deliveries, the original address is given in angle brackets after the final delivery address, which might be a pipe or a file. If intermediate address(es) exist between the original and the final address, the last of these is given in parentheses after the final address. The R and T fields record the router and transport that were used to process the address.
If a shadow transport was run after a successful local delivery, the log line for the successful delivery has an item added on the end, of the form
ST=<shadow transport name>
If the shadow transport did not succeed, the error message is put in parentheses afterwards.
When more than one address is included in a single delivery (for example, two SMTP RCPT commands in one transaction) the second and subsequent addresses are flagged with ``->'' instead of ``=>''. When two or more messages are delivered down a single SMTP connection, an asterisk follows the IP address in the log lines for the second and subsequent messages.
The generation of a reply message by a filter file gets logged as a ``delivery'' to the addressee, preceded by ``>''.
The log_selector option can be used to request the logging of additional data when a message is received. See section 44.13 below.
When a message is discarded as a result of the command ``seen finish'' being obeyed in a filter file which generates no deliveries, a log entry of the form
2002-12-10 00:50:49 16auJc-0001UB-00 => discarded
<low.club@bridge.example> R=userforward
is written, to record why no deliveries are logged. When a message is discarded because it is aliased to ``:blackhole:'' the log line is like this:
1999-03-02 09:44:33 10HmaX-0005vi-00 => :blackhole: <hole@nowhere.example> R=blackhole_router
When a delivery is deferred, a line of the following form is logged:
2002-12-19 16:20:23 16aiQz-0002Q5-00 == marvin@endrest.example
R=dnslookup T=smtp defer (146): Connection refused
In the case of remote deliveries, the error is the one that was given for the last IP address that was tried. Details of individual SMTP failures are also written to the log, so the above line would be preceded by something like
2002-12-19 16:20:23 16aiQz-0002Q5-00 Failed to connect to
mail1.endrest.example [192.168.239.239]: Connection refused
When a deferred address is skipped because its retry time has not been reached, a message is written to the log, but this can be suppressed by setting an appropriate value in log_selector.
If a delivery fails, a line of the following form is logged:
2002-12-19 16:20:23 16aiQz-0002Q5-00 ** jim@trek99.example
<jimtrek99.example>: unknown mail domain
If a delivery does not actually take place because the -N option has been used to suppress it, a normal delivery line is written to the log, except that ``=>'' is replaced by ``*>''.
A line of the form
2002-10-31 09:00:11 16ZCW1-0005MB-00 Completed
is written to the main log when a message is about to be removed from the spool at the end of its processing.
Various other types of log entry are written from time to time. Most should be self-explanatory. Among the more common are:
retry time not reached An address previously suffered a temporary error during routing or local delivery, and the time to retry has not yet arrived. This message is not written to an individual message log file unless it happens during the first delivery attempt.
retry time not reached for any host An address previously suffered temporary errors during remote delivery, and the retry time has not yet arrived for any of the hosts to which it is routed.
spool file locked An attempt to deliver a message cannot proceed because some other Exim process is already working on the message. This can be quite common if queue running processes are started at frequent intervals. The exiwhat utility script can be used to find out what Exim processes are doing.
By setting the log_selector global option, you can disable some of Exim's default logging, or you can request additional logging. The value of log_selector is made up of names preceded by plus or minus characters. For example:
log_selector = +arguments -retry_defer
The list of optional log items is in the following table, with the default selection marked by asterisks:
| address_rewrite | address rewriting |
| all_parents | all parents in => lines |
| arguments | command line arguments |
| *connection_reject | connection rejections |
| *delay_delivery | immediate delivery delayed (message queued) |
| delivery_size | add S=nnn to => lines |
| *dnslist_defer | defers of DNS list (aka RBL) lookups |
| *etrn | ETRN commands |
| incoming_interface | incoming interface on <= lines |
| incoming_port | incoming port on <= lines |
| *lost_incoming_connection | as it says (includes timeouts) |
| *queue_run | start and end queue runs |
| received_recipients | recipients on <= lines |
| received_sender | sender on <= lines |
| *retry_defer | ``retry time not reached'' |
| sender_on_delivery | add sender to => lines |
| *size_reject | rejection because too big |
| *skip_delivery | ``message is frozen'', ``spool file is locked'' |
| smtp_confirmation | SMTP confirmation on <= lines |
| smtp_connection | SMTP connections |
| smtp_protocol_error | SMTP protocol errors |
| smtp_syntax_error | SMTP syntax errors |
| subject | contents of Subject: on <= lines |
| *tls_cipher | TLS cipher on <= lines |
| tls_peerdn | TLS peer DN on <= lines |
| all | all of the above |
More details on each of these items follows:
address_rewrite: This applies both to global rewrites and per-transport rewrites.
all_parents: Normally only the original and final addresses are logged on delivery lines; with this selector, intermediate parents are given in parentheses between them.
arguments: This causes Exim to write the arguments with which it was called to the main log. This is a debugging feature, added to make it easier to find out with what arguments certain MUAs call /usr/sbin/sendmail. The logging does not happen if Exim has given up root privilege because it was called with the -C or -D options. This facility cannot log unrecognized arguments, because the arguments are checked before the configuration file is read. The only way to log such cases is to interpose a script such as util/logargs.sh between the caller and Exim.
connection_reject: A log entry is written whenever an incoming SMTP connection is rejected, for whatever reason.
delay_delivery: A log entry is written whenever a delivery process is not started for an incoming message because the load is too high or too many messages were received on one connection. Logging does not occur if no delivery process is started because queue_only is set or -odq was used.
delivery_size: For each delivery, the size of message delivered is added to the ``=>'' line, tagged with S=.
dnslist_defer: A log entry is written if an attempt to look up a host in a DNS black list suffers a temporary error.
etrn: Every legal ETRN command that is received is logged, before the ACL is run to determine whether or not it is actually accepted. An invalid ETRN command, or one received within a message transaction is not logged by this selector (see smtp_syntax_error and smtp_protocol_error).
incoming_interface: The interface on which a message was received is added to the ``<='' line as an IP address in square brackets, tagged by I= and followed by a colon and the port number.
incoming_port: The remote port number from which a message was received is added to log entries and Received: header lines, following the IP address in square brackets, and separated from it by a colon. This is implemented by changing the value that is put in the $sender_fullhost and $sender_rcvhost variables. Recording the remote port number has become more important with the widening use of NAT (see RFC 2505).
lost_incoming_connection: A log line is written when an incoming SMTP connection is unexpectedly dropped.
queue_run: The start and end of every queue run are logged.
received_recipients: The recipients of a message are listed in the main log as soon as the message is received. The list appears at the end of the log line that is written when a message is received, preceded by the word ``for''. The addresses are listed after they have been qualified, but before any rewriting has taken place.
received_sender: The unrewritten original sender of a message is added to the end of the log line that records the message's arrival, after the word ``from'' (before the recipients if received_recipients is also set).
retry_defer: A log line is written if a delivery is deferred because a retry time has not yet been reached. However, this ``retry time not reached'' message is always omitted from individual message logs after the first delivery attempt.
sender_on_delivery: The message's sender address is added to every delivery and bounce line, tagged by F= (for ``from'').
size_reject: A log line is written whenever a message is rejected because it is too big.
skip_delivery: A log line is written whenever a message is skipped during a queue run because it is frozen or because another process is already delivering it.
smtp_confirmation: The response to the final ``.'' in the SMTP dialogue for outgoing messages is added to delivery log lines in the form ``C="<text>"''. A number of MTAs (including Exim) return an identifying string in this response.
smtp_connection: A log line is written whenever an SMTP connection is established or closed. (By contrast, lost_incoming_connection applies only when the closure is unexpected.) This applies to connections from local processes that use -bs as well as to TCP/IP connections. If a connection is dropped in the middle of a message, a log line is always written, whether this selector is set or not, but otherwise nothing is written at the start and end of connections unless this selector is enabled. For TCP/IP connections to an Exim daemon, the current number of connections is included in the log message for each new connection, but note that the count is reset if the daemon is restarted.
smtp_protocol_error: A log line is written for every SMTP protocol error encountered.
smtp_syntax_error: A log line is written for every SMTP syntax error encountered. An unrecognized command is treated as a syntax error. For an external connection, the host identity is given; for an internal connection using -bs the sender identification (normally the calling user) is given.
subject: The subject of the message is added to the arrival log line, preceded by ``T='' (T for ``topic'', since S is already used for ``size'').
tls_cipher: When a message is received over an encrypted connection, the cipher used is added to the arrival line, preceded by X=.
tls_peerdn: When a message is received over an encrypted connection, and a certificate was supplied by the client, the peer DN is added to the arrival log line, preceded by DN=.
In addition to the general log files, Exim writes a log file for each message that it handles. The names of these per-message logs are the message ids, and they are kept in the msglog sub-directory of the spool directory. Each message log contains copies of the log lines that apply to the message. This makes it easier to inspect the status of an individual message without having to search the main log. A message log is deleted when processing of the message is complete, unless preserve_message_logs is set, but this should be used only with great care because they can fill up your disc very quickly.