Two kinds of SMTP processing are supported: SMTP over TCP/IP, and so-called `batched SMTP'. The latter is the name for a process in which batches of messages are stored in files, using SMTP commands as separators and to contain the envelope information. Such batches are delivered to or received from other systems using some transport mechanism other than Exim. For each kind of SMTP processing there are two aspects: outgoing and incoming.
This is implemented by the `smtp' transport. If the greeting line from the remote host contains the string `ESMTP', Exim sends an EHLO command instead of HELO, and if it is told that the SIZE parameter is supported, it adds SIZE=<n> to each subsequent MAIL command. The value of <n> is the message size plus the value of the `size_addition' option (default 1024) to allow for additions to the message such as per-transport header lines, or changes made in a transport filter. If `size_addition' is set negative, the use of SIZE is suppressed.
Responses from the remote host are supposed to be terminated by CR followed by LF. However, there are known to be hosts that do not send CR characters, so in order to be able to interwork with such hosts, Exim treats LF on its own as a line terminator.
If a message contains a number of different addresses, all those with the same characteristics (for example, the same envelope sender) that resolve to the same set of hosts, in the same order, are sent in a single SMTP transaction, even if they are for different domains, unless there are more than the setting of the `max_rcpts' option in the `smtp' transport allows, in which case they are split into groups containing no more than `max_rcpts' addresses each. If `remote_max_parallel' is greater than one, such groups may be sent in parallel sessions. The order of hosts with identical MX values is not significant when checking whether addresses can be batched in this way.
When the `smtp' transport suffers a temporary failure that is not message-related, Exim updates its transport-specific database, which contains records indexed by host name that remember which messages are waiting for each particular host. It also updates the retry database with new retry times. Exim's retry hints are based on host name plus IP address, so if one address of a multi-homed host is broken, it will soon be skipped most of the time. See the next section for more detail about error handling.
When a message is successfully delivered over a TCP/IP SMTP connection, Exim looks in the hints database for the transport to see if there are any queued messages waiting for the host to which it is connected. If it finds one, it creates a new Exim process using the `-MC' option (which can only be used by a process running as root or the Exim user) and passes the TCP/IP socket to it. The new process does only those deliveries that are routed to the connected host, and may in turn pass the socket on to a third process, and so on. The `batch_max' option of the `smtp' transport can be used to limit the number of messages sent down a single connection. The second and subsequent messages delivered down an SMTP connection are identified in the main log by the addition of an asterisk after the closing square bracket of the IP address.
When a delivery process that is part of a queue run passes a socket on to a new delivery process, it writes the new process' id to a file in Exim's spool directory whose name is `qr' followed by the process id of the queue runner. It also passes on the queue runner's process id using the `-MCQ' option, in case the socket is passed on to yet another process. The queue running process reads the file when the original process ends, and waits for each process listed therein to finish before proceeding. Because each process writes the process id of its child into the file before it itself finishes, the end of the file is reached only when the entire tree of processes spawned by the original delivery process is complete.
Three different kinds of error are recognized for outgoing SMTP: host errors, message errors, and recipient errors.
In all cases, if there are other hosts (or IP addresses) available for the current set of addresses (for example, from multiple MX records), they are tried in this run for any undelivered addresses, subject of course to their own retry data. In other words, recipient error retry data does not take effect until the next delivery attempt.
Some hosts have been observed to give temporary error responses to every MAIL command at certain times (`insufficient space' has been seen). It would be nice if such circumstances could be recognized, and defer data for the host itself created, but this is not possible within the current Exim design. What actually happens is that retry data for every (host, message) combination is created.
The reason that timeouts after MAIL and RCPT are treated specially is that these can sometimes arise as a result of the remote host's verification procedures. Exim makes this assumption, and treats them as if a temporary error response had been received. A timeout after `.' is treated specially because it is known that some broken implementations fail to recognize the end of the message if the last character of the last line is a binary zero. Thus is it helpful to treat this case as a message error.
Timeouts at other times are treated as host errors, assuming a problem with the host, or the connection to it. If a timeout after MAIL, RCPT, or `.' is really a connection problem, the assumption is that at the next try the timeout is likely to occur at some other point in the dialogue, causing it then to be treated as a host error.
There is experimental evidence that some MTAs drop the connection after the terminating `.' if they don't like the contents of the message for some reason, in contravention of the RFC, which indicates that a 5xx response should be given. That is why Exim treats this case as a message rather than a host error, in order not to delay other messages to the same host.
Variable Envelope Return Paths -- see `ftp://koobera.math.uic.edu/www/proto/verp.txt' -- can be supported in Exim by using the `return_path' generic transport option to rewrite the return path at transport time. For example, the following could be used on an smtp transport:
return_path = "\
${if match {$return_path}{^(.+?)-request@your.domain\\$}\
{$1-request=$local_part%$domain@your.domain}fail}"
This has the effect of rewriting the return path (envelope sender) on all outgoing SMTP messages, if the local part of the original return path ends in `-request', and the domain is `your.domain'. The rewriting inserts the local part and domain of the recipient into the return path. If, for example, a message with return path `somelist-request@your.domain' is sent to `subscriber@other.domain', the return path is rewritten as
somelist-request=subscriber%other.domain@your.domain
For this to work, you must arrange for outgoing messages that have `-request' in their return paths to have just a single recipient. This can be done by setting
max_rcpt = 1
in the smtp transport. Otherwise a single copy of a message might be addressed to several different recipients in the same domain, in which case `$local_part' is not available (because it is not unique). Of course, if you do start sending out messages with this kind of return path, you must also configure Exim to accept the bounce messages that come back to those paths. Typically this would be done by setting a `suffix' option in a suitable director.
The overhead incurred in using VERP depends very much on the size of the message, the number of recipient addresses that resolve to the same remote host, and the speed of the connection over which the message is being sent. If a lot of addresses resolve to the same host and the connection is slow, sending a separate copy of the message for each address may take substantially longer than sending a single copy with many recipients (for which VERP cannot be used).
Incoming SMTP messages can be accepted in one of two ways: by running a listening daemon, or by using `inetd'. In the latter case, the entry in `/etc/inetd.conf' should be like this:
smtp stream tcp nowait exim /opt/exim/bin/exim in.exim -bs
Exim distinguishes between this case and the case of a user agent using the `-bs' option by checking whether the standard input is a socket using the `getpeername()' function.
By default, Exim does not make a log entry when a remote hosts connects or disconnects (either via the daemon or `inetd'), unless the disconnection is unexpected. It can be made to write such log entries by setting the `log_smtp_connections' option.
Commands from the remote host are supposed to be terminated by CR followed by LF. However, there are known to be hosts that do not send CR characters, so in order to be able to interwork with such hosts, Exim treats LF on its own as a line terminator.
The amount of disc space available is checked whenever SIZE is received on a MAIL command, independently of whether `message_size_limit' or `check_spool_space' is configured, unless `smtp_check_spool_space' is set false. A temporary error is given if there isn't enough. If `check_spool_space' is set, the check is for that amount of space plus the value given with SIZE, that is, it checks that the addition of the incoming message will not reduce the space below the threshold.
When a message is successfully received, Exim includes the local message id in its response to the final `.' that terminates the data. If the remote host logs this text it can help with tracing what has happened to a message.
The Exim daemon can limit the number of simultaneous incoming connections it is prepared to handle (see the `smtp_accept_max' option). It can also limit the number of simultaneous incoming connections from a single remote host (see the `smtp_accept_max_per_host' option). Additional connection attempts are rejected using the SMTP temporary error code 421.
On some operating systems the SIGCHLD signal that is used to detect when a subprocess has finished can get lost at busy times. However, the daemon looks for completed subprocesses every time it wakes up, so provided there are other things happening (new incoming calls, starts of queue runs), the completion of processes created to handle incoming calls should get noticed eventually. If, however, Exim appears not to be accepting as many incoming connections as expected, sending the daemon a SIGCHLD signal will wake it up and cause it to check for any completed subprocesses.
When running as a daemon, Exim can reserve some SMTP slots for specific hosts, and can also be set up to reject SMTP calls from non-reserved hosts at times of high system load -- for details see the `smtp_accept_reserve', `smtp_load_reserve', and `smtp_reserve_hosts' options. The load check applies in both the daemon and `inetd' cases.
If neither `queue_only' nor the `-odq' command line option is set, Exim normally starts a delivery process for each message received. However, the number of simultaneously running delivery processes started in this way can be limited by the `smtp_accept_queue' and `smtp_accept_queue_per_connection' options, and the `queue_only_load' option can specify a system load average above which immediate delivery is suspended. When either limit is reached, subsequently received messages are just put on the input queue.
The controls that involve counts of incoming SMTP calls (`smtp_accept_max' `smtp_accept_queue', `smtp_accept_reserve') are not available when Exim is started up from the `inetd' daemon, since each connection is handled by an entirely independent Exim process. Control by load average is, however, available with `inetd'.
Exim can be configured to verify addresses in incoming SMTP commands as they are received. See chapter "Verification of incoming mail" for details. It can also be configured to rewrite addresses at this time -- before any syntax checking is done. See section "The SMTP-time rewriting flag" in chapter "Address rewriting".
The SMTP command VRFY is accepted only when the configuration option `smtp_verify' is set, and if so, it runs exactly the same code as when Exim is called with the `-bv' option.
The SMTP command EXPN is is permitted only if the calling host matches `smtp_expn_hosts' (add `localhost' if you want calls to 127.0.0.1 to be able to use it). A single-level expansion of the address is done. EXPN is treated as an `address test' (similar to the `-bt' option) rather than a verification (the `-bv' option). If an unqualified local part is given as the argument to EXPN, it is qualified with `qualify_domain'.
Rejections of VRFY and EXPN commands are logged on the main and reject logs, and VRFY verification failures are logged on the main log for consistency with RCPT failures.
The SMTP command DEBUG is not supported at all. Occurrences of this command are rejected, and the incident is logged.
RFC 1985 describes an SMTP command called ETRN that is designed to overcome the security problems of the TURN command (which has fallen into disuse). Exim recognizes ETRN if the calling host matches `smtp_etrn_hosts'. Attempts to use ETRN from other hosts are logged on the main and reject logs; when ETRN is accepted, it is logged on the main log.
The ETRN command is concerned with `releasing' messages that are awaiting delivery to certain hosts. As Exim does not organize its message queue by host, the only form of ETRN that is internally supported is the one where the text starts with the `#' prefix, in which case the remainder of the text is specific to the SMTP server. A valid ETRN command causes a run of Exim with the `-R' option to happen, with the remainder of the ETRN text as its argument. For example,
ETRN #brigadoon
causes a delivery attempt on all messages with undelivered addresses containing the text `brigadoon'.
For more control over what ETRN does, the `smtp_etrn_command' option can used. This specifies a command that is run whenever ETRN is received. For example:
smtp_etrn_command = /etc/etrn_command $domain $sender_host_address
The string is split up into arguments which are independently expanded. The expansion variable `$domain' is set to the argument of the ETRN command, and no syntax checking is done on the contents of this argument. A new freestanding process is created to run the command. Exim does not wait for it to complete, so its status code is not checked. As Exim is normally running under its own uid and gid when receiving incoming SMTP, it is not possible for it to change them before running the command.
Both the `appendfile' and `pipe' transports can be used for handling batched SMTP. Each has an option called `bsmtp' which, if set to anything other than `none' causes the message to be output in SMTP format. The message is written to the file or pipe preceded by the SMTP commands MAIL and RCPT, and followed by a line containing a single dot. The SMTP command HELO is not normally used, but if the transport's `bsmtp_helo' option is set true, a HELO command line precedes each message.
Lines in the message starting with a dot get an extra dot added. If the `prefix' option is set, its contents are included after the SMTP commands, and the contents of `suffix' appear at the end of the message, before the terminating dot; normally these options are specified as empty, to override the defaults.
The value of the `bsmtp' option determines how multiple addresses in a single message may be batched, if other conditions permit. If the value of `bsmtp' is `one', then there is no batching, and a copy of the message is output for each address. If the value is `domain' then a single copy (with multiple RCPT commands) is output for all addresses that have the same domain. If the value is `all' then only a single copy of the message is written. The batching is further constrained by other parameters:
Here is an example of a transport and router for batched SMTP:
# transport
smtp_appendfile:
driver = appendfile
directory = /var/bsmtp/${host}
bsmtp = all
prefix =
suffix =
user = exim
# router
route_append:
driver = domainlist
transport = smtp_appendfile
route_list = "some.domain batch.host"
This causes messages addressed to `some.domain' to be written in batched SMTP format to `/var/bsmtp/batch.host', with only a single copy of each message. Note that prefix and suffix must be explicitly changed from their defaults.
The `-bS' command line option causes Exim to accept one or more messages by reading SMTP on the standard input, but to generate no responses. If the caller is trusted, then the senders in the MAIL commands are believed; otherwise the sender is always the caller of Exim. Unqualified senders and receivers are not rejected (there seems little point) but instead just get qualified. If `sender_verify' is set, sender verification takes place only if `sender_verify_batch' is set (it defaults unset). Receiver verification and administrative rejection is not done, even if configured. HELO and EHLO act as RSET; VRFY, EXPN, ETRN, HELP, and DEBUG act as NOOP; QUIT quits.
If any error is detected while reading a message, including a missing `.' at the end, Exim gives up immediately. It writes details of the error to the standard output in a stylized way that the calling program should be able to make some use of automatically, for example:
554 Unexpected end of file Transaction started in line 10 Error detected in line 14
It writes a more verbose version, for human consumption, to the standard error file, for example:
An error was detected while processing a file of BSMTP input. The error message was: 501 '>' missing at end of address The SMTP transaction started in line 10. The error was detected in line 12. The SMTP command at fault was: rcpt to:<malformed@in.com.plete 1 previous message was successfully processed. The rest of the batch was abandoned.
The return code from Exim is zero only if there were no errors. It is 1 if some messages were accepted before an error was detected, and 2 if no messages were accepted.
Go to the first, previous, next, last section, table of contents.