The smtp transport delivers messages over TCP/IP connections using the SMTP or LMTP protocol. The list of hosts to try can either be taken from the address that is being processed (having been set up by the router), or specified explicitly for the transport. Timeout and retry processing (see chapter 32) is applied to each IP address independently.
The sending of multiple messages over a single TCP/IP connection can arise in two ways:
If a message contains more than max_rcpt (see below) addresses that are routed to the same host, more than one copy of the message has to be sent to that host. In this situation, multiple copies may be sent in a single run of the smtp transport over a single TCP/IP connection. (What Exim actually does when it has too many addresses to send in one message also depends on the value of the global remote_max_parallel option. Details are given in section 43.1.)
When a message has been successfully delivered over a TCP/IP connection, Exim looks in its hints database to see if there are any other messages awaiting a connection to the same host. If there are, a new delivery process is started for one of them, and the current TCP/IP connection is passed on to it. The new process may in turn send multiple copies and possibly create yet another process.
For each copy sent over the same TCP/IP connection, a sequence counter is incremented, and if it ever gets to the value of connection_max_messages, no further messages are sent over that connection.
At the start of a run of the smtp transport, the values of $host and $host_address are the name and IP address of the first host on the host list passed by the router. However, when the transport is about to connect to a specific host, and while it is connected to that host, $host and $host_address are set to the values for that host. These are the values that are in force when the helo_data, hosts_try_auth, interface, serialize_hosts, and the various TLS options are expanded.
The private options of the smtp transport are as follows:
When a host specified in hosts or fallback_hosts (see below) turns out to be the local host, or is listed in hosts_treat_as_local, delivery is deferred by default. However, if allow_localhost is set, Exim goes on to do the delivery anyway. This should be used only in special cases when the configuration ensures that no looping will result (for example, a differently configured Exim is listening on the port to which the message is sent).
When Exim has authenticated as a client, this option sets a value for the AUTH= item on outgoing MAIL commands, overriding any existing authenticated sender value. If the string expansion is forced to fail, the option is ignored. Other expansion failures cause delivery to be deferred. If the result of expansion is an empty string, that is also ignored.
If the SMTP session is not authenticated, the expansion of authenticated_sender still happens (and can cause the delivery to be deferred if it fails), but no AUTH= item is added to MAIL commands.
This option allows you to use the smtp transport in LMTP mode to deliver mail to Cyrus IMAP and provide the proper local part as the “authenticated sender”, via a setting such as:
authenticated_sender = $local_part
This removes the need for IMAP subfolders to be assigned special ACLs to allow direct delivery to those subfolders.
Because of expected uses such as that just described for Cyrus (when no domain is involved), there is no checking on the syntax of the provided value.
This sets a timeout for receiving a response to an SMTP command that has been sent out. It is also used when waiting for the initial banner line from the remote host. Its value must not be zero.
This sets a timeout for the connect() function, which sets up a TCP/IP call to a remote host. A setting of zero allows the system timeout (typically several minutes) to act. To have any effect, the value of this option must be less than the system timeout. However, it has been observed that on some systems there is no system timeout, which is why the default value for this option is 5 minutes, a value recommended by RFC 1123.
This controls the maximum number of separate message deliveries that are sent over a single TCP/IP connection. If the value is zero, there is no limit. For testing purposes, this value can be overridden by the -oB command line option.
This sets a timeout for the transmission of each block in the data portion of the message. As a result, the overall timeout for a message depends on the size of the message. Its value must not be zero. See also final_timeout.
This option controls what happens when all remote IP addresses for a given domain have been inaccessible for so long that they have passed their retry cutoff times.
In the default state, if the next retry time has not been reached for any of them, the address is bounced without trying any deliveries. In other words, Exim delays retrying an IP address after the final cutoff time until a new retry time is reached, and can therefore bounce an address without ever trying a delivery, when machines have been down for a long time. Some people are unhappy at this prospect, so...
If delay_after_cutoff is set false, Exim behaves differently. If all IP addresses are past their final cutoff time, Exim tries to deliver to those IP addresses that have not been tried since the message arrived. If there are none, of if they all fail, the address is bounced. In other words, it does not delay when a new message arrives, but immediately tries those expired IP addresses that haven't been tried since the message arrived. If there is a continuous stream of messages for the dead hosts, unsetting delay_after_cutoff means that there will be many more attempts to deliver to them.
If the hosts or fallback_hosts option is being used, and the gethostbyname option is false, the RES_DEFNAMES resolver option is set. See the qualify_single option in chapter 17 for more details.
If the hosts or fallback_hosts option is being used, and the gethostbyname option is false, the RES_DNSRCH resolver option is set. See the search_parents option in chapter 17 for more details.
String expansion is not applied to this option. The argument must be a colon-separated list of host names or IP addresses. Fallback hosts can also be specified on routers, which associate them with the addresses they process. As for the hosts option without hosts_override, fallback_hosts specified on the transport is used only if the address does not have its own associated fallback host list. Unlike hosts, a setting of fallback_hosts on an address is not overridden by hosts_override. However, hosts_randomize does apply to fallback host lists.
If Exim is unable to deliver to any of the hosts for a particular address, and the errors are not permanent rejections, the address is put on a separate transport queue with its host list replaced by the fallback hosts, unless the address was routed via MX records and the current host was in the original MX list. In that situation, the fallback host list is not used.
Once normal deliveries are complete, the fallback queue is delivered by re-running the same transports with the new host lists. If several failing addresses have the same fallback hosts (and max_rcpt permits it), a single copy of the message is sent.
The resolution of the host names on the fallback list is controlled by the gethostbyname option, as for the hosts option. Fallback hosts apply both to cases when the host list comes with the address and when it is taken from hosts. This option provides a “use a smart host only if delivery fails” facility.
This is the timeout that applies while waiting for the response to the final line containing just “.” that terminates a message. Its value must not be zero.
If this option is true when the hosts and/or fallback_hosts options are being used, names are looked up using gethostbyname() (or getipnodebyname() when available) instead of using the DNS. Of course, that function may in fact use the DNS, but it may also consult other sources of information such as /etc/hosts.
The value of this option is expanded, and used as the argument for the EHLO or HELO command that starts the outgoing SMTP session.
Hosts are associated with an address by a router such as dnslookup, which finds the hosts by looking up the address domain in the DNS. However, addresses can be passed to the smtp transport by any router, and not all of them can provide an associated host list. The hosts option specifies a list of hosts which are used if the address being processed does not have any hosts associated with it. The hosts specified by hosts are also used, whether or not the address has its own hosts, if hosts_override is set.
The string is first expanded, before being interpreted as a colon-separated list of host names or IP addresses. If the expansion fails, delivery is deferred. Unless the failure was caused by the inability to complete a lookup, the error is logged to the panic log as well as the main log. Host names are looked up either by searching directly for address records in the DNS or by calling gethostbyname() (or getipnodebyname() when available), depending on the setting of the gethostbyname option. When Exim is compiled with IPv6 support, if a host that is looked up in the DNS has both IPv4 and IPv6 addresses, both types of address are used.
During delivery, the hosts are tried in order, subject to their retry status,
unless hosts_randomize is set.
This option is for use with broken hosts that announce ESMTP facilities (for example, PIPELINING) and then fail to implement them properly. When a host matches hosts_avoid_esmtp, Exim sends HELO rather than EHLO at the start of the SMTP session. This means that it cannot use any of the ESMTP facilities such as AUTH, PIPELINING, SIZE, and STARTTLS.
Exim will not try to start a TLS session when delivering to any host that matches this list. See chapter 37 for details of TLS.
This option limits the number of IP addresses that are tried for any one delivery, in cases where attempts to connect suffer host errors. The limit does not apply, for example, to cases where temporary address errors (4xx responses to RCPT commands) are received. Section 30.4 describes in detail how the value of this option is used.
For any host that matches this list, a connection on which a TLS session has been started will not be passed to a new delivery process for sending another message on the same connection. See section 37.5 for an explanation of when this might be needed.
If this option is set and the hosts option is also set, any hosts that are attached to the address are ignored, and instead the hosts specified by the hosts option are always used. This option does not apply to fallback_hosts.
If this option is set, and either the list of hosts is taken from the hosts or the fallback_hosts option, or the hosts supplied by the router were not obtained from MX records (this includes fallback hosts from the router), and were not randomizied by the router, the order of trying the hosts is randomized each time the transport runs. Randomizing the order of a host list can be used to do crude load sharing.
When hosts_randomize is true, a host list may be split into groups whose order is separately randomized. This makes it possible to set up MX-like behaviour. The boundaries between groups are indicated by an item that is just + in the host list. For example:
hosts = host1:host2:host3:+:host4:host5
The order of the first three hosts and the order of the last two hosts is randomized for each use, but the first three always end up before the last two. If hosts_randomize is not set, a + item in the list is ignored.
This option provides a list of servers for which authentication must succeed before Exim will try to transfer a message. If authentication fails for servers which are not in this list, Exim tries to send unauthenticated. If authentication fails for one of these servers, delivery is deferred. This temporary error is detectable in the retry rules, so it can be turned into a hard failure if required. See also hosts_try_auth, and chapter 33 for details of authentication.
Exim will insist on using a TLS session when delivering to any host that matches this list. See chapter 37 for details of TLS. Note: This option affects outgoing mail only. To insist on TLS for incoming messages, use an appropriate ACL.
This option provides a list of servers to which, provided they announce authentication support, Exim will attempt to authenticate as a client when it connects. If authentication fails, Exim will try to transfer the message unauthenticated. See also hosts_require_auth, and chapter 33 for details of authentication.
This option specifies which interface to bind to when making an outgoing SMTP call. The variables $host and $host_address refer to the host to which a connection is about to be made during the expansion of the string. Forced expansion failure, or an empty string result causes the option to be ignored. Otherwise, after expansion, the string must be a colon-separated list of IP addresses, for example:
interface = <; 192.168.123.123 ; 3ffe:ffff:836f::fe86:a061
The first interface of the correct type (IPv4 or IPv6) is used for the outgoing connection. If none of them are the correct type, the option is ignored. If interface is not set, or is ignored, the system's IP functions choose which interface to use if the host has more than one.
This option controls the setting of SO_KEEPALIVE on outgoing TCP/IP socket connections. When set, it causes the kernel to probe idle connections periodically, by sending packets with “old” sequence numbers. The other end of the connection should send a acknowledgement if the connection is still okay or a reset if the connection has been aborted. The reason for doing this is that it has the beneficial effect of freeing up certain types of connection that can get stuck when the remote host is disconnected without tidying up the TCP/IP call properly. The keepalive mechanism takes several hours to detect unreachable hosts.
This option limits the number of RCPT commands that are sent in a single SMTP message transaction. Each set of addresses is treated independently, and so can cause parallel connections to the same host if remote_max_parallel permits this.
When this option is set, the smtp transport can handle a number of addresses
containing a mixture of different domains provided they all resolve to the same
list of hosts. Turning the option off restricts the transport to handling only
one domain at a time. This is useful if you want to use $domain in an
expansion for the transport, because it is set only when there is a single
domain involved in a remote delivery.
This option specifies the TCP/IP port on the server to which Exim connects. If it begins with a digit it is taken as a port number; otherwise it is looked up using getservbyname(). The default value is normally “smtp”, but if protocol is set to “lmtp”, the default is “lmtp”. If the expansion fails, or if a port number cannot be found, delivery is deferred.
If this option is set to “lmtp” instead of “smtp”, the default value for the port option changes to “lmtp”, and the transport operates the LMTP protocol (RFC 2033) instead of SMTP. This protocol is sometimes used for local deliveries into closed message stores. Exim also has support for running LMTP over a pipe to a local process – see chapter 28.
Exim normally includes both the host name and the IP address in the key it constructs for indexing retry data after a temporary delivery failure. This means that when one of several IP addresses for a host is failing, it gets tried periodically (controlled by the retry rules), but use of the other IP addresses is not affected.
However, in some dialup environments hosts are assigned a different IP address each time they connect. In this situation the use of the IP address as part of the retry key leads to undesirable behaviour. Setting this option false causes Exim to use only the host name. This should normally be done on a separate instance of the smtp transport, set up specially to handle the dialup hosts.
Because Exim operates in a distributed manner, if several messages for the same host arrive at around the same time, more than one simultaneous connection to the remote host can occur. This is not usually a problem except when there is a slow link between the hosts. In that situation it may be helpful to restrict Exim to one connection at a time. This can be done by setting serialize_hosts to match the relevant hosts.
Exim implements serialization by means of a hints database in which a record is written whenever a process connects to one of the restricted hosts. The record is deleted when the connection is completed. Obviously there is scope for records to get left lying around if there is a system or program crash. To guard against this, Exim ignores any records that are more than six hours old.
If you set up this kind of serialization, you should also arrange to delete the relevant hints database whenever your system reboots. The names of the files start with misc and they are kept in the spool/db directory. There may be one or two files, depending on the type of DBM in use. The same files are used for ETRN serialization.
If a remote SMTP server indicates that it supports the SIZE option of the MAIL command, Exim uses this to pass over the message size at the start of an SMTP transaction. It adds the value of size_addition to the value it sends, to allow for headers and other text that may be added during delivery by configuration options or in a transport filter. It may be necessary to increase this if a lot of text is added to messages.
Alternatively, if the value of size_addition is set negative, it disables the use of the SIZE option altogether.
The value of this option must be the absolute path to a file which contains the client's certificate, for use when sending a message over an encrypted connection. The values of $host and $host_address are set to the name and address of the server during the expansion. See chapter 37 for details of TLS.
Note: This option must be set if you want Exim to use TLS when sending messages as a client. The global option of the same name specifies the certificate for Exim as a server; it is not automatically assumed that the same certificate should be used when Exim is operating as a client.
The value of this option must be the absolute path to a file which contains the client's private key, for use when sending a message over an encrypted connection. The values of $host and $host_address are set to the name and address of the server during the expansion. If this option is unset, the private key is assumed to be in the same file as the certificate. See chapter 37 for details of TLS.
The value of this option must be a list of permitted cipher suites, for use when setting up an encrypted connection. The values of $host and $host_address are set to the name and address of the server during the expansion. See chapter 37 for details of TLS; note that this option is used in different ways by OpenSSL and GnuTLS (see section 37.1).
When the server host is not in hosts_require_tls, and there is a problem in setting up a TLS session, this option determines whether or not Exim should try to deliver the message unencrypted. If it is set false, delivery to the current host is deferred; if there are other hosts, they are tried. If this option is set true, Exim attempts to deliver unencrypted after a 4xx response to STARTTLS. Also, if STARTTLS is accepted, but the subsequent TLS negotiation fails, Exim closes the current connection (because it is in an unknown state), opens a new one to the same host, and then tries the delivery in clear.
The value of this option must be the absolute path to a file containing permitted server certificates, for use when setting up an encrypted connection. Alternatively, if you are using OpenSSL, you can set tls_verify_certificates to the name of a directory containing certificate files. This does not work with GnuTLS; the option must be set to the name of a single file if you are using GnuTLS. The values of $host and $host_address are set to the name and address of the server during the expansion of this option. See chapter 37 for details of TLS.
The hosts_max_try option limits the number of hosts that are tried for a single delivery. However, despite the term “host” in its name, the option actually applies to each IP address independently. In other words, a multihomed host is treated as several independent hosts, just as it is for retrying.
Many of the larger ISPs have multiple MX records which often point to multihomed hosts. As a result, a list of a dozen or more IP addresses may be created as a result of routing one of these domains.
Trying every single IP address on such a long list does not seem sensible; if several at the top of the list fail, it is reasonable to assume there is some problem that is likely to affect all of them. Roughly speaking, the value of hosts_max_try is the maximum number that are tried before deferring the delivery. However, the logic cannot be quite that simple.
Firstly, IP addresses that are skipped because their retry times have not arrived do not count, and in addition, addresses that are past their retry limits are also not counted, even when they are tried. This means that when some IP addresses are past their retry limits, more than the value of hosts_max_retry may be tried. The reason for this behaviour is to ensure that all IP addresses are considered before timing out an email address.
Secondly, when the hosts_max_try limit is reached, Exim looks down the host list to see if there is a subsequent host with a different (higher valued) MX. If there is, that host is used next, and the current IP address is used but not counted. This behaviour helps in the case of a domain with a retry rule that hardly ever delays any hosts, as is now explained:
Consider the case of a long list of hosts with one MX value, and a few with a higher MX value. If hosts_max_try is small (the default is 5) only a few hosts at the top of the list are tried at first. With the default retry rule, which specifies increasing retry times, the higher MX hosts are eventually tried when those at the top of the list are skipped because they have not reached their retry times.
However, it is common practice to put a fixed short retry time on domains for large ISPs, on the grounds that their servers are rarely down for very long. Unfortunately, these are exactly the domains that tend to resolve to long lists of hosts. The short retry time means that the lowest MX hosts are tried every time. The attempts may be in a different order because of random sorting, but without the special MX check mentioned about, the higher MX hosts would never be tried at all because the lower MX hosts are never all past their retry times.
With the special check, Exim tries least one address from each MX value, even if the hosts_max_try limit has already been reached.