A number of Exim configuration options contain lists of domains, hosts, email addresses, or local parts. For example, the hold_domains option contains a list of domains whose delivery is currently suspended. These lists are also used as data in ACL statements (see chapter 37).
Each item in one of these lists is a pattern to be matched against a domain, host, email address, or local part, respectively. In the sections below, the different types of pattern for each case are described, but first we cover some general facilities that apply to all four kinds of list.
Each list is expanded as a single string before it is used. If the expansion is forced to fail, Exim behaves as if the item it is testing (domain, host, address, or local part) is not in the list. Other expansion failures cause temporary errors.
If an item in a list is a regular expression, backslashes, dollars and possibly other special characters in the expression must be protected against misinterpretation by the string expander. The easiest way to do this is to use the \N expansion feature to indicate that the contents of the regular expression should not be expanded. For example, in an ACL you might have:
deny senders = \N^\d{8}\w@.*\.baddomain\.example$\N :
${lookup{$domain}lsearch{/badsenders/bydomain}}
The first item is a regular expression that is protected from expansion by \N, whereas the second uses the expansion to obtain a list of unwanted senders based on the receiving domain.
After expansion, the list is split up into separate items for matching. Normally, colon is used as the separator character, but this can be varied if necessary, as described in section 6.13.
Items in a list may be positive or negative. Negative items are indicated by a leading exclamation mark, which may be followed by optional white space. A list defines a set of items (domains, etc). When Exim processes one of these lists, it is trying to find out whether a domain, host, address, or local part (respectively) is in the set that is defined by the list. It works like this:
The list is scanned from left to right. If a positive item is matched, the subject that is being checked is in the set; if a negative item is matched, the subject is not in the set. If the end of the list is reached without the subject having matched any of the patterns, it is in the set if the last item was a negative one, but not if it was a positive one. For example, the list in
domainlist relay_domains = !a.b.c : *.b.c
matches any domain ending in .b.c except for a.b.c. Domains that match neither a.b.c nor **.b.c* do not match, because the last item in the list is positive. However, if the setting were
domainlist relay_domains = !a.b.c
then all domains other than a.b.c would match because the last item in the list is negative. In other words, a list that ends with a negative item behaves as if it had an extra item :* on the end.
If an item in a domain, host, address, or local part list is an absolute file name (beginning with a slash character), each line of the file is read and processed as if it were an independent item in the list, except that further file names are not allowed. Empty lines in the file are ignored, and the file may also contain comment lines:
For domain and host lists, if a # character appears anywhere in a line of the file, it and all following characters are ignored.
Because local parts may legitimately contain # characters, a comment in an address list or local part list file is recognized only if # is preceded by white space or the start of the line. For example:
not#comment@x.y.z # but this is a comment
Putting a file name in a list has the same effect as inserting each line of the file as an item in the list (blank lines and comments excepted). However, there is one important difference: the file is read each time the list is processed, so if its contents vary over time, Exim's behaviour changes.
If a file name is preceded by an exclamation mark, the sense of any match within the file is inverted. For example, if
hold_domains = !/etc/nohold-domains
and the file contains the lines
!a.b.c *.b.c
then a.b.c is in the set of domains defined by hold_domains, whereas any domain matching *.b.c is not.
As will be described in the sections which follow, lookups can be used in lists to provide indexed methods of checking list membership. There has been some confusion about the way lsearch lookups work in lists. Because an lsearch file contains plain text and is scanned sequentially, it is sometimes thought that it is allowed to contain wild cards and other kinds of non-constant pattern. This is not the case. The keys in an lsearch file are always fixed strings, just as for any other single-key lookup type.
If you want to use a file to contain wild-card patterns, just give the file name on its own, without a search type, as described in the previous section.
A list of domains, hosts, email addresses, or local parts can be given a name which is then used to refer to the list elsewhere in the configuration. This is particularly convenient if the same list is required in several different places. It also allows lists to be given meaningful names, which can improve the readability of the configuration. For example, it is conventional to define a domain list called local_domains for all the domains that are handled locally on a host, by a configuration line such as
domainlist local_domains = localhost:my.dom.example
Named lists are referenced by giving their name preceded by a plus sign, so, for example, a router that is intended to handle local domains would be configured with the line
domains = +local_domains
The first router in a configuration is often one that handles all domains except the local ones, using a configuration with a negated item like this:
dnslookup: driver = dnslookup domains = ! +local_domains transport = remote_smtp no_more
The four kinds of name list are created by configuration lines starting with the words domainlist, hostlist, addresslist, or localpartlist, respectively. Then there follows the name that you are defining, followed by an equals sign and the list itself. For example:
hostlist relay_hosts = 192.168.23.0/24 : my.friend.example addresslist bad_senders = cdb;/etc/badsenders
A named list may refer to other named lists:
domainlist dom1 = first.example : second.example domainlist dom2 = +dom1 : third.example domainlist dom3 = fourth.example : +dom2 : fifth.example
At first sight, these lists might seem to be no different from macros in the configuration file. However, macros are just textual substitutions. If you write
ALIST = host1 : host2 auth_advertise_hosts = !ALIST
it probably won't do what you want, because that is exactly the same as
auth_advertise_hosts = !host1 : host2
Notice that the second host name is not negated. However, if you use a host list, and write
hostlist alist = host1 : host2 auth_advertise_hosts = ! +alist
the negation applies to the whole list, and so that is equivalent to
auth_advertise_hosts = !host1 : !host2
Named lists may have a performance advantage. When Exim is routing an address or checking an incoming message, it caches the result of tests on named lists. So, if you have a setting such as
domains = +local_domains
on several of your routers, the actual test is done only for the first one. However, the caching works only if there are no expansions within the list itself or any sublists that it references. In other words, caching happens only for lists that are known to be the same each time they are referenced.
By default, there may be up to 16 named lists of each type. This limit can be extended by changing a compile-time variable. The use of domain and host lists is recommended for concepts such as local domains, relay domains, and relay hosts. The default configuration is set up like this.
Domain lists contain patterns that are to be matched against a mail domain. The following types of item may appear in domain lists:
If a pattern consists of a single @ character, it matches the local host name, as set by the primary_hostname option (or defaulted). This makes it possible to use the same configuration file on several different hosts that differ only in their names.
If a pattern consists of the string @[] it matches any local IP interface address, enclosed in square brackets, as in an email address that contains a domain literal. The use of domain literals is dying out in today's Internet.
If a pattern consists of the string @mx_any it matches any domain that has an MX record pointing to the local host, or to any host that is listed in hosts_treat_as_local. The items @mx_primary and @mx_secondary are similar, except that the first matches only when a primary MX target is the local host, and the second only when no primary MX target is the local host, but a secondary MX target is. ``Primary'' means an MX record with the lowest preference value - there may of course be more than one of them.
If a pattern starts with an asterisk, the remaining characters of the pattern are compared with the terminating characters of the domain. The use of ``*'' in domain lists differs from its use in partial matching lookups. In a domain list, the character following the asterisk need not be a dot, whereas partial matching works only in terms of dot-separated components. For example, a domain list item such as *key.ex matches donkey.ex as well as cipher.key.ex.
If a pattern starts with a circumflex character, it is treated as a regular expression, and matched against the domain using a regular expression matching function. The circumflex is treated as part of the regular expression. References to descriptions of the syntax of regular expressions are given in chapter 8.
Warning: Because domain lists are expanded before being processed, you must escape any backslash and dollar characters in the regular expression, or use the special \N sequence (see chapter 11) to specify that it is not to be expanded (unless you really do want to build a regular expression by expansion, of course).
If a pattern starts with the name of a single-key lookup type followed by a semicolon (for example, ``dbm;'' or ``lsearch;''), the remainder of the pattern must be a file name in a suitable format for the lookup type. For example, for ``cdb;'' it must be an absolute path:
domains = cdb;/etc/mail/local_domains.cdb
The appropriate type of lookup is done on the file using the domain name as the key. In most cases, the data that is looked up is not used; Exim is interested only in whether or not the key is present in the file. However, when a lookup is used for the domains option on a router, the data is preserved in the $domain_data variable and can be referred to in other options.
Any of the single-key lookup type names may be preceded by ``partial<n>-'', where the <n> is optional, for example,
domains = partial-dbm;/partial/domains
This causes partial matching logic to be invoked; a description of how this works is given in section 9.5.
Any of the single-key lookup types may be followed by an asterisk. This causes a default lookup for a key consisting of a single asterisk to be done if the original lookup fails. This is not a useful feature when using a domain list to select particular domains (because any domain would match), but it might have value if the result of the lookup is being used via the $domain_data expansion variable.
If the pattern starts with the name of a query-style lookup type followed by a semicolon (for example, ``nisplus;'' or ``ldap;''), the remainder of the pattern must be an appropriate query for the lookup type, as described in chapter 9. For example:
hold_domains = mysql;select domain from holdlist \ where domain = '$domain';
In most cases, the data that is looked up is not used (so for an SQL query, for example, it doesn't matter what field you select). Exim is interested only in whether or not the query succeeds. However, when a lookup is used for the domains option on a router, the data is preserved in the $domain_data variable and can be referred to in other options.
If none of the above cases apply, a caseless textual comparison is made between the pattern and the domain.
Here is an example which uses several different kinds of pattern:
domainlist funny_domains = \
@ : \
lib.unseen.edu : \
*.foundation.fict.example : \
\N^[1-2]\d{3}\.fict\.example$\N : \
partial-dbm;/opt/data/penguin/book : \
nis;domains.byname : \
nisplus;[name=$domain,status=local],domains.org_dir
There are obvious processing trade-offs among the various matching modes. Using an asterisk is faster than a regular expression, and listing a few names explicitly probably is too. The use of a file or database lookup is expensive, but may be the only option if hundreds of names are required. Because the patterns are tested in order, it makes sense to put the most commonly matched patterns earlier.
Host lists are used to control what remote hosts are allowed to do. For example, some hosts may be allowed to use the local host as a relay, and some may be permitted to use the SMTP ETRN command. Hosts can be identified in two different ways, by name or by IP address. In a host list, some types of pattern are matched to a host name, and some are matched to an IP address. You need to be particularly careful with this when single-key lookups are involved, to ensure that the right value is being used as the key.
If a host list item is the empty string, it matches only when no remote host is involved. This is the case when a message is being received from a local process using SMTP on the standard input, that is, when a TCP/IP connection is not used.
The special pattern ``*'' in a host list matches any host or no host. Neither the IP address nor the name is actually inspected.
If an IPv4 host calls an IPv6 host and the call is accepted on an IPv6 socket, the incoming address actually appears in the IPv6 host as ``::ffff:<v4address>''. When such an address is tested against a host list, it is converted into a traditional IPv4 address first. (Not all operating systems accept IPv4 calls on IPv6 sockets, as there have been some security concerns.)
The following types of pattern in a host list check the remote host by inspecting its IP address:
If the pattern is a plain domain name (not a regular expression, not starting with *), Exim calls gethostbyname() to find its associated IP address(es). This typically causes a forward DNS lookup of the name. The result is compared with the IP address of the subject host.
If the pattern is ``@'', the primary host name is substituted and used as a domain name, as just described.
If the pattern is an IP address, it is matched against the IP address of the subject host. IPv4 addresses are given in the normal ``dotted-quad'' notation. IPv6 addresses can be given in colon-separated format (when IPv6 support is compiled into Exim), but the colons have to be doubled so as not to be taken as item separators when the default list separator is used.
If the pattern is ``@[]'', it matches the IP address of any IP interface on the local host. For example, if the local host is an IPv4 host with one interface address 10.45.23.56, these two ACL statements have the same effect:
accept hosts = 127.0.0.1 : 10.45.23.56 accept hosts = @[]
If the pattern is an IP address followed by a slash and a mask length (for example 10.11.42.0/24), it is matched against the IP address of the subject host under the given mask. The mask uses CIDR notation; it specifies the number of address bits that must match, starting from the most significant end of the address. Thus, an entire network of hosts can be included (or excluded) by a single item. The above example matches all 256 addresses in the 10.11.42.0 network. Here is another example which shows an IPv4 and an IPv6 network:
recipient_unqualified_hosts = 192.168.0.0/16: \ 3ffe::ffff::836f::::/48
The doubling of list separator characters applies only when these items appear inline in a host list. It is not required when indirecting via a file. For example,
recipient_unqualified_hosts = /opt/exim/unqualnets
could make use of a file containing
172.16.0.0/12 3ffe:ffff:836f::/48
to have exactly the same effect as the previous example. When listing small numbers of IPv6 addresses inline, it is usually more convenient to use the facility for changing separator characters. This list contains the same two networks:
recipient_unqualified_hosts = <; 172.16.0.0/12; \ 3ffe:ffff:836f::/48
The separator is changed to semicolon by the leading ``<;'' at the start of the list.
When a host is to be identified by a lookup of its complete IP address, the pattern can take this form:
net-<search-type>;<search-data>
For example:
hosts_lookup = net-cdb;/hosts-by-ip.db
For a single-key lookup type, the text form of the IP address of the subject host is used as the lookup key. IPv6 addresses are converted to an unabbreviated form, using lower case letters, with full stops (periods) as separators because colon is the key terminator in lsearch files.
For a query-style lookup, the variable $sender_host_address can be used in the query. For example:
hosts_lookup = net-postgresql;\ select ip from hostlist where ip='$sender_host_address'
In both cases, the data returned by the lookup is not used.
Lookups can also be performed using masked IP addresses, using patterns of this form:
net<number>-<search-type>;<search-data>
For example:
net24-dbm;/networks.db
If the search type is a single-key lookup, the IP address of the subject host is masked using <number> as the mask length. A textual string is constructed from the masked value, followed by the mask, and this is used as the lookup key. For example, if the host's IP address is 192.168.34.6, the key that is looked up for the above example is ``192.168.34.0/24''. IPv6 addresses are converted to a text value using lower case letters and full stops (periods) as separators instead of the more usual colon, because colon is the key terminator in lsearch files. Full, unabbreviated IPv6 addresses are always used.
Warning: Specifing net32- (for an IPv4 address) or net128- (for an IPv6 address) is not the same as specifing just net- without a number. In the former case the key strings include the mask value, whereas in the latter case the IP address is used on its own.
If the search type is a query-style lookup, <number> is not relevant, and this type of pattern is no different to the previous kind, that is, net- without a number. If you want to use masked IP addresses in database queries, you can use the mask expansion operator.
The remaining types of pattern that can appear in host lists require Exim to know the name of the remote host. They are all wildcarded names of different kinds. (If a complete name is given without any wildcarding, it is used to find an IP address to match against, as described in the previous section.)
If the remote host name is not already known when Exim encounters one of these patterns, gethostbyaddr() is used to find it from the IP address. This typically causes a reverse DNS lookup to occur. Although many sites on the Internet are conscientious about maintaining reverse DNS data for their hosts, there are also many that do not do this. Consequently, a name cannot always be found, and this may lead to unwanted effects.
If the DNS lookup fails, that is, if there is no reverse DNS entry for the IP address, Exim behaves as if the host does not match the list. This may not always be what you want to happen. To change Exim's behaviour, the special item ``+include_unknown'' may appear in the list (at top level - it is not recognized in an indirected file). If any subsequent items require a host name, but no name can be found, Exim behaves as if the host does match the list. For example,
host_reject_connection = +include_unknown:*.enemy.ex
rejects connections from any host whose name matches *.enemy.ex, and also any hosts whose name it cannot find.
Warning: Take care when configuring host lists with wildcarded name patterns. Consider what will happen if a name cannot be found.
As a result of aliasing, hosts may have more than one name. When processing any of the following types of pattern, all the host's names are checked:
If a pattern starts with ``*'' the remainder of the item must match the end of the host name. For example, *.b.c matches all hosts whose names end in .b.c. This special simple form is provided because this is a very common requirement. Other kinds of wildcarding require the use of a regular expression.
If the item starts with ``^'' it is taken to be a regular expression which is matched against the host name. For example,
^(a|b)\.c\.d$
is a regular expression which matches either of the two hosts a.c.d or b.c.d. When a regular expression is used in a host list, you must take care that backslash and dollar characters are not misinterpreted as part of the string expansion. The simplest way to do this is to use \N to mark that part of the string as non-expandable. For example:
sender_unqualified_hosts = \N^(a|b)\.c\.d$\N : ....
If a pattern is of the form
<search-type>;<filename or query>
for example
dbm;/host/accept/list
the host name is looked up using the search type and file name or query (as appropriate). If the lookup succeeds, the host matches the item. The actual data that is looked up is not used.
Warning 1: When using this kind of pattern with a single-key lookup, you must have host names as keys in the file, not IP addresses. If you want to do lookups based on IP addresses, you must precede the search type with ``net-'' (see the previous section). There is, however, no reason why you could not use two items in the same list, one doing an address lookup and one doing a name lookup, both using the same file.
Warning 2: For a query-style lookup, what to lookup is given explicitly in the query, but Exim always ensures that the host name is available before running the query for this type of pattern. If you are not using the host name in your query, you should be using the net- form of search described in the previous section, so that Exim does not look up the host name unnecessarily.
If you have name lookups or wildcarded host names and IP addresses in the same host list, you should normally put the IP addresses first. For example, in an ACL you could have:
accept hosts = 10.9.8.7 : *.friend.example
The reason for this lies in the left-to-right way that Exim processes lists. It can test IP addresses without doing any DNS lookups, but when it reaches an item that requires a host name, it fails if it cannot find a host name to compare with the pattern. If the above list is given in the opposite order, the accept statement fails for a host whose name cannot be found, even if its IP address is 10.9.8.7.
If you really do want to do the name check first, and still recognize the IP address, you can rewrite the ACL like this:
accept hosts = *.friend.example accept hosts = 10.9.8.7
If the first accept fails, Exim goes on to try the second one. See chapter 37 for details of ACLs.
Address lists contain patterns which are matched against mail addresses. There is one special case to be considered: the sender address of a bounce message is always empty. You can test for this by providing an empty item in an address list. For example, you can set up a router to process bounce messages only by this option setting:
senders = :
The presence of the colon creates an empty item. If you do not provide any data, the list is empty and matches nothing. The empty sender can also be detected by a regular expression that matches an empty string.
The following kinds of pattern are supported in address lists:
If (after expansion) a pattern starts with ``^'', a regular expression match is done against the complete address, with the pattern as the regular expression. You must take care that backslash and dollar characters are not misinterpreted as part of the string expansion. The simplest way to do this is to use \N to mark that part of the string as non-expandable. For example:
deny senders = \N^\d{8}.+@spamhaus.example$\N : ...
The \N sequences are removed by the expansion, so the item does start with ``^'' by the time it is being interpreted as an address pattern.
If a pattern is of the form ``@@<lookup-item>'' (for example, ``@@lsearch;/some/file''), the address that is being checked is split into a local part and a domain. The domain is looked up in the file. If it is not found, there is no match. If it is found, the data that is looked up from the file is treated as a colon-separated list of local part patterns, each of which is matched against the subject local part in turn.
The lookup may be a partial one, and/or one involving a search for a default keyed by ``*'' (see section 9.4). The local part patterns that are looked up can be regular expressions or begin with ``*'', or even be further lookups. They may also be independently negated. For example, with
deny senders = @@dbm;/etc/reject-by-domain
the data from which the DBM file is built could contain lines like
baddomain.com: !postmaster : *
to reject all senders except postmaster from that domain. If a local part that actually begins with an exclamation mark is required, it has to be specified using a regular expression. In lsearch files, an entry may be split over several lines by indenting the second and subsequent lines, but the separating colon must still be included at line breaks. White space surrounding the colons is ignored. For example:
aol.com: spammer1 : spammer2 : ^[0-9]+$ : spammer3 : spammer4
As in all colon-separated lists in Exim, a colon can be included in an item by doubling.
If the last item in the list starts with a right angle-bracket, the remainder of the item is taken as a new key to look up in order to obtain a continuation list of local parts. The new key can be any sequence of characters. Thus one might have entries like
aol.com: spammer1 : spammer 2 : >*
xyz.com: spammer3 : >*
*: ^\d{8}$
in a file that was searched with @@dbm*, to specify a match for 8-digit local parts for all domains, in addition to the specific local parts listed for each domain. Of course, using this feature costs another lookup each time a chain is followed, but the effort needed to maintain the data is reduced. It is possible to construct loops using this facility, and in order to catch them, the chains may be no more than fifty items long.
Complete addresses can be looked up by using a pattern that consists of a lookup type, a semicolon, and the data for the lookup. For example:
deny senders = cdb;/etc/blocked.senders : \
mysql;select address from blocked where \
address='${quote_mysql:$sender_address}'
For a single-key lookup type, Exim uses the complete address as the key. Partial matching (section 9.5) cannot be used, and is ignored if specified, with an entry being written to the panic log. However, you can configure a lookup default, as described in section 9.4.
If a pattern contains an @ character, but is not a regular expression or a lookup as described above, the local part of the subject address is compared with the local part of the pattern, which may start with an asterisk. If the local parts match, the domain is checked in exactly the same way as for a pattern in a domain list. For example, the domain can be wildcarded, refer to a named list, be be or a lookup:
deny senders = *@*.spamming.site:\ *@+hostile_domains:\ bozo@partial-lsearch;/list/of/dodgy/sites
If a local part that begins with an exclamation mark is required, it has to be specified using a regular expression, because otherwise the exclamation mark is treated as a sign of negation.
If a pattern is not one of the above syntax forms, that is, if a pattern which is not a regular expression or a lookup does not contain an @ character, it is matched against the domain part of the subject address. The only two formats that are recognized this way are a literal domain, or a domain pattern that starts with *. In both these cases, the effect is the same as if *@ precedes the pattern.
Note: there is an important difference between the address list items in these two examples:
senders = +my_list senders = *@+my_list
In the first one, my_list is a named address list, whereas in the second example it is a named domain list.
Domains in email addresses are always handled caselessly, but for local parts case may be significant on some systems (see caseful_local_part for how Exim deals with this when routing addresses). However, RFC 2505 (Anti-Spam Recommendations for SMTP MTAs) suggests that matching of addresses to blocking lists should be done in a case-independent manner. Since most address lists in Exim are used for this kind of control, Exim attempts to do this by default.
The domain portion of an address is always lowercased before matching it to an address list. The local part is lowercased by default, and any string comparisons that take place are done caselessly. This means that the data in the address list itself, in files included as plain file names, and in any file that is looked up using the ``@@'' mechanism, can be in any case. However, the keys in files that are looked up by a search type other than lsearch (which works caselessly) must be in lower case, because these lookups are not case-independent.
To allow for the possibility of caseful address list matching, if an item in an address list is the string ``+caseful'', the original case of the local part is restored for any comparisons that follow, and string comparisons are no longer case-independent. This does not affect the domain, which remains in lower case. However, although independent matches on the domain alone are still performed caselessly, regular expressions that match against an entire address become case-sensitive after ``+caseful'' has been seen.
Case-sensitivity in local part lists is handled in the same way as for address lists, as just described. The ``+caseful'' item can be used if required. In a setting of the local_parts option in a router with caseful_local_part set false, the subject is lowercased and the matching is initially case-insensitive. In this case, ``+caseful'' will restore case-sensitive matching in the local part list, but not elsewhere in the router. If caseful_local_part is set true in a router, matching in the local_parts option is case-sensitive from the start.
If a local part list is indirected to a file (see section 10.3), comments are handled in the same way as address lists - they are recognized only if the # is preceded by white space or the start of the line. Otherwise, local part lists are matched in the same way as domain lists, except that the special items that refer to the local host (@, @[], @mx_any, @mx_primary, and @mx_secondary) are not recognized. Refer to section 10.6 for details of the other available item types.