Address rewriting

Some people believe that configured address rewriting is a Mortal Sin. Others believe that life is not possible without it. Exim provides the facility; you do not have to use it. There are two cases that are commonly encountered:

• The company `hitch.fict.book' has a number of machines that exchange mail with each other behind a firewall, but only a single gateway to the outer world. The gateway rewrites `*.hitch.fict.book' as `hitch.fict.book'.
• A machine rewrites the local parts of its own users so that, for example, `fp42@hitch.fict.book' becomes `Ford.Prefect@hitch.fict.book'.

In general, rewriting addresses from one's own system or domain has some legitimacy. Rewriting other addresses should be done only with great care and in special circumstances.

Address rewriting can be applied both to envelope addresses and addresses in header lines. Exim's rewriting rules specify to which addresses they apply.

• Rewriting rules for envelope addresses are also applied to new addresses that are generated by aliasing or forwarding operations, unless `no_rewrite' is set on the relevant directors.
• Rewriting of headers happens when a message is received, which is why it is possible to rewrite `Bcc:' headers (Exim removes `Bcc:' headers only when the `-t' option is used). It does not apply to headers that are added by the generic driver option `add_headers'.

Exim's rewriting configuration appears as the sixth part of the runtime configuration file. It can be tested by the `-brw' command line option. This takes an address (which can be a full RFC 822 address) as its argument. The output is a list of how the address would be transformed by the rewriting rules for each of the different places it might appear, that is, for each different header and for the envelope sender and recipient fields. For example,

exim -brw ph10@exim.work.shop

might produce the output

  sender: Philip.Hazel@exim.work.shop
    from: Philip.Hazel@exim.work.shop
      to: ph10@exim.work.shop
      cc: ph10@exim.work.shop
     bcc: ph10@exim.work.shop
reply-to: Philip.Hazel@exim.work.shop
env-from: Philip.Hazel@exim.work.shop
  env-to: ph10@exim.work.shop

which shows that rewriting has been set up for that address when used in any of the source fields, but not when it appears as a recipient address.

Rewriting rules

The rewriting configuration consists of lines of rewriting rules in the form

<source pattern>  <replacement>  <flags>

The flags are single characters which may appear in any order. Spaces and tabs between them are ignored.

The formats of source patterns and replacement strings are described below. Each is terminated by white space. If a replacement string contains spaces, which can happen for certain forms of expansion expression, it must be enclosed in double quotes, and the normal quoting conventions apply inside them.

Long rules (whether using quotes or not) can be split over several lines by terminating all but the last with a backslash character. Leading white space on continuation lines is ignored. Consequently, a continuation backslash should never appear immediately after the source pattern or replacement string, but instead should follow its terminating white space.

For each address that could potentially be rewritten, the rules are scanned in order, and replacements from earlier rules can themselves be replaced as a result of later rules (but see the `q' and `R' flags).

The order in which header and envelope addresses are rewritten is undefined, may change between releases, and must not be relied on. For example, the replacement string for a header rewrite must not assume that the message's envelope sender address has (or has not) already been rewritten.

`$local_part' and `$domain' can be used in the replacement string to refer the address that is being rewritten. Note that complete lookup-driven rewriting can be done by a line of the form

*@*   ${lookup ...

where the lookup key is derived from `$1' and `$2' or `$local_part' and `$domain'.

Rewriting patterns

The source pattern can be in one of the following forms. Note that it is not enclosed in quotes, and there is no special processing of any characters. In particular, if it is a regular expression, backslash characters should not be doubled.

• An address containing a local part and a domain, either of which may start with an asterisk, implying independent wildcard matching, for example

  
  *@orchestra-land.fict.book
  

  If the domain is specified as a single @ character, it matches the primary host name. After matching, the numerical variables refer to the character strings matched by asterisks, with `$1' associated with the first asterisk, while `$0' refers to the entire address. For example, if the pattern

  
  *queen@*.fict.book
  

  is matched against the address `hearts-queen@wonderland.fict.book' then

  
  $0 = `hearts-queen@wonderland.fict.book'
  $1 = `hearts-'
  $2 = `wonderland'
  

  Note that if the local part does not start with an asterisk, but the domain does, then it is `$1' that contains the wild part of the domain.
• A local part, possibly starting with an asterisk, and a lookup item (as in a domain list), for example

  
  root@lsearch;/special/domains
  

  If there is an asterisk in the local part, the value of the wild part is placed in the first numerical variable. If the lookup is a partial one, the wild part of the domain is placed in the next numerical variable, and the fixed part of the domain is placed in the succeeding variable. Thus, for example, if the address `foo@bar.baz.com' is processed by a rewriting rule of the form

  
  *@partial-dbm;/some/dbm/file    <replacement string>
  

  and the key in the file that matches the domain is `*.baz.com', then

  
  $1 = `foo'
  $2 = `bar'
  $3 = `baz.com'
  

  If the address `foo@baz.com' is looked up, this matches the same wildcard file entry, and in this case `$2' is set to the empty string, but `$3' is still set to `baz.com'. If a non-wild key is matched in a partial lookup, then again `$2' is set to the empty string and `$3' is set to the whole domain. For non-partial lookups, no numerical variables are set.
• A local part, possibly starting with an asterisk, and a regular expression (as in a domain list), for example

  
  *.queen@^(wonderland|lookingglass).fict.book$
  

  If there is an asterisk in the local part, the value of the wild part is placed in the first numerical variable. Any substrings captured by the regular expression are placed in numerical variables starting at `$1' if there is no asterisk in the local part, or at `$2' if there is.
• A lookup without a local part, for example

  
  partial-dbm;/rewrite/database
  

  This works as for an address-list configuration item -- the domain is first looked up, possibly partially, and if that fails, the whole address is then looked up (not partially). When a partial lookup succeeds, the numerical variable `$1' contains the wild part of the domain, and `$2' contains the fixed part. The `@@' form of address-list lookup can also be used.
• A single regular expression. This is matched against the entire address, with the domain part lower-cased. After matching, the numerical variables refer to the bracketed `capturing' sub-expressions, with `$0' referring to the entire address. For example, if the pattern

  
  ^(red|white).king@(wonderland|lookingglass).fict.book$
  

  is matched against the address `red.king@lookingglass.fict.book' then

  
  $0 = `red.king@lookingglass.fict.book'
  $1 = `red'
  $2 = `lookingglass'
  

  Note that because the pattern part of a rewriting rule is terminated by white space, no white space may be present in the regular expression.

Rewriting replacements

If the replacement string for a rule is a single asterisk, then addresses that match the pattern and flags are not rewritten, and no subsequent rewriting rules are scanned. For example,

hatta@lookingglass.fict.book  *  f

specifies that `hatta@lookingglass.fict.book' is never to be rewritten in `From:' headers.

Otherwise, the replacement string is expanded and must yield a fully qualified address. Within the expansion, the variables `$local_part' and `$domain' refer to the address that is being rewritten. Any letters they contain retain their original case -- they are not lower cased. The numerical variables are set up according to the type of pattern that matched the address, as described above. If the expansion is forced to fail by the presence of `fail' in a conditional or lookup item, rewriting by the current rule is abandoned. Any other expansion failure causes the entire rewriting operation to be abandoned, and an entry written to the panic log.

Flags specifying which headers and envelope addresses to rewrite

If none of the following flag letters, nor the `S' flag (see section "The SMTP-time rewriting flag") are present, the rewriting rule applies to all headers and to both the sender and recipient fields of the envelope. Otherwise, the rewriting rule is skipped unless the relevant addresses are being processed.

E       rewrite all envelope fields
F       rewrite the envelope From field
T       rewrite the envelope To field
b       rewrite the `Bcc:' header
c       rewrite the `Cc:' header
f       rewrite the `From:' header
h       rewrite all headers
r       rewrite the `Reply-to:' header
s       rewrite the `Sender:' header
t       rewrite the `To:' header

You should be particularly careful about rewriting `Sender:' headers, and restrict this to special known cases in your own domains.

The SMTP-time rewriting flag

The rewrite flag `S' specifies a rewrite at SMTP time, as soon as an address is received and before any other processing; even before syntax checking. The pattern is required to be a regular expression. This applies to both sender and recipient addresses, and allows for the handling of addresses that are not compliant with RFC 822 (for example, `bang paths' in batched SMTP input). Because of this, the variables `$local_part' and `$domain' are not available during the expansion of the replacement string.

Flags controlling the rewriting process

There are four flags which control the way the rewriting process works. These take effect only when a rule is invoked, that is, when the address is of the correct type (matches the flags) and matches the pattern.

• If the `Q' flag is set on a rule, the rewritten address is permitted to be an unqualified local part. It is qualified with `qualify_recipient'. In the absence of `Q' the rewritten address must always include a domain.
• If the `q' flag is set on a rule, then no further rewriting rules are considered, even if no rewriting actually takes place because of a `fail' in the expansion. The `q' flag is not effective if the address is of the wrong type (does not match the flags) or does not match the pattern.
• The `R' flag causes a successful rewriting rule to be re-applied to the new address, up to ten times. It can be combined with the `q' flag, to stop rewriting once it fails to match (after at least one successful rewrite).
• When an address in a header is rewritten, the rewriting normally applies only to the working part of the address, with any comments and RFC 822 `phrase' left unchanged. For example, rewriting might change

  
  From: Ford Prefect <fp42@restaurant.hitch.fict.book>
  

  into

  
  From: Ford Prefect <prefectf@hitch.fict.book>
  

  Sometimes there is a need to replace the whole address item, and this can be done by adding the flag letter `w' to a rule. If this is set on a rule that causes an address in a header to be rewritten, the entire address is replaced, not just the working part. The replacement must be a complete RFC 822 address, including the angle brackets if necessary. When the `w' flag is set on a rule that causes an envelope address to be rewritten, all but the working part of the replacement address is discarded.

The additional relay checking flag

The `X' flag is a slightly strange oddity that adds additional checking to `sender_address_relay'. Whenever an address passes the `sender_address_relay' check, if there are any rewriting rules with the `X' flag set, the address is rewritten and if this makes any change to the address, it must verify successfully for the relaying to be permitted.

We use this in Cambridge as follows: users have a centrally registered address in the virtual domain `cam.ac.uk', but there are a number of different hosts where they actually have their accounts and from which they can read mail using IMAP or POP. It is desirable to prevent them using hosts other than those on which they have accounts as outgoing relays, and yet to permit the sending addresses to contain the `cam.ac.uk' domain. Since the user names are the same on the relay hosts as in the `cam.ac.uk' domain, a rewriting rule of the form

*@cam.ac.uk  $1@${qualify_domain}  X

means that any sender address of the form `user@cam.ac.uk' is acceptable only if `user' has an account on the local host. This also has the virtue of detecting typos in the configurations of users' MUAs.

Rewriting examples

Here is an example of the two common rewriting paradigms:

*@*.hitch.book.fict  $1@hitch.book.fict
*@hitch.book.fict    ${lookup{$1}dbm{/etc/realnames}\
                     {$value}fail}@hitch.book.fict bctfrF

Note the use of `fail' in the lookup expansion. This causes the string expansion to fail, and in this context it has the effect of leaving the original address unchanged, but Exim goes on to consider subsequent rewriting rules, if any, since the `q' flag is not present in that rule. An alternative to `fail' would be to supply `$1' explicitly, which would cause the rewritten address to be the same as before, at the cost of a small bit of processing. Not supplying either of these is an error, since the rewritten address would then contain no local part.

The first example above replaces the domain with a superior, more general domain. This may not be desirable for certain local parts. If the rule

root@*.hitch.book.fict  *

were inserted as the first rule, rewriting is suppressed if the local part is `root' at any domain ending in `hitch.book.fict'.

*@*.hitch.book.fict  "${if !eq {$sender_host_address}{}\
                      {$1@hitch.book.fict}fail}"

The replacement string is quoted in this example because it contains white space.

Exim does not handle addresses in the form of `bang paths'. If it sees such an address it treats it as an unqualified local part which it qualifies with the local qualification domain (if the source of the message is local or if the remote host is permitted to send unqualified addresses). Rewriting can sometimes be used to handle simple bang paths with a fixed number of components. For example, the rule

^([^!]+)!(.*)@your\.domain$   $2@$1

rewrites a two-component bang path `host.name!user' as the domain address `user@host.name'. However, there is a security implication in doing this as a normal rewriting rule for envelope addresses. It can provide a backdoor method for using your system as a relay, since the incoming addresses appear to be local. If the bang path addresses are received via SMTP, it is safer to use the `S' flag to rewrite them as they are received, so that relay checking can be done on the rewritten addresses.

Go to the first, previous, next, last section, table of contents.