* SMTP (2821) Implementation Information, Used in Preparing RFC2821bis * Ten implementors responded to a questionnaire, with responses focusing on which features were *not* implemented, or not interoperable. # 1 Implemented since 1998 not implemented VRFY and EXPN ---------------------------------------------------------------- #2 Implemented 2005 all MUSTS implemented not implemented: no Received header prepended because internal storage used no source-routed addresses ---------------------------------------------------------------- #3 originally written in 1993, rewritten in 2006 do not implement: We don't implement forwarding mail using a corrected email address (3.4), i.e we never return 251 response. We don't generate multiple FOR addresses in Received: header fields, But luckily this feature was removed from 2821bis. EXPN & VRFY are implemented but disabled by default problems: Lack of a definition of qcontent is causing us some troubles. If the definition of qcontent from RFC 2822 is used, then spaces are not allowed in local-parts (unlike RFC 822). A lack of spaces in local- part is a problem for a gateway from another mail system, where that system's addresses are encoded in the local-part. ---------------------------------------------------------------- #4 Implement 2001 not implemented: we implement all of 2821 any differences from 2821 are considered bugs and fixed when found VRFY/EXPN are minimal: no useful data is generated never generates 211, 251, 252, 452, 502, 504, 551, or 552 responses. does not handle the obs- syntax strictly according to the spec; in many cases I ignore them entirely. (For example, 2822, referred to by 2821, says that \ quotes even a NUL, in obs-qp; I ignore that requirement.) ---------------------------------------------------------------- #5 Implemented 2001 not implemented: ================================================== RFC 2821 says that the sender must shuffle equal-preference MX hosts, whereas multiple A records per hostname must be used in the order as received. They make the bogus assumption that a hostname with multiple A records corresponds to one machine with multiple network interfaces. Postfix does not assume that one MX record == one host. When the MX record expands into a group of A records, Postfix will not skip the entire group just because one member fails. Postfix treats each A record as if it was listed under a separate MX record. In other words, it does not assume that one MX record == one host. ================================================== RFC 2821 section 4.5.3.1 says that a 552 RCPT TO reply must be treated as if the server replied with 452. However, this causes "too much mail data" to be treated as a recoverable error, which is wrong. I'll stick with RFC 821. ================================================== RFC 2821: An SMTP server is not allowed to "clean up" mail except in the case of original submissions. Postfix inserts missing but required message headers. It may be configured to munge addresses in transit mail. ================================================== RFC 2821 pedantism: Section 4.1.2 says that SMTP servers that receive a command in which invalid character codes have been employed, and for which there are no other reasons for rejection, MUST reject that command with a 501 response. Postfix attempts to be 8-bit clean. 8-bit cleanness definitely was an issue in the old days because people had legitimate needs to use 8-bit but there was no legitimate way to do so. ================================================== RFC 2821 violates RFC 821, which says that NOOP takes no parameters. ================================================== RFC 2821 new feature: Section 3.5.1 requires that the VRFY response is either "full name " or "user@domain". Postfix replies with the address that was provided by the client, whether or not it is in fully qualified domain form or not. Thanks to rampant abuse VRFY is often turned off so this is no longer an issue. ================================================== RFC 2821 incompatibility and brain damage: Section 4.5.1 requires that VRFY is implemented. RFC 821 specifies that VRFY is optional. It gets even worse: section 3.5.3 says that a 502 (command recognized but not implemented) reply is not fully compliant. Thus, an RFC 2821 compliant implementation cannot refuse to supply information in reply to VRFY queries. That is simply bogus. The only reply we could supply is a generic 252 reply. This causes spammers to add tons of bogus addresses to their mailing lists (spam harvesting by trying out large lists of potential recipient names with VRFY). Thanks to rampant abuse VRFY is often turned off so this is no longer an issue. ================================================== problems with: Problematic were the NOOP/VRFY changes that made legitimate RFC821 behavior non-compliant. ---------------------------------------------------------------- #6 Implemented originally 1996, revised for 2821 in 2002 not implemented: For instance, in section 4.5.3.1 we haven't implemented the restrictions there - eg our software will handle a command line of 2000 characters or more even though RFC 2821 says it has a maximum of 512 characters. (It won't GENERATE lines longer than 512 characters). Similarly, in section 4.5.3.2, we don't currently have different timeouts for different commands, the timeouts are all 5 minutes. This is allowed by RFC 2821 even though the 'DATA termination' Timeout SHOULD be longer. We have implemented all commands, and as far as we know, all 'MUST' requirements have been implemented. Some 'SHOULD' recommendations have not been implemented - eg Section 4.5.4.1 "A client SHOULD keep a list of hosts it cannot reach and corresponding connection timeouts, rather than just retrying queued mail items." - we haven't implemented that as it hasn't been an issue for our customer base that it just retries all previously temporarily failed items. ---------------------------------------------------------------- #7 Implemented 2002 revised latest mid-2007 not implemented VRFY always returns 252, EXPN not implemented problems with VRFY and EXPN. 64 character limit on local-part, particularly when bloated up with BATV and VERP. ---------------------------------------------------------------- #8 Implemented 2002-2003 not implemented: 251/551 response rerouting is not supported on either the client or server side. IPv6 domain literals are not spported. Our VRFY implementation doesn't support ambiguous matching and Returning multiple addresses. Full user name information is not returned by either VRFY or EXPN. Multiple entries in a Received: FOR clause are not supported. Our default configuration does not honor rules (section 2.4, 3.6.3, 3.9 and elsewhere) about not touching in-transit messages (although this can be enabled we strongly discourage doing so). Additionally, any NULs are dropped unconditionally from message content. We deal with the requirement that "552 Too many recipients" errors be treated as a transient failure (due to a bug in RFC 821) not by treating all 552 RCPT TO responses as transient errors but instead by checking the response text against a configurable list. The response is treated as transient if it is found on the list. problems with: We allow "postmaster" as an address and deal with it by treating it as belonging to a site-defined default domain. However, in many cases there simply is no sensible default for this. GIGO. Note that similar issues exist in multi-domain setups for VRFY when no domain is specified. It should be noted that the only relay policy that simultaneously meets the requirements that: (1) Bare CR and LF not be emitted (section 2.3.8) (2) Bare CR and LF not be interpretted as line breaks (section 2.3.8) (3) Message content must not be modified in transit (section 2.4, 3.6.3, 3.9 and elsewhere) is one where messages containing bare CR and LF are rejected as invalid. Unfortunately legitimate messages often contain bare CR and LF so this is not a tenable strategy in practice. Our "solution" is to drop bare CRs by default. Bare LFs are treated as a normal character at the SMTP level by default, however, subsequent message processing will usually result in bare LFs in message content turning into line breaks. The last sentence in section 3.5.3 states that VRFY checking SHOULD be more thorough than RCPT TO checking. The checks we perform are the same for both and are as thorough as they can be. Why would we want to unnecessarily weaken RCPT TO checking to comply with this? 2821bis suggests that the maximum number of Received: fields allowed default to 100. Our default is 50 - we find 100 doesn't catch loops soon enough. The rule that connections can only be dropped when the server needs to shut down, and in particular cannot be done in response to bad commands is one our implementation honors by default. However, we provide options to drop connections after a specified amount of certain forms of bad client behavior (e.g., too many invalid commands). Our customers insist on this capability and we believe it is widely used, irrespective of its standards status. other comments: The text about the interaction of IPv6 and MX records in section 5.2 is excellent. It is refreshing to see such a pragmatic approach taken here, although once sufficient operational experience exists it would be great to describe the various options in an ancillary BCP. The text about the right thing for a client to do in the event of a connection drop is in a very odd place (RSET command description, section 4.1.1.5). A more general location would be better. Additionally, the text states that implementations SHOULD be preated to deal with connection drops. That's not good enough � connection drops are a fact of life and every implementation MUST be prepared to handle them. Finally, the recommendation that a connection drop be treated as if a QUIT was received ignores the fact that abnormal terminating conditions need to be logged as such. Accordingly, correct treatement of dropped connections SHOULD be the same as QUIT from the perspective of any outstanding transaction but MAY be logged differently. The requirement that domains always be FQDNs ends up being somewhat problematic in practice. On many modern systems it is actually nontrivial for that system to reliably determine its own FQDN short of explicit configuration. Naive implementations will fail to comply given the present state of OS APIs. This leads to short form names in Recieved: fields, as an argument to EHLO and various other places. Perhaps any revision work on socket APIs should take this into account. The requirement near the end of section 4.2 saying that SMTP clients MUST base their actions only on reply codes and not on reply text is nonsensical and contradictory. First and foremost, it directly contradicts 2821bis itself in the case of VRFY and EXPN responses, where the client does look at the response text and may act differently depending on what it says. It also effectively disallows the use of enhanced reply codes defined elsewhere. Finally, it effectively prohibits clients from working around server bugs and inconsistencies by using reply code text to determine what server you're talking to, such as how we have elected to deal with the 552 response issue as previously described. This is one of those requirements that was added with the best of intentions: To try and prevent people from doing silly and stupid stuff. However, it goes way too far and interferes with too many useful, legitimate and even essential services. One solution would be to add more exceptions to the already long list that's already in the specification). But by the time you have them all you're probably better off listing the reply coddes where it isn't OK to look at the text because the list may well be shorter! A better solution would be to change the spec to say that response text should only be looked at by clients in special cases and MUST NOT be used to override the semantics of the reply code itself. ---------------------------------------------------------------- #9 Implemented 2001-2002 not implemented: All MUST requirements from RFC2821 are implemented. There are a few cases where nonconformant behavior can be configured. For example, the RFC states that, "The custom of accepting lines ending only in , as a concession to non-conforming behavior on the part of some UNIX systems, has proven to cause more interoperability problems than it solves, and SMTP server systems MUST NOT do this, even in the name of improved robustness. In particular, the sequence "." (bare line feeds, without carriage returns) MUST NOT be treated as equivalent to . as the end of mail data indication." Our MTA can be configured to accept LF as a line terminator, although this is not the default and sites are warned against doing this. I believe that all SHOULD [NOT] behavior is conformed to as well, although there are cases where the MTA can be configured to behave in ways that are not conformant to the SHOULD [NOT] language in the spec. ---------------------------------------------------------------- #10 Implemented 1997, revised 2002 for 2821 not implemented: Return-Path not prepended to message upon receipt