Protocols

SMTP — the way out

The protocol every email is sent with dates from 1982 and consists of sentences you could read aloud. Listen in once and most of the system's problems explain themselves afterwards.

A complete send

This is what it looks like when one server hands a message to another. C: is the sending machine, S: the receiving one.

S: 220 mail.example.com ESMTP ready
C: EHLO sender.example
S: 250-mail.example.com
S: 250-SIZE 52428800
S: 250-STARTTLS
S: 250 8BITMIME
C: STARTTLS
S: 220 2.0.0 Ready to start TLS
   … encrypted from here on, the dialogue starts over …
C: MAIL FROM:<anna@sender.example>
S: 250 2.1.0 Ok
C: RCPT TO:<bert@example.com>
S: 250 2.1.5 Ok
C: DATA
S: 354 End data with <CR><LF>.<CR><LF>
C: From: Anna <anna@sender.example>
C: To: Bert <bert@example.com>
C: Subject: Lunch
C:
C: Half twelve?
C: .
S: 250 2.0.0 Ok: queued as 4A2F1C09CA
C: QUIT
S: 221 2.0.0 Bye

Twenty lines and the message is on its way. The structure comes from RFC 821 of 1982 and is still valid unchanged; everything added since hangs off the EHLO in line two.

What is contained in this dialogue

The reply codes say whose fault it is

Every reply begins with a three-digit number, and the first digit is the important one. 2xx means done. 4xx means: not right now, try again later — the sending server keeps the message and delivers it again. 5xx means a final no, with an error report to the sender.

That distinction is why greylisting works: a server answers an unknown sender with 4xx once. Real mail servers come back, many bulk senders do not.

The sender is stated, not verified

In MAIL FROM the sending machine says in whose name it is delivering. Nothing in the protocol provides for checking that. You can put any address you like there, and formally it is correct SMTP.

It is into exactly this gap that SPF, DKIM and DMARC were later built — not as a change to the protocol but as three procedures alongside it.

The envelope is not the letterhead

Note that the addresses appear twice: once in MAIL FROM and RCPT TO, then again in the From: and To: lines after DATA. Those are two different things, and they need not match. The distinction has such far-reaching consequences that it gets a page of its own.

Encryption is an add-on

STARTTLS appears in the list of capabilities the receiving server enumerates after EHLO. It is an extension from 2002, not a basic property. If the other side does not offer it, the transfer happens in clear text — and the sending server will as a rule do just that rather than refuse to deliver.

What is encrypted is therefore the leg between two servers, not the message. At every intermediate stop it lies there unencrypted. That is the difference between transport encryption and end-to-end encryption, and the two are regularly confused.

Passing on rather than delivering

SMTP is a relay race. The sending server hands over to the next one, that one confirms it has taken charge, and from then on it is responsible. If it cannot get any further it tries again over hours or days, and only then reports back to the sender.

From which follows something that surprises many people: a successful handover does not mean the message has arrived. It means the next server has accepted it. What happens after that — forwarding, filtering, silent discarding — the sender does not necessarily get to hear about.

Submitting is not the same as delivering

When a mail program sends a message it is not addressing the same service as a server that delivers. Delivery runs over port 25, submission by users over port 587 — with authentication, mandatory encryption, and the server's right to amend or refuse the message.

That separation came in 1998 and was one of the most effective measures against bulk sending there has ever been: it lets network operators block port 25 out of consumer networks without preventing anyone from sending mail.

Why it is allowed to look this old

A line-based clear-text protocol looks antiquated today. It has, however, proved extraordinarily durable, for one reason: you can read it. Debugging SMTP consists of reading along with the dialogue — no tooling, no intermediate format, no interpretation. What the server says is right there.

The problems of the email system do not lie in this protocol. They lie in what it does not provide for: no check on the sender, no report on a message's final fate, no cost for sending.