emailmarketing.net

ESMTP Extensions for Senders — SIZE, PIPELINING, CHUNKING, DSN

What RFC 1870 SIZE, RFC 2920 PIPELINING, RFC 3030 CHUNKING/BINARYMIME, and RFC 3461 DSN (NOTIFY/RET/ENVID/ORCPT) do, and when each matters on an ESP's outbound path.

Foundationalesp-operatorsender

ESMTP extensions are negotiated per session: the server lists keywords in its EHLO response; the client may then use the corresponding commands/parameters. A sending MTA should parse the EHLO extension list on every connection and adapt — these four extensions are the ones with direct outbound-throughput and bounce-quality consequences. (8BITMIME/RFC 6152 and SMTPUTF8 are covered in MIME & Encoding and SMTPUTF8/EAI; STARTTLS in transport security.)

RFC 1870 — SIZE (message size declaration)

  • EHLO keyword: SIZE with optional numeric parameter = the fixed maximum message size in octets the server will accept. SIZE 0 means no fixed maximum; SIZE with no number conveys no limit information.
  • MAIL parameter: MAIL FROM:<...> SIZE=nnn (1–20 digits). Declared size = octets the client will transmit after the 354, including CRLF pairs but excluding the terminating dot and dot-stuffing doubles — i.e., headers + body as sent. An estimate is fine; heuristics should err high.
  • Server replies: 552 (permanent — declared size exceeds the fixed maximum; don't retry there) vs. 452 (temporary — insufficient resources right now; retry later). The server may check at MAIL time or after DATA; it may accept a message larger than declared, and should not issue a post-DATA 552 unless the actual size exceeded the declared size.

When it matters: declaring SIZE turns an over-limit send into a cheap pre-DATA rejection instead of transferring megabytes and getting 552 after the final dot — at scale that is real bandwidth and queue time. Reading the advertised limit lets the MTA fail oversized messages locally with a clear bounce ("exceeds example.com's 26214400-octet limit") and lets a platform enforce per-destination composition limits. Remember the advertised limit is on the encoded message — base64 inflates ~37% (see MIME & Encoding). Distinguish 552 from SIZE (message too big — a content problem) from RFC 5321's 552 "exceeded storage allocation" (mailbox full — a recipient problem): the enhanced status code (5.3.4 vs 5.2.2) disambiguates.

RFC 2920 — PIPELINING (command batching)

  • EHLO keyword: PIPELINING, no parameters. Never pipeline into a server that didn't advertise it.
  • What may be pipelined: RSET, MAIL FROM, RCPT TO (also the obsolete SEND/SOML/SAML) may appear anywhere in a group. EHLO, DATA, VRFY, EXPN, TURN, QUIT, NOOP may only be the last command in a group (their success/failure changes state; NOOP is usable as a synchronization point).
  • Client obligations: check every status in the group, correlating replies to commands by count — you MUST NOT assume DATA failed just because all RCPTs failed; read its reply. Without nonblocking I/O the client must keep each group within the TCP window to avoid deadlock.
  • Server obligations: reply in order received; buffer replies to pipelineable commands but never to the group-enders; issue a positive DATA reply only if ≥1 valid RCPT was accepted; if it 354'd with no valid recipients it must accept-and-discard the message; never flush the TCP input buffer.

When it matters: without pipelining, a transaction costs one round-trip per command (MAIL, each RCPT, DATA = N+2 RTTs). With it, MAIL FROM + all RCPT TOs + DATA go out in one write ≈ 1 RTT before content. Over a 100 ms transatlantic path, that's the difference between ~3–4 messages/second/connection and ~10+ — pipelining is the single cheapest throughput multiplier an outbound MTA has, before adding connections (which provider connection caps limit). Virtually all major receivers advertise it. One caution: some anti-spam layers score "early talkers" (clients that send before the banner or pipeline without the advertisement) as bot behavior — negotiate strictly.

RFC 3030 — CHUNKING (BDAT) and BINARYMIME

  • EHLO keywords: CHUNKING (enables BDAT) and BINARYMIME (binary content without transfer encoding; usable only together with CHUNKING).
  • BDAT syntax: BDAT chunk-size [LAST] — the exact octet count follows immediately after the command's CRLF. No dot-stuffing, no <CRLF>.<CRLF> terminator scan; the final chunk carries LAST (and may be zero-length). Server MUST 250 each successful chunk.
  • Errors: on a mid-transaction failure the server MUST still consume and discard the announced chunk octets before replying 4xx/5xx; after any 4xx/5xx to a BDAT the client MUST NOT send further BDAT segments — issue RSET to clear the transaction.
  • Mixing rules: DATA and BDAT MUST NOT be mixed in one transaction (503), though both may appear in different transactions of one session. BODY=BINARYMIME on MAIL commits the transaction to BDAT — DATA after it is 503. Accepting BINARYMIME obliges the receiver to preserve every bit of every octet onward; text/* must still use CRLF lines.

When it matters: for typical marketing mail, rarely — messages are small and already 7-bit-safe, and several large receivers don't advertise CHUNKING, so the DATA path must exist anyway. It matters for very large messages (no terminator scanning, precise accounting), for genuinely binary payloads without base64's ~37% inflation, and BDAT is the native framing in some modern submission paths. Interop note: BDAT handling bugs in filters/gateways have historically caused parsing discrepancies (it was an SMTP-smuggling vector in receivers), so some operators deliberately keep it off; treat it as an optimization to enable per-destination, never a requirement.

RFC 3461 — DSN extension (NOTIFY / RET / ENVID / ORCPT)

The SMTP side of delivery status notifications — it controls when DSNs are generated and threads correlation data through; RFC 3464 defines the bounce format itself.

  • EHLO keyword: DSN, no parameters.

Per-recipient (RCPT TO) parameters:

Parameter Values / syntax Semantics
NOTIFY NEVER | comma-list of SUCCESS,FAILURE,DELAY NEVER must stand alone: no DSN under any conditions. SUCCESS: DSN on delivery. FAILURE: DSN on failure. DELAY: sender is willing to receive delayed DSNs. Absent = receiver default, effectively FAILURE or FAILURE,DELAY
ORCPT addr-type;xtext-address (e.g. rfc822;user+2Btag@example.com), ≤500 chars Original recipient as the sender specified it; propagated unchanged through forwarding so the DSN's Original-Recipient field survives rewrites. At submission it must match the RCPT TO address

Per-message (MAIL FROM) parameters:

Parameter Values / syntax Semantics
RET FULL | HDRS Return the whole message vs. headers only in failure DSNs (success DSNs return headers at most). Absent: receiver's choice
ENVID xtext, ≤100 chars (UAs advised ≤38) Opaque sender transaction ID, echoed verbatim in the DSN's Original-Envelope-Id — direct bounce-to-send correlation without parsing message headers
  • xtext encoding: printable ASCII 33–126 except + and = pass through; everything else becomes +HH (uppercase hex). Values must be US-ASCII before encoding.
  • Command-line consequence: a conforming MTA must handle command lines of at least 1036 characters (512 base + 530 for ORCPT/NOTIFY on RCPT).
  • Server neutrality: a server MUST NOT refuse MAIL/RCPT merely because of valid DSN parameters; malformed/duplicate parameters draw 501.
  • Relay rules: to a DSN-capable next hop, ENVID/RET/NOTIFY/ORCPT MUST be propagated exactly (ORCPT case-sensitively; a relay may add ORCPT if absent). To a non-DSN next hop the parameters MUST NOT be sent; instead the relaying client compensates: NOTIFY=SUCCESS + accepted → it MUST generate a "relayed" DSN; failure (5xx) → "failed" DSN unless NOTIFY=NEVER; NOTIFY=NEVER → no DSN ever.
  • Mailing lists: redistribution MUST NOT carry the original submission's DSN parameters (the list sets its own, with its own return path), so success-tracking stops at the list; a NOTIFY=SUCCESS submission gets its success DSN from the list-delivery MTA.

When it matters for an ESP:

  • NOTIFY=NEVER is the operational gem: RFC 3834 recommends it when sending auto-replies, and it's equally useful on any traffic whose bounces you intend to detect by other means or deliberately don't want (e.g., probe traffic) — where the receiver honors it, it suppresses NDR generation at the source. Do not use it on normal campaign mail: you need failure DSNs for bounce processing.
  • ENVID gives per-message bounce correlation that survives even when the returned content is stripped — a complement to VERP (VERP survives non-RFC-3464 bounces; ENVID survives VERP-hostile forwarders). Cheap to set; set it.
  • RET=HDRS keeps failure DSNs small — polite and bandwidth-saving when your messages are heavy.
  • NOTIFY=SUCCESS is not a delivery-tracking tool at scale: most large mailbox providers ignore it or don't emit success DSNs, and requesting per-recipient success notifications across millions of messages would be its own traffic problem. Rely on SMTP acceptance + engagement signals instead.

Reading an EHLO response — quick sender checklist

Keyword seen Sender action
SIZE n Reject/bounce locally anything whose encoded size > n; declare SIZE= on MAIL
PIPELINING Batch MAIL+RCPTs+DATA into one write; verify every reply by count
8BITMIME May send BODY=8BITMIME; otherwise ensure QP/base64 encoding (never "just send 8-bit")
CHUNKING May use BDAT; keep DATA fallback
DSN Attach ENVID/RET (and NOTIFY where appropriate); expect RFC 3464-format bounces
SMTPUTF8 EAI recipients deliverable on this hop — see SMTPUTF8/EAI
STARTTLS Negotiate TLS before mail (required by MTA-STS policies)

See also

#rfc#esmtp#smtp-extension#size#pipelining#chunking#bdat#dsn#notify#envid