patch smtplib:when SMTPDataError, rset crashes with sslerror

@warsaw

BPO	1481032
Nosy	@warsaw, @terryjreedy, @tiran, @bitdancer, @zvyn
Files	smtplib_nosideeffecterror.patch issue1481032.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2006-05-03.11:59:13.000>
labels = ['3.7', 'type-bug', 'library']
title = 'patch smtplib:when SMTPDataError, rset crashes with sslerror'
updated_at = <Date 2016-09-08.15:03:02.164>
user = 'https://bugs.python.org/kxroberto'

bugs.python.org fields:

activity = <Date 2016-09-08.15:03:02.164>
actor = 'christian.heimes'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2006-05-03.11:59:13.000>
creator = 'kxroberto'
dependencies = []
files = ['18613', '36057']
hgrepos = []
issue_num = 1481032
keywords = ['patch']
message_count = 10.0
messages = ['50175', '50176', '50177', '114657', '114734', '147683', '223800', '226087', '226101', '275036']
nosy_count = 8.0
nosy_names = ['barry', 'terry.reedy', 'kxroberto', 'christian.heimes', 'hwundram', 'r.david.murray', 'jesstess', 'zvyn']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'test needed'
status = 'pending'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue1481032'
versions = ['Python 3.5', 'Python 3.6', 'Python 3.7']

Linked PRs

gh-43311: Add regression test for SSLError masking SMTPDataError in rset() #154866

File "smtplib.pyc", line 690, in sendmail
File "smtplib.pyc", line 445, in rset
File "smtplib.pyc", line 370, in docmd
File "smtplib.pyc", line 344, in getreply
File "smtplib.pyc", line 159, in readline
sslerror: (8, 'EOF occurred in violation of protocol')

traced from a py2.3 - yet unchanged.

=> hides original error SMTPDataError. such
SMTPDataError may have forced a disconnect of server.

patch for py2.3,py2.4,..

( I have this patch in my MUST-DO-PATCHes after any
Python installation. )

--

the patch passes on any error in this rset() location.
it also patches a error in PLAIN authentication

could cleanly catch socket.sslerror / socket.error /
EnvironmentError, yet ssl not always there ...
rset() is tested otherwise, so the nacked except is ok?

(there should maybe be special a common base Exception
class "IOBaseError" for : EnvironmentError(IOError,
OSError), EOFError, socket.error, socket.sslerror,
ftplib.all_errors, etc. Nice as it and not all IO
sublibs have to be imported to catch such errors.)

---

the same problem is with smtp.quit() on many SSL'ed
connections (without any other errors occuring): a
final socket.sslerror is raised during quit().
There may be a problem in the termination code of
ssl-FakeSockets or ssl.c . Or the same type of error
catch (on IOBaseError) should be applied. I am not sure

in my apps I (must) catch on smtp.quit() generally.

(Compare also bug bpo-978833 / shutdown(2)-remedy in
httplib's SSL FakeSocket - this shutdown(2) remedy
patch of bpo-978833 I still have it on my MUST list
(py2.3/py2.4 installations), otherwise this FakeSocket
doesn't close fully in a FTPS application (where
termination on data channel is crucial for getting
response on the control channel) - and most likely puts
tremendous connection load on HTTPS servers because of
stale unterminated HTTPS connections while the bug may
not be obvious in casual usage. I'm not completely
clear about the nature of this error. Thus, what I say
is based on trial-and-error.

-robert

Logged In: YES
user_id=791932

Where is the patch? :-)

Logged In: YES
user_id=972995

here (i forgot the upload checkbox?)

The change to the encode_plain method in the patch was done via bpo-1075928 on 2004-12-06. The try/except change around a call to self.rset() was never implemented.

still I think all 3 self.rset()'s in SMTP.sendmail, which are followed by a raise <someerror> statement have to be bracketed with an except clause or so - nowadays better like """
try: self.res()
except (EnvironmentError, SMTPException):
pass
""", as all socket.error, sslerror seem meanwhile correctly inherited (via IOError).

Because the original error must pop up, not a side effect of state cleaning!

ping!
perhaps I forgot to write that I uploaded the cleaned patch also on 2010-08-23. I really think this simple patch is necessary. Just seen the same problem again - as I forgot the patch in one of my recent Python update installations.

When SMTPDataError, SMTPRecipientsRefused, SMTPSenderRefused should be raised, very likely a subsequent error (like closed connection etc.) overlappes the original Exception during self.rset(), and one will not be able to locate the problem on user level in eons ..

This patch is still in my vital-patches collection which I have to apply after every Python update since years.

This bug was resolved for SMTPServerDisconnected exceptions but not for SSLError. Is it still reproducible? If so, the attached patch should fix it for 3.x (couldn't test if it does because I don't know how to reproduce the error).

David, do you agree with either of the proposed changes?

Milan's patch should be correct. It would be nice to have a reproducer.

Is this still an issue? The bug is 10 years old and the patch two years.

This looks resolved at main — I can't reproduce the masking. A TLS server that replies 554 to DATA and then breaks the connection (TCP RST, raw close, or clean close_notify) always surfaces SMTPDataError(554, ...) from sendmail(). Forcing a literal ssl.SSLError ([SSL: RECORD_LAYER_FAILURE]) on the rset() read gives SMTPServerDisconnected with the SSLError as __context__, which _rset() swallows.

Two changes since 2006 close it: bpo-17498 (afb151a) routes all three sendmail() error paths through _rset(), and ssl.SSLError subclasses OSError, so send()/getreply() already normalize it to SMTPServerDisconnected — the case that couldn't be tested back in 2014.

There's no regression test for the TLS variant (test__rest_from_mail_cmd is plaintext only); I can add one if that's worth having.

Reproducer (drop a throwaway cert alongside: openssl req -x509 -newkey rsa:2048 -keyout key.pem -out cert.pem -days 1 -nodes -subj /CN=localhost)

"""gh-43311 reproducer: does an SSLError from rset() after an SMTPDataError still
mask the original error at tip? Drives a real TLS SMTP server that 5xx-rejects
DATA then drops the connection, and reports which exception smtplib surfaces.

Run with the locally built interpreter: ./python.exe repro_tls.py
"""
import socket, ssl, struct, threading, smtplib, os

HERE = os.path.dirname(os.path.abspath(__file__))
CERT = os.path.join(HERE, "cert.pem")
KEY = os.path.join(HERE, "key.pem")


def make_server_ctx():
    ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
    ctx.load_cert_chain(CERT, KEY)
    return ctx


def make_client_ctx():
    ctx = ssl._create_unverified_context()
    return ctx


def readline(conn):
    buf = b""
    while not buf.endswith(b"\r\n"):
        b = conn.recv(1)
        if not b:
            break
        buf += b
    return buf


def serve(disconnect_mode, ready, port_holder):
    """One-shot TLS SMTP server. Rejects DATA with 554, then disconnects per mode."""
    lsock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    lsock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    lsock.bind(("127.0.0.1", 0))
    lsock.listen(1)
    port_holder.append(lsock.getsockname()[1])
    ready.set()
    raw, _ = lsock.accept()
    ctx = make_server_ctx()
    try:
        conn = ctx.wrap_socket(raw, server_side=True)
    except ssl.SSLError:
        lsock.close()
        return
    try:
        conn.sendall(b"220 fake ESMTP ready\r\n")
        readline(conn)                       # EHLO
        conn.sendall(b"250 fake.localhost\r\n")
        readline(conn)                       # MAIL FROM
        conn.sendall(b"250 OK\r\n")
        readline(conn)                       # RCPT TO
        conn.sendall(b"250 OK\r\n")
        readline(conn)                       # DATA
        conn.sendall(b"554 Transaction failed, closing now\r\n")
        # Now abruptly break the TLS channel so the client's _rset() hits a dead
        # connection -- the exact 2006 scenario (server aborts mid/after DATA).
        if disconnect_mode == "rst":
            # SO_LINGER 0 -> TCP RST on close, no TLS close_notify => client read
            # raises ssl.SSLError / ConnectionResetError on the next recv.
            conn.setsockopt(socket.SOL_SOCKET, socket.SO_LINGER,
                            struct.pack("ii", 1, 0))
            conn.close()
        elif disconnect_mode == "raw_close":
            # Close the underlying fd without TLS shutdown -> unexpected EOF.
            os.close(conn.fileno())
        elif disconnect_mode == "clean":
            conn.close()                     # clean TLS close_notify -> empty read
    except OSError:
        pass
    finally:
        lsock.close()


def run(disconnect_mode):
    ready = threading.Event()
    port_holder = []
    t = threading.Thread(target=serve, args=(disconnect_mode, ready, port_holder),
                         daemon=True)
    t.start()
    ready.wait(5)
    port = port_holder[0]
    cli_ctx = make_client_ctx()
    surfaced = None
    try:
        s = smtplib.SMTP_SSL("127.0.0.1", port, context=cli_ctx, timeout=5)
        s.sendmail("me@example.com", ["you@example.com"], "Subject: hi\r\n\r\nbody")
        surfaced = ("NO EXCEPTION", None)
    except BaseException as e:
        surfaced = (type(e).__name__, e)
    t.join(2)
    return surfaced


if __name__ == "__main__":
    for mode in ("rst", "raw_close", "clean"):
        name, exc = run(mode)
        print(f"\n=== disconnect_mode={mode!r} ===")
        print(f"  surfaced exception: {name}")
        print(f"  value: {exc!r}")
        is_data = name == "SMTPDataError"
        print(f"  -> original SMTPDataError surfaced (not masked): {is_data}")

=== disconnect_mode='rst' ===
  surfaced exception: SMTPDataError
  value: SMTPDataError(554, b'Transaction failed, closing now')
  -> original SMTPDataError surfaced (not masked): True

=== disconnect_mode='raw_close' ===
  surfaced exception: SMTPDataError
  value: SMTPDataError(554, b'Transaction failed, closing now')
  -> original SMTPDataError surfaced (not masked): True

=== disconnect_mode='clean' ===
  surfaced exception: SMTPDataError
  value: SMTPDataError(554, b'Transaction failed, closing now')
  -> original SMTPDataError surfaced (not masked): True

pythongh-43311: Add NEWS entry

Navigation Menu

Repository navigation

patch smtplib:when SMTPDataError, rset crashes with sslerror #43311

Linked PRs

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

patch smtplib:when SMTPDataError, rset crashes with sslerror #43311

Description

Linked PRs

Activity

kxroberto commented on May 3, 2006

hwundram commented on May 5, 2006

kxroberto commented on May 5, 2006

BreamoreBoy commented on Aug 22, 2010

kxroberto commented on Aug 23, 2010

kxroberto commented on Nov 15, 2011

zvyn commented on Jul 23, 2014

terryjreedy commented on Aug 29, 2014

bitdancer commented on Aug 29, 2014

tiran commented on Sep 8, 2016

soreavis commented on Jul 19, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions