http.client.HTTPResponse.read() Permanently Closes the Connection on None (tested)

A non-blocking raw stream signals "no data ready yet" by returning None -- distinct from the empty read that means real end-of-stream. We measured that HTTPResponse.read(amt)'s sized-read path does s = self.fp.read(amt) and then if not s and amt: self._close_conn() -- treating a falsy result as the connection ending early and closing it as a side effect, before even returning. Every other guide in this family has a working fix: catch the None and retry the read. Here that fix doesn't work, because by the time the caller sees the None, _close_conn() has already run -- the next read() call hits if self.fp is None: return b"" as its very first line and returns empty forever. The real, complete, Content-Length-declared body is gone, one read call away and unrecoverable.

1. A real, valid response body behind one transient None, mid-read

import http.client, io

class FlakyRaw(io.RawIOBase):
    """Sends headers cleanly, then one transient None mid-body, then the body."""
    def __init__(self, header_bytes, body_bytes):
        self.header_bytes, self.body_bytes, self.stage = header_bytes, body_bytes, 0
    def readable(self):
        return True
    def readinto(self, b):
        if self.stage == 0:
            n = len(self.header_bytes)
            b[:n] = self.header_bytes
            self.stage = 1
            return n
        if self.stage == 1:
            self.stage = 2
            return None            # transient: no data ready yet, mid-body
        if self.stage == 2:
            n = len(self.body_bytes)
            b[:n] = self.body_bytes
            self.stage = 3
            return n               # the real, complete body
        return 0                   # real EOF after that

class FakeSocket:
    def __init__(self, raw):
        self.raw = raw
    def makefile(self, mode, *a, **kw):
        return io.BufferedReader(self.raw)
    def close(self):
        pass
    def sendall(self, data):
        pass

body = b"hello world, this is the response body"
headers = b"HTTP/1.1 200 OK\r\nContent-Length: %d\r\n\r\n" % len(body)
conn = http.client.HTTPConnection("example.com")
conn.sock = FakeSocket(FlakyRaw(headers, body))
conn.request("GET", "/")
resp = conn.getresponse()

chunk = resp.read(1024)
print("chunk:", chunk)
print("resp.fp is None (connection closed)?", resp.fp is None)
retry = resp.read(1024)
print("retrying read() after the None:", retry)
chunk: None
resp.fp is None (connection closed)? True
retrying read() after the None: b''

Measured: the body is valid and complete, one read call away. resp.read(1024) returned None and closed the connection in the same call. Retrying read() -- the fix that works for every other finding in this family -- returns b'' forever, because the connection is already gone. The data is not delayed; it is lost.

2. The fix has to happen one layer down, before HTTPResponse ever sees the None

def read_retrying_on_none(buffered_reader, chunk_size=65536):
    out = io.BytesIO()
    while True:
        chunk = buffered_reader.read(chunk_size)
        if chunk is None:
            continue   # retry: not ready yet, not real EOF
        if not chunk:
            break      # real EOF
        out.write(chunk)
    out.seek(0)
    return out

raw2 = FlakyRaw(headers, body)
br2 = io.BufferedReader(raw2)
raw_bytes = read_retrying_on_none(br2).read()
recovered_body = raw_bytes.split(b"\r\n\r\n", 1)[1]
print("recovered body:", recovered_body)
print("matches original:", recovered_body == body)
recovered body: b'hello world, this is the response body'
matches original: True

Measured: this only works by draining the raw, non-blocking socket stream directly, retrying on None before any of it reaches http.client's own connection-management logic -- here reconstructing the full response (headers and body) from raw bytes. Once a None reaches HTTPResponse.read() itself, it's too late -- the fix has to sit underneath the socket http.client wraps, not around the calls you make to HTTPResponse.

Frequently asked questions

Is this documented behavior?

http.client's documentation describes read(amt) as reading and returning up to amt bytes of the response body; it says nothing about None or non-blocking sockets. We measured that a None-returning underlying read is treated identically to a real early connection close, which is itself undocumented for the None case specifically.

Why is this worse than the already-confirmed HTTPResponse chunked-response finding?

The already-confirmed finding (readline()-based, used for chunked responses) raises a catchable RemoteDisconnected exception -- a caller can catch it, know something went wrong, and decide what to do. Here, read(amt) returns None with no exception at all, and silently closes the connection as a side effect -- there's no exception to catch, and by the time code notices the None, retrying is already futile.

Does this affect the unbounded read() (no argument) the same way?

Not the same way. We measured read() with no argument separately: with Content-Length it raises TypeError: object of type 'NoneType' has no len(), with chunked encoding IncompleteRead(0 bytes read), and with no length it returns None and closes the connection -- see our guide on resp.read() failing three different ways. The sized-read case shown here is the one hit by callers that loop with a fixed chunk size, like xmlrpc.client.Transport.parse_response()'s while data := stream.read(1024):.

Is this related to the other None/EOF findings?

Yes -- same family already confirmed in json.load(), quopri.decode(), codecs.StreamReader, and others: a correctly-returned non-blocking None gets mishandled one layer above io itself. This is the most severe case in the family so far, because it's the only one where the standard fix of catching and retrying the None doesn't work -- the bug destroys the connection before a caller even gets the chance.

Written with AI assistance. Measured on 2026-10-09 with Python 3.13.5's built-in http.client.HTTPResponse.read(amt), reading a genuinely valid, complete Content-Length response body through a custom non-blocking socket whose underlying stream returns None once, mid-body, before the real bytes. PlainHeaders is our extension; Python's http.client and io modules are built in.