shutil.copyfileobj() Treats None Like EOF (Tested)

A non-blocking file-like source signals "no data ready yet" by returning None from read() -- distinct from the empty bytes (b'') that mean real end-of-stream. io.BufferedReader.read() itself gets this right: it returns None as-is when nothing is ready. But shutil.copyfileobj(), the function behind shutil.copy() and commonly used to stream between file-like objects, checks its loop condition with a plain if not buf: break. In Python, not None and not b'' are both True -- so the very first transient None ends the copy exactly as if the source had hit real EOF, with no exception and no indication that more data was still on its way.

1. The source: a non-blocking raw stream that is briefly not ready

import io

class FlakyRaw(io.RawIOBase):
    def __init__(self):
        self.calls = 0
    def readable(self):
        return True
    def readinto(self, b):
        self.calls += 1
        if self.calls == 1:
            return None              # transient: no data ready yet
        if self.calls == 2:
            data = b"hello world"
            b[:len(data)] = data
            return len(data)
        return 0                     # real EOF on later calls

src = io.BufferedReader(FlakyRaw())
print("direct read():", repr(src.read(64 * 1024)), "calls so far:", src.raw.calls)

direct read(): None calls so far: 1

Measured: called directly, BufferedReader.read() correctly returns None -- a caller checking is None can tell "not ready" apart from "done" and retry.

2. shutil.copyfileobj() on the same source stops after that first None

import shutil, io

class FlakyRaw(io.RawIOBase):
    def __init__(self):
        self.calls = 0
    def readable(self):
        return True
    def readinto(self, b):
        self.calls += 1
        if self.calls == 1:
            return None
        if self.calls == 2:
            data = b"hello world"
            b[:len(data)] = data
            return len(data)
        return 0

src = io.BufferedReader(FlakyRaw())
dst = io.BytesIO()
shutil.copyfileobj(src, dst)
print("copied bytes:", repr(dst.getvalue()))
print("raw.read calls made:", src.raw.calls)

copied bytes: b''
raw.read calls made: 1

Measured: copyfileobj() made exactly one read call, got None, and returned -- with dst completely empty and no exception raised at all. The "hello world" that the second call would have produced is simply lost; nothing in the call site indicates the copy was incomplete.

3. Why: copyfileobj's loop check is "falsy", not "is None" or "is b''"

import inspect, shutil
print(inspect.getsource(shutil.copyfileobj))

# (abbreviated from the standard library source)
def copyfileobj(fsrc, fdst, length=0):
    ...
    fsrc_read = fsrc.read
    fdst_write = fdst.write
    while True:
        buf = fsrc_read(length)
        if not buf:
            break
        fdst_write(buf)

Measured: the loop's only stop condition is if not buf. That is true for real EOF (b''), but it is equally true for a transient None -- the function has no way to distinguish "the source is exhausted" from "the source just wasn't ready for this one call," because it never checks which of the two falsy values it actually got.

4. A source that never returns at all: ordinary files are unaffected

This only matters for file-like objects whose read() can legitimately return None -- non-blocking raw streams, certain sockets or pipes wrapped in io.RawIOBase. A plain open() file or io.BytesIO never returns None from read(), so shutil.copyfileobj() is safe for the overwhelming majority of its real-world uses -- the gap only opens when something non-blocking is on the other end.

5. The fix: don't feed a None-capable source straight to copyfileobj

def safe_copy(fsrc, fdst, length=64 * 1024):
    while True:
        buf = fsrc.read(length)
        if buf is None:
            continue              # retry: not ready yet, not EOF
        if not buf:
            break                  # real EOF
        fdst.write(buf)

src2 = io.BufferedReader(FlakyRaw())
dst2 = io.BytesIO()
safe_copy(src2, dst2)
print("safe_copy result:", repr(dst2.getvalue()))

safe_copy result: b'hello world'

Measured: explicitly checking is None before the general falsy check reads the real data correctly instead of stopping on the first transient signal. shutil.copyfileobj() itself offers no such option -- it must be avoided, not configured, when the source can return None.

Written with AI assistance. Measured on 2026-10-09 with Python 3.13.5's built-in shutil.copyfileobj() and io.BufferedReader wrapping a custom non-blocking RawIOBase whose readinto() returns None once (no data ready yet) before returning real data. PlainHeaders is our extension; Python's shutil and io modules are built in.