tomllib.load() Misdiagnoses a Non-Blocking None (tested)

A non-blocking raw stream signals "no data ready yet" by returning None from readinto() -- distinct from the empty read that means real end-of-stream. We measured that tomllib.load() mishandles this signal in an unusually misleading way: rather than a generic crash or a content-blaming error, it raises TypeError: File must be opened in binary mode, e.g. use open('foo.toml', 'rb') -- even when the file object passed in is already a binary io.BufferedReader. The error actively diagnoses the wrong problem.

1. A real, valid TOML document behind one transient None

import io, tomllib

class FlakyRaw(io.RawIOBase):
    def __init__(self, payload):
        self.calls = 0
        self.sent = False
        self.payload = payload
    def readable(self):
        return True
    def readinto(self, b):
        self.calls += 1
        if self.calls == 1:
            return None            # transient: no data ready yet
        if not self.sent:
            self.sent = True
            n = len(self.payload)
            b[:n] = self.payload
            return n               # the real, valid TOML bytes
        return 0                   # real EOF after that

payload = b'key = "value"\n'
raw = FlakyRaw(payload)
br = io.BufferedReader(raw)   # genuinely binary mode
try:
    data = tomllib.load(br)
    print("data:", data)
except Exception as e:
    print("CRASHED:", type(e).__name__, e)
print("raw calls made:", raw.calls)

CRASHED: TypeError File must be opened in binary mode, e.g. use `open('foo.toml', 'rb')`
raw calls made: 1

Measured: br is an io.BufferedReader -- already binary mode, exactly as the error message demands. The TOML document is valid and complete, sitting one read call away. The error is simply wrong about what went wrong.

2. Why: tomllib's own binary-mode check is fooled by None

import inspect
print(inspect.getsource(tomllib.load))

# (from the standard library source)
def load(fp, /, *, parse_float=float):
    b = fp.read()
    try:
        s = b.decode()
    except AttributeError:
        raise TypeError(
            "File must be opened in binary mode, e.g. use `open('foo.toml', 'rb')`"
        ) from None
    return loads(s, parse_float=parse_float)

Measured: tomllib.load() calls fp.read() and immediately tries b.decode(), assuming b is bytes. If a caller passes a text-mode file object by mistake, read() returns str, which has no .decode() method, and the resulting AttributeError is caught and turned into the helpful "open it in binary mode" message -- a real, useful check for that real mistake. But None also has no .decode() method, so the identical except AttributeError catches that too, and reports the exact same binary-mode diagnosis for a completely different problem: a transient non-blocking read that was never about text vs. binary mode at all.

3. Contrast: plain BufferedReader.read() on the identical raw stream

raw2 = FlakyRaw(b"irrelevant for this check")
br2 = io.BufferedReader(raw2)
print("plain read():", repr(br2.read()))

plain read(): None

Measured: the io layer preserves the None/EOF distinction correctly; tomllib.load()'s own exception-handling logic is what turns that correct signal into an incorrect diagnosis.

4. The fix: drain into a real buffer first, retrying on None

def drain_retrying_on_none(raw_stream, chunk_size=65536):
    out = io.BytesIO()
    br3 = io.BufferedReader(raw_stream)
    while True:
        chunk = br3.read(chunk_size)
        if chunk is None:
            continue              # retry: not ready yet, not EOF
        if not chunk:
            break                  # real EOF
        out.write(chunk)
    out.seek(0)
    return out

raw3 = FlakyRaw(payload)
safe_buf = drain_retrying_on_none(raw3)
print("safe data:", tomllib.load(safe_buf))

safe data: {'key': 'value'}

Measured: resolving the None/EOF distinction before the bytes reach tomllib.load() parses the document correctly. If you ever see this exact error on a file object you already opened with 'rb', the real cause is almost certainly not your file mode.

Frequently asked questions

Is this documented behavior?

The tomllib documentation for load() says the file object must support binary read() and mentions nothing about None or non-blocking streams. We measured that a None-returning source triggers the same error path tomllib designed for an unrelated, genuine mistake (passing a text-mode file object).

How can I tell the real binary-mode mistake from this one?

We measured that genuinely passing a text-mode file object produces an AttributeError on a str object when .decode() is attempted, with the same resulting TypeError message; there is no way to distinguish the two causes from the exception alone. If the object you passed is verifiably already binary (e.g. you opened it with 'rb' or it's an io.BufferedReader), suspect a non-blocking None instead of your own file mode.

Does this affect tomllib.loads() too?

No -- loads() takes a str directly, not a file object, so there is no read() call involved and this specific failure mode does not apply to it.

Is this related to the other None/EOF findings?

Yes -- same family already confirmed in shutil.copyfileobj, gzip/bz2/lzma, tarfile/zipfile, and ElementTree.parse(): a correctly-returned non-blocking None gets mishandled one layer above io itself. tomllib's version is unusual in that the resulting error actively names the wrong root cause, rather than just crashing generically or blaming the content.

Written with AI assistance. Measured on 2026-10-09 with Python 3.13.5's built-in tomllib.load(), reading a genuinely valid TOML document through an io.BufferedReader (already in binary mode) over a custom non-blocking RawIOBase whose readinto() returns None once before returning the real TOML bytes. PlainHeaders is our extension; Python's tomllib and io modules are built in.