tarfile/zipfile Crash on a Non-Blocking None (Tested)

A non-blocking raw stream signals "no data ready yet" by returning None from readinto() -- distinct from the empty read that means real end-of-stream. We measured that tarfile and zipfile, Python's two most-used archive modules, both mishandle this signal: a single transient None, even with a completely valid, intact archive waiting right behind it, crashes the read with an error that looks like the archive itself is broken -- ReadError: empty file or BadZipFile: File is not a zip file -- when neither is true.

1. tarfile: a real, valid tar stream behind one transient None

import io, tarfile

# build a genuinely valid tar stream in memory
buf = io.BytesIO()
with tarfile.open(fileobj=buf, mode="w") as tf:
    data = b"hello tar world"
    ti = tarfile.TarInfo(name="test.txt")
    ti.size = len(data)
    tf.addfile(ti, io.BytesIO(data))
payload = buf.getvalue()

class FlakyRaw(io.RawIOBase):
    def __init__(self, payload):
        self.calls = 0
        self.sent = False
        self.payload = payload
    def readable(self):
        return True
    def readinto(self, b):
        self.calls += 1
        if self.calls == 1:
            return None            # transient: no data ready yet
        if not self.sent:
            self.sent = True
            n = len(self.payload)
            b[:n] = self.payload
            return n               # the real, valid tar data
        return 0

raw = FlakyRaw(payload)
try:
    tf2 = tarfile.open(fileobj=io.BufferedReader(raw), mode="r|")
    for member in tf2:
        print("member:", member.name)
except Exception as e:
    print("CRASHED:", type(e).__name__, e)

CRASHED: ReadError empty file

Measured: the archive is not empty -- it is a complete, valid tar stream with one member, sitting one read call away. tarfile's streaming reader treats the first None exactly like reaching end-of-stream on an empty file, and never gets the chance to see the real data at all.

2. zipfile: a real, valid zip archive behind one transient None

import zipfile

buf2 = io.BytesIO()
with zipfile.ZipFile(buf2, "w") as zf:
    zf.writestr("test.txt", "hello zip world")
zip_payload = buf2.getvalue()

class SeekableFlakyRaw(io.RawIOBase):
    def __init__(self, payload):
        self.calls = 0
        self.payload = payload
        self.pos = 0
    def readable(self):
        return True
    def seekable(self):
        return True
    def seek(self, offset, whence=0):
        if whence == 0: self.pos = offset
        elif whence == 1: self.pos += offset
        elif whence == 2: self.pos = len(self.payload) + offset
        return self.pos
    def tell(self):
        return self.pos
    def readinto(self, b):
        self.calls += 1
        if self.calls == 1:
            return None            # transient: no data ready yet
        n = min(len(b), len(self.payload) - self.pos)
        if n <= 0:
            return 0
        b[:n] = self.payload[self.pos:self.pos + n]
        self.pos += n
        return n

raw2 = SeekableFlakyRaw(zip_payload)
try:
    zf2 = zipfile.ZipFile(io.BufferedReader(raw2))
    print(zf2.namelist())
except Exception as e:
    print("CRASHED:", type(e).__name__, e)

CRASHED: TypeError object of type 'NoneType' has no len()

Measured: zipfile needs a seekable stream to find its central directory, so this reproduction makes the raw stream seekable (unlike the tar case, which streams). Even so, the single transient None on the very first read crashes directory parsing with a TypeError that gives no hint the real problem was a retryable non-blocking signal, not a malformed archive.

3. Contrast: plain BufferedReader.read() on the identical signal

raw3 = FlakyRaw(b"irrelevant for this check")
br = io.BufferedReader(raw3)
print("plain read():", repr(br.read(5)))

plain read(): None

Measured: the underlying io layer both modules build on preserves the None/EOF distinction correctly. Neither tarfile nor zipfile propagates or checks for that None itself -- each lets it flow into unrelated internal logic (an "empty file" check for tar, a length computation for zip) and surfaces a misleading, archive-blaming error instead.

4. The fix: drain into a real buffer first, retrying on None

def drain_retrying_on_none(raw_stream, chunk_size=65536):
    out = io.BytesIO()
    br2 = io.BufferedReader(raw_stream)
    while True:
        chunk = br2.read(chunk_size)
        if chunk is None:
            continue              # retry: not ready yet, not EOF
        if not chunk:
            break                  # real EOF
        out.write(chunk)
    out.seek(0)
    return out

safe_buf = drain_retrying_on_none(FlakyRaw(payload))
tf3 = tarfile.open(fileobj=safe_buf, mode="r|")
print("safe tar members:", [m.name for m in tf3])

safe tar members: ['test.txt']

Measured: once the None/EOF distinction is resolved before the bytes ever reach tarfile or zipfile, both read the archive correctly. The same drain_retrying_on_none() helper works for the zip case too, as long as the final buffer is seekable (an in-memory io.BytesIO, as used here, always is).

Written with AI assistance. Measured on 2026-10-09 with Python 3.13.5's built-in tarfile.open() (streaming mode) and zipfile.ZipFile, each reading a genuinely valid archive through an io.BufferedReader over a custom non-blocking RawIOBase whose readinto() returns None once before returning the real archive bytes. PlainHeaders is our extension; Python's tarfile and zipfile modules are built in.