gzip Output Is Not Reproducible By Default (tested)

Compressing the exact same bytes twice with Python's gzip module looks like it should produce identical output. We measured it producing a different SHA-256 hash every time, which silently breaks anything that hashes or diffs compressed output — build caches, CDN cache keys, content-addressed storage, reproducible-build checks.

1. gzip.compress() gives a different hash every run

import gzip, hashlib, time

data = b"hello world, this is test content for gzip reproducibility"

out1 = gzip.compress(data)
time.sleep(1.1)
out2 = gzip.compress(data)

out1 == out2
False

hashlib.sha256(out1).hexdigest()[:16]
'69c35134a5ddc5d8'
hashlib.sha256(out2).hexdigest()[:16]
'c431c07d6905fcee'

out1[:10]
b'\x1f\x8b\x08\x00\xd8]\xc1j\x02\xff'
out2[:10]
b'\x1f\x8b\x08\x00\xd9]\xc1j\x02\xff'

Measured: same input, same Python process, same machine — different output every call. Only 4 bytes differ (a little-endian Unix timestamp at offset 4-7 of the gzip header, the MTIME field), but that's enough to change the whole-file hash.

2. gzip.open() to a real file adds a second source of drift: the filename

with gzip.open("/tmp/test1.gz", "wb") as f:
    f.write(data)
time.sleep(1.1)
with gzip.open("/tmp/test2.gz", "wb") as f:
    f.write(data)

open("/tmp/test1.gz","rb").read()[:10]
b'\x1f\x8b\x08\x08\xe0]\xc1j\x02\xff'
open("/tmp/test2.gz","rb").read()[:10]
b'\x1f\x8b\x08\x08\xe1]\xc1j\x02\xff'

# decompressed content is identical either way
gzip.open("/tmp/test1.gz","rb").read() == gzip.open("/tmp/test2.gz","rb").read()
True

Measured: the 4th byte is 0x08 here instead of 0x00 — the FNAME flag bit is set, because gzip.open() on a path embeds that path's basename (with .gz stripped) into the header, on top of the same timestamp drift from part 1. The decompressed content matches byte-for-byte; only the compressed container differs.

3. mtime=0 fixes the timestamp but not the filename

with gzip.GzipFile("/tmp/test3.gz", "wb", mtime=0) as f:
    f.write(data)
with gzip.GzipFile("/tmp/test4.gz", "wb", mtime=0) as f:
    f.write(data)

open("/tmp/test3.gz","rb").read() == open("/tmp/test4.gz","rb").read()
False

open("/tmp/test3.gz","rb").read()[:16]
b'\x1f\x8b\x08\x08\x00\x00\x00\x00\x02\xfftest3\x00'
open("/tmp/test4.gz","rb").read()[:16]
b'\x1f\x8b\x08\x08\x00\x00\x00\x00\x02\xfftest4\x00'

Measured: passing mtime=0 zeroes the timestamp bytes (confirmed identical in both files), but the output is still unequal — the embedded filename (test3\x00 vs test4\x00, null-terminated) still differs because each file was opened by a different path. A partial fix like this looks like it worked (no error, output shrank to only differing in one spot) but still fails any exact byte-comparison or hash check.

4. The actual fix: write through a fileobj and set filename=""

with open("/tmp/test5.gz", "wb") as raw:
    with gzip.GzipFile(fileobj=raw, mode="wb", mtime=0, filename="") as f:
        f.write(data)
with open("/tmp/test6.gz", "wb") as raw:
    with gzip.GzipFile(fileobj=raw, mode="wb", mtime=0, filename="") as f:
        f.write(data)

open("/tmp/test5.gz","rb").read() == open("/tmp/test6.gz","rb").read()
True

Measured: writing through an already-open file object (so there's no path for GzipFile to derive a name from) and explicitly passing both mtime=0 and filename="" produces byte-identical output across separate runs, separate file paths, and separate processes. Per the Python gzip docs, mtime and filename are the only two header fields that vary between otherwise-identical compressions — pin both and the output is fully deterministic. gzip.compress(data, mtime=0) is the simpler equivalent when you don't need a filename-bearing header at all, since compress() never embeds a filename.

Frequently asked questions

Why does gzip.compress() produce different output for the same input in Python?

We measured it embedding the current Unix timestamp (MTIME) into the gzip header by default, so two calls a second apart differ in 4 header bytes even though the compressed data itself is identical.

Does gzip.open() add anything else that changes between runs besides the timestamp?

Yes — we measured gzip.open() on a file path also embedding that path's basename (with .gz stripped) as the FNAME field, so compressing the same content to differently-named files produces different bytes even with the same timestamp.

Does mtime=0 make gzip output fully reproducible?

Not by itself — we measured mtime=0 zeroing the timestamp bytes but the embedded filename still differed between gzip.GzipFile('/tmp/test3.gz', ...) and gzip.GzipFile('/tmp/test4.gz', ...), so the output stayed unequal.

How do I get byte-for-byte reproducible gzip output in Python?

We confirmed writing through fileobj= (an already-open file object, so there's no path to derive a name from) combined with mtime=0 and filename='' produces identical output across separate runs and file paths.

Does non-reproducible gzip output affect the decompressed content?

No — we verified the decompressed bytes were identical in every case. Only the compressed container's header bytes (and therefore its hash) vary; this only matters if something hashes, diffs, or caches the compressed bytes themselves.

Written with AI assistance. Measured on 2026-10-03 with Python 3.13.5's built-in gzip module: compressed the same bytes twice (via gzip.compress(), gzip.open() to a file, and gzip.GzipFile() to a file) one second apart, compared the resulting bytes and SHA-256 hashes. PlainHeaders is our extension; Python's gzip module is built in.