zlib crc32 Per-Chunk Sums Are Wrong (tested)
Computing zlib.crc32() separately on each chunk of a file and adding the results looks like a reasonable way to get a whole-file checksum without holding everything in memory. We measured what that actually produces: a number with no relationship to the real checksum.
1. Adding independent chunk checksums gives the wrong answer
import zlib
data = b"hello world this is a test"
mid = len(data) // 2
c1, c2 = data[:mid], data[mid:]
zlib.crc32(data)
477349405
(zlib.crc32(c1) + zlib.crc32(c2)) & 0xffffffff
3481577407Measured: the summed value isn't close to the real checksum — it's not even a valid CRC-32 of anything. No exception is raised; zlib.crc32() has no way to know the two calls were meant to represent parts of the same stream, so it just returns two independent, unrelated 32-bit values.
2. Passing the running value forward is the correct way
zlib.crc32(c2, zlib.crc32(c1))
477349405Measured: zlib.crc32()'s second argument is the running checksum from everything processed so far, not a value to combine afterward. Chaining it — feeding each chunk's result into the next call as the starting value — reproduces the exact same checksum as computing it over the whole string in one call.
3. Why: CRC-32 isn't a simple additive checksum
Per the Python zlib docs, crc32(data[, value]) explicitly describes value as "the starting value of the checksum," meant to be threaded through successive calls on the pieces of one stream. CRC-32 is computed via polynomial division over the whole bitstream, not a sum over independent pieces, so there's no mathematical shortcut that lets you checksum chunks in isolation and combine the results with ordinary addition afterward.
4. What to actually do
When checksumming a stream in chunks (e.g., validating a large download as it arrives), always pass the previous chunk's result as the value argument to the next zlib.crc32() call, starting from the default of 0 for the first chunk. Never compute crc32() on each chunk independently and add, XOR, or otherwise combine the results after the fact — only the running-value chain produces the correct whole-stream checksum.
Frequently asked questions
Can you add two zlib.crc32() checksums from different chunks to get the checksum of the combined data?
No — we measured the summed value being completely unrelated to the real checksum of the combined data. zlib.crc32() has no way to detect or correct for checksums computed independently.
What is the second argument to zlib.crc32() actually for?
We measured it working as the running checksum value from everything processed so far — passing chunk 1's result as the starting value for chunk 2's crc32() call reproduces the exact whole-data checksum.
Does zlib.crc32() raise an error if you combine chunk checksums incorrectly?
No — we measured it returning a valid-looking 32-bit number either way, with no indication that the chaining was done wrong. There's no exception to catch this mistake.
Why can't CRC-32 checksums just be summed like a simple total?
Based on the Python docs and our test, CRC-32 is computed via polynomial division over the entire bitstream, not an additive process over independent pieces, so there's no shortcut that lets separate chunk checksums be combined by addition afterward.
How do I correctly checksum a large file in chunks with zlib.crc32()?
We'd pass each chunk's result as the value argument to the next crc32() call, starting from 0 (the default) for the first chunk, so the checksum threads through the whole stream in order.
Written with AI assistance. Measured on 2026-10-03 with Python 3.13.5's built-in zlib module: split data into two chunks, computed the whole-data crc32 directly, then compared it against chaining zlib.crc32(chunk2, zlib.crc32(chunk1)) versus naively adding zlib.crc32(chunk1) + zlib.crc32(chunk2). PlainHeaders is our extension; Python's zlib module is built in.