TensorFlow ResourceExhaustedError: OOM when allocating tensor (tested)

tf.errors.ResourceExhaustedError: OOM when allocating tensor with shape[...] and type float on ...device:CPU:0 by allocator cpu means one tensor did not fit in memory. Below: reproduced on TensorFlow 2.21, how to compute the size from the message, and a batched tf.data loop that handled a 64 GB computation in 200 MB steps.

1. Reproduce

b1.py asks TensorFlow for one 400 GB tensor on the CPU:

import tensorflow as tf

try:
    x = tf.zeros([100_000, 100_000, 10])  # 10^11 float32 = 400 GB
except tf.errors.ResourceExhaustedError as e:
    print(type(e).__name__)
    print(str(e)[:300])

ResourceExhaustedError
{{function_node __wrapped__Fill_device_/job:localhost/replica:0/task:0/device:CPU:0}} OOM when allocating tensor with shape[100000,100000,10] and type float on /job:localhost/replica:0/task:0/device:CPU:0 by allocator cpu [Op:Fill] name:

The message tells you the exact shape, dtype and device that failed. Multiply the shape by the dtype size (float32 = 4 bytes) to see how much memory that one tensor needed: here 100,000 × 100,000 × 10 × 4 bytes = 400 GB. Before raising, TensorFlow also logs Allocation of 400000000000 exceeds 10% of free system memory and OP_REQUIRES failed at constant_op.cc:176 : RESOURCE_EXHAUSTED to stderr.

2. Fix: process in batches and keep only what you need

The usual real cause is feeding the whole dataset (or a huge batch) through a layer at once. b2.py runs a Dense(500_000) layer over 32,000 rows: in one call the output alone is 64 GB. Batching it with tf.data (100 rows = 200 MB per step) and keeping only the argmax works:

import time
import numpy as np
import tensorflow as tf

X = np.random.default_rng(0).random((32_000, 4), dtype=np.float32)
layer = tf.keras.layers.Dense(500_000)  # 32_000 x 500_000 float32 output = 64 GB

try:
    layer(X)  # whole dataset in one call
except tf.errors.ResourceExhaustedError as e:
    print("all at once:", type(e).__name__, "-", str(e).split("}}")[-1].split(" on /job")[0].strip())

t = time.time()
best = []
for batch in tf.data.Dataset.from_tensor_slices(X).batch(100):
    logits = layer(batch)                  # 100 x 500_000 = 200 MB
    best.append(tf.argmax(logits, axis=1))  # keep only what you need
best = tf.concat(best, axis=0)
print("batched:", best.shape, best.dtype.name, f"{time.time() - t:.0f}s")

all at once: ResourceExhaustedError - OOM when allocating tensor with shape[32000,500000] and type float
batched: (32000,) int64 63s

• Training: lower batch_size in model.fit until the error stops; halving it halves activation memory.
• Inference: model.predict(x, batch_size=...) batches the forward pass, but still concatenates all outputs, so if the output is the huge part, loop over batches and reduce each one, as above.
• Inputs: stream from disk with tf.data instead of loading a giant NumPy array; use float32 (or float16) instead of float64.
• Model size: a huge Dense after Flatten on big images is a common cause; add pooling or reduce units.
• GPU: the same error appears with device:GPU:0 and allocator GPU_0_bfc; the same batching fixes apply. We only measured on CPU.

Related: tf.function retracing warning, TensorFlow Keras errors after Keras 3.

Written with AI assistance. Measured on 2026-10-24 with tensorflow-cpu 2.21.0, NumPy 2.5.4 and Python 3.13.5 on Linux (b2.py ran under ulimit -v 8 GB so it could not eat the machine). The scripts ran verbatim and the output was pasted unedited (TensorFlow startup log lines filtered). PlainTables is our extension; TensorFlow is free.