UnicodeEncodeError: 'charmap' codec can't encode character (tested)

UnicodeEncodeError: 'charmap' codec can't encode character (or 'ascii' codec) means Python turned text into bytes with an encoding that has no byte for one of your characters. We measured where it comes from on Python 3.13.5 and which setting fixes each case.

1. Reading the message

s = "café ✓ 日本"
for enc in ["ascii", "cp1252", "latin-1"]:
    try:
        s.encode(enc)
    except UnicodeEncodeError as e:
        print(f"{enc}: {e}")
print("utf-8:", s.encode("utf-8"))

Output:

ascii: 'ascii' codec can't encode character '\xe9' in position 3: ordinal not in range(128)
cp1252: 'charmap' codec can't encode character '\u2713' in position 5: character maps to <undefined>
latin-1: 'latin-1' codec can't encode character '\u2713' in position 5: ordinal not in range(256)
utf-8: b'caf\xc3\xa9 \xe2\x9c\x93 \xe6\x97\xa5\xe6\x9c\xac'

Measured on Python 3.13.5: the message names the codec that failed, the character, and its position. 'charmap' codec means a Windows code page such as cp1252 (it has é and € but not ✓ or CJK). 'ascii' fails on anything past 127, latin-1 on anything past 255. UTF-8 can encode every character, which is why almost every fix ends with "use UTF-8".

2. Writing a file: pass encoding="utf-8"

s = "price: 5€ ✓"
try:
    with open("out.txt", "w", encoding="cp1252") as f:   # what open() picks on many Windows setups
        f.write(s)
except UnicodeEncodeError as e:
    print("cp1252 file:", e)
with open("out.txt", "w", encoding="utf-8") as f:
    f.write(s)
print("utf-8 file:", open("out.txt", encoding="utf-8").read())

Output:

cp1252 file: 'charmap' codec can't encode character '\u2713' in position 10: character maps to <undefined>
utf-8 file: price: 5€ ✓

Measured: with cp1252 (the default for open() on many Windows machines) the write fails on ✓; with encoding="utf-8" it works. Without an explicit encoding=, open() uses the locale encoding, so the same script can work on Linux/macOS and fail on Windows. Always pass encoding="utf-8" for text files; the same goes for csv files, Path.write_text() and Path.read_text().

3. print(): the terminal or pipe encoding

print("café ✓")

Run under different settings:

$ PYTHONIOENCODING=ascii python3 b3.py
UnicodeEncodeError: 'ascii' codec can't encode character '\xe9' in position 3: ordinal not in range(128)
$ PYTHONIOENCODING=utf-8 python3 b3.py
café ✓
$ PYTHONIOENCODING=ascii PYTHONUTF8=1 python3 b3.py
UnicodeEncodeError: 'ascii' codec can't encode character '\xe9' in position 3: ordinal not in range(128)
$ LC_ALL=C python3 b3.py
café ✓

Measured on Linux: print() encodes with sys.stdout.encoding. Forcing it to ASCII reproduces the error; PYTHONIOENCODING=utf-8 fixes it, and note that PYTHONIOENCODING wins over PYTHONUTF8=1. LC_ALL=C did not fail: since 3.7, Python switches to UTF-8 under the C/POSIX locale. On Windows (not tested here), the console itself has been UTF-8 since 3.6, so the classic 'charmap' codec can't encode from print() usually appears when output is redirected to a file or pipe; set PYTHONUTF8=1 or PYTHONIOENCODING=utf-8, or call sys.stdout.reconfigure(encoding="utf-8") at startup.

4. When you really need ASCII: errors=

s = "café ✓"
print(s.encode("ascii", errors="replace"))
print(s.encode("ascii", errors="ignore"))
print(s.encode("ascii", errors="backslashreplace"))
print(s.encode("ascii", errors="xmlcharrefreplace"))

Output:

b'caf? ?'
b'caf '
b'caf\\xe9 \\u2713'
b'café ✓'

If the target can only take a limited charset, choose what happens to the rest: replace gives ?, ignore silently drops characters (we'd avoid it: café became caf), backslashreplace keeps escapes, xmlcharrefreplace gives HTML entities. open() and sys.stdout.reconfigure() accept the same errors= argument.

5. Logging: "--- Logging error ---" and the program carries on

import logging, io
buf = io.TextIOWrapper(io.BytesIO(), encoding="cp1252")
h = logging.StreamHandler(buf)
log = logging.getLogger("x"); log.addHandler(h)
log.warning("done ✓")          # logging catches it and prints "--- Logging error ---" to stderr
print("program kept running")

Output:

--- Logging error ---
Traceback (most recent call last):
  File "/usr/lib/python3.13/logging/__init__.py", line 1154, in emit
    stream.write(msg + self.terminator)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/encodings/cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeEncodeError: 'charmap' codec can't encode character '\u2713' in position 5: character maps to <undefined>
Call stack:
  File "/tmp/ue/b5.py", line 5, in <module>
    log.warning("done ✓")          # logging catches it and prints "--- Logging error ---" to stderr
Message: 'done ✓'
Arguments: ()
program kept running

Measured: logging handlers catch the exception, print --- Logging error --- with the traceback to stderr, and keep going; the record is lost. For files, use logging.FileHandler(path, encoding="utf-8") (or logging.basicConfig(filename=..., encoding="utf-8"), 3.9+).

Written with AI assistance. Measured on 2026-10-09 with Python 3.13.5 on Linux: encoding to ascii, cp1252 and latin-1, a cp1252 file write, print() under PYTHONIOENCODING, PYTHONUTF8 and LC_ALL=C, the errors= handlers, and a logging handler on a cp1252 stream. Every block was run verbatim and its output pasted from that run. Windows consoles were not tested; Windows behaviour above is from the Python docs. PlainHeaders is our extension; everything else here is built-in Python.