LookupError: unknown encoding (tested)

LookupError: unknown encoding: X means Python has no codec by that name. It's almost always a name from another system (MySQL's utf8mb4, Windows' ansi) arriving via a config, database or header. We measured which names fail on Python 3.13.5 and how to map them.

1. Which names Python knows

for enc in ["utf8", "UTF-8", "utf_8", "latin1", "cp1252", "utf-8-sig", "utf8mb4", "unicode", "ansi", "windows-1252", "x-user-defined"]:
    try:
        "a".encode(enc)
        print(f"{enc!r}: ok")
    except LookupError as e:
        print(f"{enc!r}: LookupError: {e}")

Output:

'utf8': ok
'UTF-8': ok
'utf_8': ok
'latin1': ok
'cp1252': ok
'utf-8-sig': ok
'utf8mb4': LookupError: unknown encoding: utf8mb4
'unicode': LookupError: unknown encoding: unicode
'ansi': LookupError: unknown encoding: ansi
'windows-1252': ok
'x-user-defined': LookupError: unknown encoding: x-user-defined

Measured on Python 3.13.5: spelling and case don't matter for real codecs (utf8, UTF-8, utf_8, windows-1252 all work). The names that fail come from other systems: utf8mb4 (MySQL), ansi (Windows/Notepad), unicode (Excel and editors' "Save as Unicode", which means UTF-16), and x-user-defined (browsers).

2. Map foreign names to Python codecs

import codecs
for enc in ["utf8mb4", "ansi", "unicode"]:
    try:
        open("f.txt", "w", encoding=enc)
    except LookupError as e:
        print("open:", e)
print(codecs.lookup("windows-1252").name, codecs.lookup("UTF8").name, codecs.lookup("latin1").name)
def py_encoding(name, fallback="utf-8"):
    aliases = {"utf8mb4": "utf-8", "utf8mb3": "utf-8", "ansi": "cp1252"}
    name = aliases.get(name.strip().lower(), name)
    try:
        return codecs.lookup(name).name
    except LookupError:
        return fallback
for n in ["utf8mb4", "ANSI", "ISO-8859-1", "bogus"]:
    print(n, "->", py_encoding(n))

Output:

open: unknown encoding: utf8mb4
open: unknown encoding: ansi
open: unknown encoding: unicode
cp1252 utf-8 iso8859-1
utf8mb4 -> utf-8
ANSI -> cp1252
ISO-8859-1 -> iso8859-1
bogus -> utf-8

codecs.lookup() tells you if a name is valid and returns its canonical form. For names coming from a database, header or config file, map the known foreign ones (utf8mb4/utf8mb3 → utf-8, ansi → usually cp1252 on Western-language Windows) and fall back to UTF-8 for anything unknown. Excel's "Unicode Text" is UTF-16 with a BOM, so use "utf-16" for those files.

3. Email with a bogus charset

import email.message
from email.parser import BytesParser
raw = b"Content-Type: text/plain; charset=utf8mb4\r\n\r\ncaf\xc3\xa9"
msg = BytesParser().parsebytes(raw)
cs = msg.get_content_charset()
print("charset:", cs)
try:
    print(msg.get_payload(decode=True).decode(cs))
except LookupError as e:
    print("decode:", e)
from email import policy
msg2 = BytesParser(policy=policy.default).parsebytes(raw)
try:
    msg2.get_content()
except LookupError as e:
    print("policy.default get_content():", e)

Output:

charset: utf8mb4
decode: unknown encoding: utf8mb4
policy.default get_content(): unknown encoding: utf8mb4

Measured: the email module passes the declared charset straight to decode(), so a message saying charset=utf8mb4 raises LookupError with both the legacy API and policy.default's get_content(). Catch LookupError and retry with your mapped name, or decode with "utf-8", errors="replace".

4. requests hides it (and may garble text)

import requests
r = requests.models.Response()
r._content = "café".encode("utf-8")
r.headers["Content-Type"] = "text/html; charset=utf8mb4"
r.encoding = requests.utils.get_encoding_from_headers(r.headers)
print("r.encoding:", r.encoding)
print("r.text:", r.text)               # no LookupError: requests falls back silently
r2 = requests.models.Response()
r2._content = "café".encode("cp1252")
r2.headers["Content-Type"] = "text/html; charset=ansi"
r2.encoding = requests.utils.get_encoding_from_headers(r2.headers)
print("cp1252 bytes, charset=ansi:", repr(r2.text))

Output:

r.encoding: utf8mb4
r.text: café
cp1252 bytes, charset=ansi: 'caf�'

Measured with requests 2.32.3: r.text catches the LookupError and decodes as UTF-8 with replacement characters. That works for utf8mb4 (it is UTF-8) but silently turns cp1252 text labelled ansi into caf�. If text looks wrong, check r.encoding and set it yourself before reading r.text.

Written with AI assistance. Measured on 2026-10-09 with Python 3.13.5 and requests 2.32.3 on Linux: encode() and open() with real and foreign encoding names, codecs.lookup normalisation, an email with charset=utf8mb4 under the legacy and default policies, and requests' r.text with charset=utf8mb4 and charset=ansi. Every block was run verbatim and its output pasted from that run. The meaning of ANSI and Unicode on Windows is from Microsoft's documentation, not tested here. PlainHeaders is our extension; everything else here is built-in Python or requests.