Python requests: get the HTTP error response message (tested)

When an API call fails, the server usually explains why in the response body. But Python's HTTPError message doesn't include it, and requests doesn't even raise by default. We measured both libraries against a local server.

Every block below imports this tiny local test server, which returns a JSON error (422), an HTML error page (502), and a success (200):

# tiny test API on 127.0.0.1:8963, started in a thread by each block
import json, threading
from http.server import BaseHTTPRequestHandler, HTTPServer
class H(BaseHTTPRequestHandler):
    def log_message(self, *a): pass
    def do_GET(self):
        routes = {"/json": (422, "application/json", json.dumps({"error": "email is invalid", "field": "email"})),
                  "/html": (502, "text/html", "<html><body><h1>502 Bad Gateway</h1></body></html>"),
                  "/ok":   (200, "application/json", '{"ok": true}')}
        code, ctype, body = routes[self.path]
        self.send_response(code); self.send_header("Content-Type", ctype); self.end_headers()
        self.wfile.write(body.encode())
srv = HTTPServer(("127.0.0.1", 8963), H)
threading.Thread(target=srv.serve_forever, daemon=True).start()

1. requests doesn't raise on 4xx/5xx, and the exception hides the body

import requests, server
r = requests.get("http://127.0.0.1:8963/json")
print("no exception! status:", r.status_code, "ok:", r.ok)
try:
    r.raise_for_status()
except requests.HTTPError as e:
    print("str(e):", e)

Output:

no exception! status: 422 ok: False
str(e): 422 Client Error: Unprocessable Content for url: http://127.0.0.1:8963/json

Measured with requests 2.32.3: a 422 response raises nothing: requests.get() returns normally and only r.ok is False. raise_for_status() turns it into an HTTPError, but its message has only the status, reason and URL. The server's explanation ("email is invalid") isn't in it. That's why logs often say just "422 Client Error" and nobody knows why.

2. Getting the actual error message: e.response

import requests, server
def get(url):
    r = requests.get(url, timeout=5)
    try:
        r.raise_for_status()
    except requests.HTTPError as e:
        resp = e.response
        try:
            detail = resp.json()
        except requests.JSONDecodeError:
            detail = resp.text[:80]
        raise RuntimeError(f"{resp.status_code} {resp.reason} from {resp.url}: {detail}") from e
    return r.json()
for path in ("/ok", "/json", "/html"):
    try:
        print(get("http://127.0.0.1:8963" + path))
    except RuntimeError as e:
        print(e)

Output:

{'ok': True}
422 Unprocessable Content from http://127.0.0.1:8963/json: {'error': 'email is invalid', 'field': 'email'}
502 Bad Gateway from http://127.0.0.1:8963/html: <html><body><h1>502 Bad Gateway</h1></body></html>

Measured: the HTTPError carries the full response in e.response. Try .json() first for APIs, fall back to .text (truncated) for HTML error pages from proxies and load balancers, and re-raise with status, reason, URL and detail in one line. raise ... from e keeps the original traceback. Catch requests.JSONDecodeError (not json.JSONDecodeError only) so it works whichever JSON library requests is using.

3. urllib: HTTPError is also the response

import urllib.request, urllib.error, server
try:
    urllib.request.urlopen("http://127.0.0.1:8963/json")
except urllib.error.HTTPError as e:
    print("str(e):", e)
    print("code:", e.code, "| reason:", e.reason, "| body:", e.read().decode())

Output:

str(e): HTTP Error 422: Unprocessable Content
code: 422 | reason: Unprocessable Content | body: {"error": "email is invalid", "field": "email"}

Measured: the standard library's urlopen() does raise on 4xx/5xx, and again the message (HTTP Error 422: Unprocessable Content) has no body. The HTTPError object is itself a file-like response: e.code, e.reason, e.headers, and e.read() for the body. You can only read the body once.

Written with AI assistance. Measured on 2026-10-09 with Python 3.13.5 and requests 2.32.3 against a local http.server returning a 422 JSON error, a 502 HTML page and a 200: requests without and with raise_for_status(), e.response.json()/text, and urllib's HTTPError. Every block was run verbatim and its output pasted from that run. PlainHeaders is our extension; requests and urllib are free.