csv.Sniffer() Silently Guesses a Wrong Delimiter (tested)

csv.Sniffer().sniff(text) is meant to auto-detect a CSV's delimiter, but we fed it two sentences of ordinary English prose and it confidently returned the letter m as the delimiter — no exception, no warning, just a wrong answer that looks like a right one if you don't check it.

1. Plain English text, sniffed with no restrictions

import csv

sample = ("This is a sentence, with a comma, but not real csv.\n"
          "Another line, also with commas.\n")
csv.Sniffer().sniff(sample).delimiter
'm'

Measured: this isn't CSV data at all, just two sentences that each happen to contain a comma — yet sniff() returned successfully with 'm' as the detected delimiter, not ',', and not an error. There is nothing in the API that flags this result as low-confidence or wrong.

2. Why 'm': consistency beats plausibility

sample.split("\n")[0].count("m"), sample.split("\n")[1].count("m")
(2, 2)
sample.split("\n")[0].count(","), sample.split("\n")[1].count(",")
(2, 1)

Measured: the letter m appears exactly twice in both lines — a perfectly consistent count — while , appears twice in the first line but only once in the second. Sniffer's heuristic picks whichever candidate character has the most consistent occurrence count across lines, and by default every character in the sample is a candidate, including ordinary letters. A comma that looks obviously right to a human loses to a letter that's merely more consistent, because the algorithm has no concept of which characters are typically delimiters.

3. Restricting candidates with delimiters= changes failure mode entirely

csv.Sniffer().sniff(sample, delimiters=",;\t|")
_csv.Error: Could not determine delimiter

Measured: passing delimiters=",;\t|" to only consider the common CSV delimiter characters doesn't make it fall back to the best among those — it raises a clean exception instead, because none of the allowed characters hit the consistency threshold the algorithm requires. This is the opposite failure mode from the unrestricted call: unrestricted, sniff() never raises and silently returns a wrong answer; restricted, it can raise on exactly the same input instead of guessing. Neither behavior is "detect that this isn't really CSV" — one guesses wrong silently, the other refuses.

4. What to actually do

Always pass an explicit delimiters= argument scoped to the characters you'd actually accept (e.g. ",;\t|") — this won't make Sniffer correct, but it turns a silent wrong guess into a loud, catchable exception when the input isn't recognizable CSV. Beyond that, treat Sniffer's output as a starting guess to validate, not a verified fact: parse a few rows with the detected dialect and check the resulting field count is consistent and sane before trusting it on a whole file, especially for data from an untrusted or unknown source.

Frequently asked questions

Can csv.Sniffer() pick a letter as the delimiter?

Yes — we measured it returning the letter 'm' as the detected delimiter for two sentences of plain English prose, with no error and no low-confidence warning of any kind.

Why would csv.Sniffer() pick a letter over an obvious comma?

We confirmed the cause: Sniffer's heuristic picks whichever character has the most consistent per-line occurrence count, and by default considers every character in the sample a candidate, not just typical delimiter punctuation. In our test, 'm' appeared exactly twice in both lines while ',' appeared twice then once — more consistent wins, regardless of plausibility.

Does csv.Sniffer(delimiters=...) fix the wrong-guess problem?

Partially — we measured that restricting to delimiters=\",;\\t|\" on the same non-CSV input doesn't produce a correct guess; it raises _csv.Error: Could not determine delimiter instead, because none of the allowed characters meet the consistency threshold. It turns a silent wrong answer into a loud, catchable failure, which is the main practical benefit.

Does csv.Sniffer() ever warn that its guess might be wrong?

No — we found no low-confidence signal in the returned Dialect object. The sniff() call either returns a dialect object (right or wrong, with no way to tell which from the return value alone) or raises Could not determine delimiter; there's no middle state.

How should I validate a dialect that Sniffer detected?

We'd parse a handful of rows with it and check that every row produces the same, sane field count before trusting it on a whole file — Sniffer's result is a starting guess, not a verified fact, especially on data from an unknown or untrusted source.

Written with AI assistance. Measured on 2026-10-03 with Python 3.13.5's built-in csv module, calling csv.Sniffer().sniff() on plain-English prose that happens to contain a comma per line, recording the exact delimiter character chosen, and comparing against the same call restricted to a candidate delimiters= set. PlainHeaders is our extension; Python's csv module is built in.