sklearn error_score='raise' and FitFailedWarning (tested)

FitFailedWarning: ... If these failures are not expected, you can try to debug them by setting error_score='raise' means some cross-validation fits failed and were scored nan. Pass error_score='raise' to GridSearchCV or cross_val_score to stop at the first failure and see the original exception.

1. The FitFailedWarning that tells you to set error_score='raise'

b4.py:

import warnings, numpy as np
from sklearn.model_selection import GridSearchCV
from sklearn.linear_model import LogisticRegression
X = np.random.default_rng(0).normal(size=(60, 3))
y = (X[:, 0] > 0).astype(int)
grid = GridSearchCV(LogisticRegression(), {"C": [1.0, -1.0]}, cv=3)  # C must be > 0
with warnings.catch_warnings(record=True) as w:
    warnings.simplefilter("always")
    grid.fit(X, y)
for x in w:
    print(f"{x.category.__name__}: {x.message}")

Output (scikit-learn 1.9.1, Python 3.13.5):

FitFailedWarning: 
3 fits failed out of a total of 6.
The score on these train-test partitions for these parameters will be set to nan.
If these failures are not expected, you can try to debug them by setting error_score='raise'.

Below are more details about the failures:
--------------------------------------------------------------------------------
3 fits failed with the following error:
Traceback (most recent call last):
  File ".../site-packages/sklearn/model_selection/_validation.py", line 855, in _fit_and_score
    estimator.fit(X_train, y_train, **fit_params)
    ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../site-packages/sklearn/base.py", line 1393, in wrapper
    estimator._validate_params()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File ".../site-packages/sklearn/base.py", line 554, in _validate_params
    validate_parameter_constraints(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self._parameter_constraints,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        self.get_params(deep=False),
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        caller_name=self.__class__.__name__,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File ".../site-packages/sklearn/utils/_param_validation.py", line 98, in validate_parameter_constraints
    raise InvalidParameterError(
    ...<2 lines>...
    )
sklearn.utils._param_validation.InvalidParameterError: The 'C' parameter of LogisticRegression must be a float in the range (0.0, inf]. Got -1.0 instead.

UserWarning: One or more of the test scores are non-finite: [0.96666667        nan]

• By default (error_score=np.nan) a failing fit does not stop GridSearchCV: it records nan for that fold, emits one FitFailedWarning that groups identical errors (3 fits failed with the following error), and the search finishes.
• The real cause is the last line of the embedded traceback, here InvalidParameterError: The 'C' parameter ... Got -1.0. The final UserWarning: One or more of the test scores are non-finite is just the consequence.

2. Failures at scoring time look different

b1.py: the fit works but predict fails on the test fold:

import warnings, sklearn, numpy as np
from sklearn.model_selection import GridSearchCV
from sklearn.neighbors import KNeighborsClassifier
print("scikit-learn", sklearn.__version__)
X = np.random.default_rng(0).normal(size=(60, 3))
y = (X[:, 0] > 0).astype(int)
# cv=3 -> each training fold has 40 rows, so n_neighbors=50 cannot work
grid = GridSearchCV(KNeighborsClassifier(), {"n_neighbors": [5, 50]}, cv=3)
with warnings.catch_warnings(record=True) as w:
    warnings.simplefilter("always")
    grid.fit(X, y)
for x in w:   # print first and last line of each warning
    lines = [l for l in str(x.message).splitlines() if l.strip()]
    print(f"{x.category.__name__}: {lines[0]}" + (f"\n    ... {lines[-1]}" if len(lines) > 1 else ""))
print("mean_test_score:", grid.cv_results_["mean_test_score"])
print("best_params_:", grid.best_params_)

scikit-learn 1.9.1
UserWarning: Scoring failed. The score on this train-test partition for these parameters will be set to nan. Details: 
    ... ValueError: Expected n_neighbors <= n_samples_fit, but n_neighbors = 50, n_samples_fit = 40, n_samples = 20
UserWarning: Scoring failed. The score on this train-test partition for these parameters will be set to nan. Details: 
    ... ValueError: Expected n_neighbors <= n_samples_fit, but n_neighbors = 50, n_samples_fit = 40, n_samples = 20
UserWarning: Scoring failed. The score on this train-test partition for these parameters will be set to nan. Details: 
    ... ValueError: Expected n_neighbors <= n_samples_fit, but n_neighbors = 50, n_samples_fit = 40, n_samples = 20
UserWarning: One or more of the test scores are non-finite: [0.91666667        nan]
mean_test_score: [0.91666667        nan]
best_params_: {'n_neighbors': 5}

• When the error happens while scoring, you get one UserWarning: Scoring failed per fold (3 here) instead of a grouped FitFailedWarning; both set the score to nan.
• best_params_ silently ignores the nan candidate and picks n_neighbors=5. If every candidate fails, you get all-nan scores, so check cv_results_["mean_test_score"].

3. error_score='raise': stop at the first failure with a real traceback

b2.py:

import numpy as np
from sklearn.model_selection import GridSearchCV
from sklearn.neighbors import KNeighborsClassifier
X = np.random.default_rng(0).normal(size=(60, 3))
y = (X[:, 0] > 0).astype(int)
grid = GridSearchCV(KNeighborsClassifier(), {"n_neighbors": [5, 50]}, cv=3,
                    error_score="raise")
grid.fit(X, y)

Last 4 lines of the traceback (the full one is 53 lines and ends with the original exception):

    raise ValueError(
    ...<3 lines>...
    )
ValueError: Expected n_neighbors <= n_samples_fit, but n_neighbors = 50, n_samples_fit = 40, n_samples = 20

Compare the three settings with cross_val_score (b3.py):

import warnings, numpy as np
from sklearn.model_selection import cross_val_score
from sklearn.neighbors import KNeighborsClassifier
warnings.simplefilter("ignore")
X = np.random.default_rng(0).normal(size=(60, 3))
y = (X[:, 0] > 0).astype(int)
knn = KNeighborsClassifier(n_neighbors=50)
print("default (nan):", cross_val_score(knn, X, y, cv=3))
print("error_score=0:", cross_val_score(knn, X, y, cv=3, error_score=0))
try:
    cross_val_score(knn, X, y, cv=3, error_score="raise")
except ValueError as ex:
    print("error_score='raise' -> ValueError:", ex)

default (nan): [nan nan nan]
error_score=0: [0 0 0]
error_score='raise' -> ValueError: Expected n_neighbors <= n_samples_fit, but n_neighbors = 50, n_samples_fit = 40, n_samples = 20

• error_score='raise' re-raises the original exception, so you can use a debugger or %debug on it. Use it while developing; switch back to the default for long searches where a few bad combinations are expected.
• error_score=0 replaces failures with 0 instead of nan. That hides the problem and drags the mean down rather than flagging it; prefer nan or 'raise'.
• error_score is accepted by GridSearchCV, RandomizedSearchCV, cross_val_score, cross_validate and validation_curve/learning_curve.

Related: sklearn: suppress warnings.

Written with AI assistance. Measured on 2026-10-23 with scikit-learn 1.9.1, numpy 2.5.4 and Python 3.13.5 on Linux. Scripts ran verbatim and the output was pasted unedited (one long traceback shortened to its last lines, as noted). PlainTables is our extension; scikit-learn is free.