Eli5 compatibility issue with XGBoost

I trained a machine learning model with XGBoost and I needed to interpret model using eli5 to get the local feature contribution. However, when I use the eli5 API eli5.explain_prediction_df(clf, sub_feature_df.iloc[0, :], top=21), it returned the following error:

TypeError: ufunc 'isnan' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''

I looked up online but it seems that no one has had this error before.

1 Like

With some trial and error, I found the solution:

Before training the XGBClassifier, we usually run clf.set_params(**kwargs) to provide the model a hyperparameter set. When you construct this parameter set, if your parameter value is the same as the default value in the XGBClassifier class, please don’t include that parameter in the parameter dictionary. Just pass in the parameter dictionary with parameters that you are going to change, this is highly likely to fix the error.

Just curious, Zeshi, are you using C3’s ML Pipelines, or are you using the python library directly in your Jupyter notebook? I don’t believe we ever expose the eli5.explain_prediction_df() API directly to the end user when using C3’s ML Pipeline interpretability.

This is for both the Jupyter notebook and the ML pipeline. When I developed the ML pipeline for this customer, C3’s ML Pipeline interpretability was not yet compatible with XGBoost. So I implemented a customized Python function for producing the interpretability.

I have not tried if this is the case for C3’s ML Pipeline interpretability. I think it worths to try to see if I have the same error.

I investigated this on C3’s Eli5InterpretTechnique and there is no error. We are all good here.

You can train a PermutationImportance model wrapped on your XGBClassifier model object

        from eli5.sklearn import PermutationImportance
        perm = PermutationImportance(gbm).fit( X, y, scoring=score )
        # Permutation Feature Importance
        feature_importances = eli5.explain_weights_df(perm, feature_names=X.columns.values)
1 Like