Tabular Explanations

In this demo we will:

• Create a deployment which can be used to classify tabular data

• Create an explainer that will offer insight into why a particular prediction was made for a given input

• Send a prediction request to the deployment

• View the explanation

Register an income classifier model

Register a pre-trained income classifier SKLearn model.

1. In the Model Catalog page, click Register a new model:

   
2. In the Register New Model wizard, enter the following information, then click Register Model:

   • Model Name: income-classifier

   • URI: gs://seldon-models/scv2/samples/mlserver_1.6.0/income-sklearn/classifier/

   • Artifact Type: SciKit Learn

   • Version: v1

   

Configure predictions schema for classifier

Edit the model metadata to update the prediction schema for the model. The prediction schema is a generic schema structure for machine learning model predictions. It is a definition of feature inputs and output targets from the model prediction. Learn more about the predictions schema at the ML Predictions Schema open source repository. Use the income classifier model predictions schema income-classifier-prediction-schema.json to edit and save the model level metadata.

1. Click the model income-classifier that you registered.

   
2. Click Edit Metadata to update the Prediction schema field associated with the model using the contents of prediction schema income-classifier-prediction-schema.json.

   
3. Click Save Metadata.

Launch a Seldon Deployment

Deploy the income classifier model from the catalog into an appropriate namespace

1. In the Model catalog, select Deploy of the Action dropdown.

   
2. Enter the deployment details in the deployment creation wizard and click Next:

   • Name: income-classifier

   • Namespace: seldon

   • Type: Seldon Deployment

   
3. The predictor details should already be filled in from the model catalog except the Model Name field. Use income for the Model Name field.

   • Model Name: income

   The Model Name is linked to the name described in the model-settings.json file, located in the Google Cloud Storage location. Changing the name in the JSON file would also require changing the Model Name, and vice versa.

1. Click Next for the remaining steps, then click Launch.

Make a prediction

1. Click the income-classifier deployment that you created.

2. In the deployment dashboard, click Predict in the left pane.

3. In the Predictpage, click Enter JSON and paste the following request payload:

{
  "inputs": [
    {
      "name": "income",
      "datatype": "INT64",
      "shape": [1, 12],
      "data": [53, 4, 0, 2, 8, 4, 2, 0, 0, 0, 60, 9]
    }
  ]
}

1. Click Predict.

Add an Explainer

The Seldon Enterprise Platform supports several explanation methods for prediction requests. In this demo, you can learn about using these two methods:

• Anchor Explainer

• Kernel SHAP Explainer

The Anchor Explainer can be used to generate explanations for tabular data, text, and image classification models. Whereas, the Kernel SHAP Explainer can be used to generate explanations for only tabular data classification models. You can use the tabular data classification capabilities of these methods because the income-classifier deployment is considered a tabular data classification model.

Add an Anchor Explainer

1. In the income-classifier deployment dashboard, click Add in the MODEL EXPLANATION card.

2. In the Model Data Type tab, select the Tabular model data type option and click Next.

3. In the Explainer Types tab, select the Anchor explainer type option and click Next.

4. In the Explainer URI tab, enter the following details:

   • Explainer URI: gs://seldon-models/scv2/samples/mlserver_1.6.0/income-sklearn/anchor-explainer

   • Explainer Project: default

   • Storage Secret: (leave blank/none)

1. Click Next for the remaining steps, then click Launch.

2. If your explainer is launched successfully, both the deployment and the explainer will show an Available status.

Add a Kernel SHAP Explainer

1. From the income-classifier deployment dashboard, click Add inside the Model Explanation card.

2. In the Model Data Type tab, select the Tabular model data type option and click Next.

3. In the Explainer Types tab, select the KernelShap explainer type option and click Next.

4. In the Explainer URI tab, enter the following details:

   • Explainer URI: gs://seldon-models/scv2/samples/mlserver_1.6.0/income-sklearn/kernel-shap-explainer-sd

   • Explainer Project: default

   • Storage Secret: (leave blank/none)

1. Click Next for the remaining steps, then click Launch.

2. If your explainer is launched successfully, both the deployment and the explainer will show an Available status.

Explain a prediction

Once the explainer of choice is successfully launched, you can now generate explanations for the prediction request made earlier.

1. Navigate to the Requests page using the left navigation drawer.

2. Click on the View explanation button to generate explanations for the request.

An explanation made using the Anchor Explainer

The Anchor explanation reveals which features most contributed to the classification of 'Low Income'. Here, for around 87% of the training data, a 'Relationship' of 'Unmarried' correlates to Low Income. Having a 'Sex' of 'Female' also correlates but is a less strong indicator. At the bottom-most screenshot, we are shown other examples of Low Income data points. This helps to reveal the patterns that the model is relying on to make predictions.

An explanation made using the Kernel SHAP Explainer

The Kernel SHAP explanation, on the other hand, reveals the top 6 contributing features towards the predicted class of 'Low Income'. The features and their corresponding SHAP values shown in the purple bars indicate the most influential features towards the predicted class, whereas the blue bars indicate the features contributing the most towards the opposite class of 'High Income'. Here, the most influential 'positive' features are 'Marital Status' of 'Separated', followed by 'Sex' of 'Female, 'Race' of 'Black, and others. The most influential 'negative' features are 'Occupation' of 'White-Collar', 'Hours per week' of '60', 'Age' of '53', and others.

Congratulations, you've created an explanation for the request! 🥳

Next Steps

Why not try our other demos? Ready to dive in? Read our operations guide to learn more about how to use Enterprise Platform.