# Outlier Detection

In a production environment, monitoring the data used for your machine learning model's inferences is essential, as data changes can significantly impact the performance of the model.

Using Alibi Detect's [VAE outlier detection](https://docs.seldon.ai/alibi-detect/outlier-detection/examples/od_vae_cifar10) method for tabular datasets, this demo helps you to identify outliers in your inference data by:

* Launching an image classifier model trained on the [CIFAR-10 dataset](https://www.cs.toronto.edu/~kriz/cifar.html). The data instances contain 32x32x3 pixels images that are classified into 10 classes such as `truck`, `frog`, `cat`, and others.
* Setting up a VAE outlier detector for this model.
* Sending a request to get an image classification.
* Sending a perturbed request to identify an outlier instance.

## Create a Seldon ML Pipeline

1. In the **Overview** page, click **Create new deployment**.
2. Enter the deployment details as follows:
   * Name: `cifar10-classifier`
   * Namespace: `seldon`
   * Type: `Seldon ML Pipeline`

![create model](/files/cQ7m430BLbgRaRDUUaqf)

3. Configure the default predictor as follows:
   * Runtime: `Tensorflow`
   * Model Project: `default`
   * Model URI: `gs://seldon-models/triton/tf_cifar10`
   * Storage Secret: (leave blank/none)

![default predictor](/files/gcFuIMJ4lzxVLuPwzMKH)

4. Click `Next` for the remaining steps, then click **Launch**.

## Add an Outlier detector

1. In the **Overview** page, select the pipeline that you created.
2. In the **Deployment Dashboard**, click **Add** in the **OUTLIER DETECTION** card.
3. Configure the detector with these parameters:
   * Detector Name: `cifar10-outlier`.
   * Storage URI: `gs://seldon-models/scv2/examples/cifar10/outlier-detector`
   * Reply URL: Leave as the default value.

{% hint style="info" %}
**Note**: If you are using a custom installation, change this parameter according to your installation. `http://seldon-request-logger.seldon-logs`
{% endhint %}

4. Click `Create Detector`. After sometime the status of the detector reads `Available`.

![setup detector](/files/Yl7lI5Gji9qByknzXEUm)

## Make Predictions

Now that the outlier detector is available, you can use of it to identify outliers in the inference data. You send two requests to the model, one with a normal image and another with a perturbed image to identify the outlier.

A frog image from the CIFAR-10 dataset in the Open Inference Protocol (OIP) format:

{% file src="/files/JmmJNWFu9YxtjZuOfZCu" %}

A perturbed image of the same frog in the Open Inference Protocol (OIP) format:

{% file src="/files/x4CRCXMJgi10kHz6wvQg" %}

1. In the deployment dashboard click **Predict** in the left pane.
2. Click **Browse** to upload the `cifar10-frog-oip.json` file.
3. Click **Predict**. The prediction request is processed and the response is displayed.
4. Click **Remove** to remove the uploaded file.
5. Click **Browse** again and upload the `cifar10-frog-perturbed-oip.json` file.
6. Click **Predict** to make a prediction with the perturbed image of the frog.

## View Outliers From Request Logs

Navigate to the **Requests** page in the left pane to view the requests made to the model and their prediction responses. Outlier score are available to the right side of each instance.

![Previously made prediction requests with their prediction responses and outlier scores](/files/6uRLfXZJlWIislJ88xe8)

You can also highlight outliers and filter them by enabling **Highlight Outliers**.

![Highlighted outlier prediction requests with their prediction responses and outlier scores](/files/DkD5xZfE8Z4Kuvy0J0FZ)

### Real-Time Outlier Monitoring

It is important to be able to monitor the outlier detection requests in real-time to ensure that the model is performing as expected and to take corrective actions when necessary.

1. Click **Monitor** in the left pane.
2. Select the **Outlier Detection** tab to view a timeline graph of outlier/inlier requests.

![A timeline graph showing the first request classified as an inlier and the second as an outlier](/files/DiEAyg5ro5dJhYAMyCKs)

## Troubleshooting

If you experience issues with this demo, see the [troubleshooting docs](/seldon-enterprise-platform/help-and-support.md) and also the [Knative](/seldon-enterprise-platform/production-environment/request-logging.md) or [Elasticsearch](/seldon-enterprise-platform/production-environment/elasticsearch.md) sections.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.seldon.ai/seldon-enterprise-platform/demos/seldon-core-v2/outlier-detection.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.