1 of 100

docs-master

Overview

Learn about the features of Alibi Detect

Alibi Detect

Alibi Detect is a source-available Python library focused on outlier, adversarial and drift detection. The package aims to cover both online and offline detectors for tabular data, text, images and time series. Both TensorFlow and PyTorch backends are supported for drift detection.

For more background on the importance of monitoring outliers and distributions in a production setting, check out this talk from the Challenges in Deploying and Monitoring Machine Learning Systems ICML 2020 workshop, based on the paper Monitoring and explainability of models in production and referencing Alibi Detect.

For a thorough introduction to drift detection, check out the talk below titled, . The talk covers what drift is and why it pays to detect it, the different types of drift, how it can be detected in a principled manner and also describes the anatomy of a drift detector.

Getting Started

Installation

Alibi Detect can be installed from PyPI or conda-forge by following the instructions below.

PyPI

Alibi Detect can be installed from with pip. We provide optional dependency buckets for several modules that are large or sometimes tricky to install. Many detectors are supported out of the box with the default install but some detectors require a specific optional dependency installation to use. For instance, the OutlierProphet detector requires the prophet installation. Other detectors have a choice of backend. For instance, the LSDDDrift detector has a choice of tensorflow or pytorch backends. The tabs below list the full set of detector functionality provided by each optional dependency.

Default installation.

The default installation provides out the box support for the following detectors:

conda-forge

To install the conda-forge version it is recommended to use , which can be installed to the baseconda enviroment with:

mamba can then be used to install alibi-detect in a conda enviroment:

Features

is an open source Python library focused on outlier, adversarial and drift detection. The package aims to cover both online and offline detectors for tabular data, text, images and time series. TensorFlow, PyTorch and (where applicable) backends are supported for drift detection. Alibi-Detect does not install these as default. See for more details.

To get a list of respectively the latest outlier, adversarial and drift detection algorithms, you can type:

Summary tables highlighting the practical use cases for all the algorithms can be found .

For detailed information on the outlier detectors:

Similar for adversarial detection:

And data drift:

Basic Usage

We will use the to illustrate the usage of outlier and adversarial detectors in alibi-detect.

First, we import the detector:

Then we initialize it by passing it the necessary arguments:

Some detectors require an additional .fit step using training data:

The detectors can be saved or loaded as described in . Finally, we can make predictions on test data and detect outliers or adversarial examples.

The predictions are returned in a dictionary with as keys meta and data. meta contains the detector's metadata while data is in itself a dictionary with the actual predictions (and other relevant values). It has either is_outlier, is_adversarial or is_drift (filled with 0's and 1's) as well as optional instance_score, feature_score or p_value as keys with numpy arrays as values.

The exact details will vary slightly from method to method, so we encourage the reader to become familiar with the in alibi-detect.

Algorithm Overview

The following tables summarize the advised use cases for the current algorithms. Please consult the method specific pages for a more detailed breakdown of each method. The column Feature Level indicates whether the detection can be done and returned at the feature level, e.g. per pixel for an image.

Outlier Detection

Detector

Tabular

Image

Time Series

Text

Categorical Features

Online

Feature Level

Adversarial Detection

Detector

Tabular

Image

Time Series

Text

Categorical Features

Online

Feature Level

Drift Detection

Detector

Tabular

Image

Time Series

Text

Categorical Features

Online

Feature Level

All drift detectors and built-in preprocessing methods support both PyTorch and TensorFlow backends. The preprocessing steps include randomly initialized encoders, pretrained text embeddings to detect drift on using the library and extraction of hidden layers from machine learning models. The preprocessing steps allow to detect different types of drift such as covariate and predicted distribution shift.

Outlier Detection

Methods

Mahalanobis Distance

source

Mahalanobis Distance

Overview

The Mahalanobis online outlier detector aims to predict anomalies in tabular data. The algorithm calculates an outlier score, which is a measure of distance from the center of the features distribution (). If this outlier score is higher than a user-defined threshold, the observation is flagged as an outlier. The algorithm is online, which means that it starts without knowledge about the distribution of the features and learns as requests arrive. Consequently you should expect the output to be bad at the start and to improve over time. The algorithm is suitable for low to medium dimensional tabular data.

The algorithm is also able to include categorical variables. The fit step first computes pairwise distances between the categories of each categorical variable. The pairwise distances are based on either the model predictions (MVDM method) or the context provided by the other variables in the dataset (ABDM method). For MVDM, we use the difference between the conditional model prediction probabilities of each category. This method is based on the Modified Value Difference Metric (MVDM) by . ABDM stands for Association-Based Distance Metric, a categorical distance measure introduced by . ABDM infers context from the presence of other variables in the data and computes a dissimilarity measure based on the . Both methods can also be combined as ABDM-MVDM. We can then apply multidimensional scaling to project the pairwise distances into Euclidean space.

Usage

Initialize

Parameters:

threshold: Mahalanobis distance threshold above which the instance is flagged as an outlier.
n_components: number of principal components used.
std_clip: feature-wise standard deviation used to clip the observations before updating the mean and covariance matrix.

Initialized outlier detector example:

Fit

We only need to fit the outlier detector if there are categorical variables present in the data. The following parameters can be specified:

X: training batch as a numpy array.
y: model class predictions or ground truth labels for X. Used for 'mvdm' and 'abdm-mvdm' pairwise distance metrics. Not needed for 'abdm'.
d_type: pairwise distance metric used for categorical variables. Currently,

It is often hard to find a good threshold value. If we have a batch of normal and outlier data and we know approximately the percentage of normal data in the batch, we can infer a suitable threshold:

Beware though that the outlier detector is stateful and every call to the score function will update the mean and covariance matrix, even when inferring the threshold.

Detect

We detect outliers by simply calling predict on a batch of instances X to compute the instance level Mahalanobis distances. We can also return the instance level outlier score by setting return_instance_score to True.

The prediction takes the form of a dictionary with meta and data keys. meta contains the detector's metadata while data is also a dictionary which contains the actual predictions stored in the following keys:

is_outlier: boolean whether instances are above the threshold and therefore outlier instances. The array is of shape (batch size,).
instance_score: contains instance level scores if return_instance_score equals True.

Examples

Tabular

Isolation Forest

source

Isolation Forest

Overview

Isolation forests (IF) are tree based models specifically used for outlier detection. The IF isolates observations by randomly selecting a feature and then randomly selecting a split value between the maximum and minimum values of the selected feature. The number of splittings required to isolate a sample is equivalent to the path length from the root node to the terminating node. This path length, averaged over a forest of random trees, is a measure of normality and is used to define an anomaly score. Outliers can typically be isolated quicker, leading to shorter paths. The algorithm is suitable for low to medium dimensional tabular data.

Usage

Initialize

Parameters:

threshold: threshold value for the outlier score above which the instance is flagged as an outlier.
n_estimators: number of base estimators in the ensemble. Defaults to 100.
max_samples: number of samples to draw from the training data to train each base estimator. If int, draw max_samples samples. If float

Initialized outlier detector example:

Fit

We then need to train the outlier detector. The following parameters can be specified:

X: training batch as a numpy array.
sample_weight: array with shape (batch size,) used to assign different weights to each instance during training. Defaults to None.

It is often hard to find a good threshold value. If we have a batch of normal and outlier data and we know approximately the percentage of normal data in the batch, we can infer a suitable threshold:

Detect

We detect outliers by simply calling predict on a batch of instances X to compute the instance level outlier scores. We can also return the instance level outlier score by setting return_instance_score to True.

is_outlier: boolean whether instances are above the threshold and therefore outlier instances. The array is of shape (batch size,).
instance_score: contains instance level scores if return_instance_score equals True.

Examples

Tabular

Auto-Encoder

Overview

The Auto-Encoder (AE) outlier detector is first trained on a batch of unlabeled, but normal (inlier) data. Unsupervised training is desireable since labeled data is often scarce. The AE detector tries to reconstruct the input it receives. If the input data cannot be reconstructed well, the reconstruction error is high and the data can be flagged as an outlier. The reconstruction error is measured as the mean squared error (MSE) between the input and the reconstructed instance.

Variational Auto-Encoding Gaussian Mixture Model

Overview

The Variational Auto-Encoding Gaussian Mixture Model (VAEGMM) Outlier Detector follows the

Auto-Encoding Gaussian Mixture Model

source

Auto-Encoding Gaussian Mixture Model

Overview

The Auto-Encoding Gaussian Mixture Model (AEGMM) Outlier Detector follows the paper. The encoder compresses the data while the reconstructed instances generated by the decoder are used to create additional features based on the reconstruction error between the input and the reconstructions. These features are combined with encodings and fed into a Gaussian Mixture Model (). The AEGMM outlier detector is first trained on a batch of unlabeled, but normal (inlier) data. Unsupervised or semi-supervised training is desirable since labeled data is often scarce. The sample energy of the GMM can then be used to determine whether an instance is an outlier (high sample energy) or not (low sample energy). The algorithm is suitable for tabular and image data.

Usage

Initialize

Parameters:

threshold: threshold value for the sample energy above which the instance is flagged as an outlier.
n_gmm: number of components in the GMM.
encoder_net: tf.keras.Sequential instance containing the encoder network. Example:

decoder_net: tf.keras.Sequential instance containing the decoder network. Example:

gmm_density_net: layers for the GMM network wrapped in a tf.keras.Sequential class. Example:

aegmm: instead of using a separate encoder, decoder and GMM density net, the AEGMM can also be passed as a tf.keras.Model.
recon_features: function to extract features from the reconstructed instance by the decoder. Defaults to a combination of the mean squared reconstruction error and the cosine similarity between the original and reconstructed instances by the AE.
data_type: can specify data type added to metadata. E.g.

Initialized outlier detector example:

Fit

We then need to train the outlier detector. The following parameters can be specified:

X: training batch as a numpy array of preferably normal data.
loss_fn: loss function used for training. Defaults to the custom AEGMM loss which is a combination of the mean squared reconstruction error, the sample energy of the GMM and a loss term penalizing small values on the diagonals of the covariance matrices in the GMM to avoid trivial solutions. It is important to balance the loss weights below so no single loss term dominates during the optimization.
w_energy: weight on sample energy loss term. Defaults to 0.1.

It is often hard to find a good threshold value. If we have a batch of normal and outlier data and we know approximately the percentage of normal data in the batch, we can infer a suitable threshold:

Detect

We detect outliers by simply calling predict on a batch of instances X to compute the instance level sample energies. We can also return the instance level outlier score by setting return_instance_score to True.

is_outlier: boolean whether instances are above the threshold and therefore outlier instances. The array is of shape (batch size,).
instance_score: contains instance level scores if return_instance_score equals True.

Examples

Tabular

Likelihood Ratios for Outlier Detection

Overview

The outlier detector described by

Prophet Detector

Overview

The Prophet outlier detector uses the time series forecasting package explained in

Spectral Residual

source

Spectral Residual

Overview

The Spectral Residual outlier detector is based on the paper Time-Series Anomaly Detection Service at Microsoft and is suitable for unsupervised online anomaly detection in univariate time series data. The algorithm first computes the of the original data. Then it computes the spectral residual of the log amplitude of the transformed signal before applying the Inverse Fourier Transform to map the sequence back from the frequency to the time domain. This sequence is called the saliency map. The anomaly score is then computed as the relative difference between the saliency map values and their moving averages. If the score is above a threshold, the value at a specific timestep is flagged as an outlier. For more details, please check out the .

Usage

Initialize

Parameters:

threshold: Threshold used to classify outliers. Relative saliency map distance from the moving average.
window_amp: Window used for the moving average in the spectral residual computation. The spectral residual is the difference between the log amplitude of the Fourier Transform and a convolution of the log amplitude over window_amp.
window_local: Window used for the moving average in the outlier score computation. The outlier score computes the relative difference between the saliency map and a moving average of the saliency map over

Initialized outlier detector example:

It is often hard to find a good threshold value. If we have a time series containing both normal and outlier data and we know approximately the percentage of normal data in the time series, we can infer a suitable threshold:

Detect

We detect outliers by simply calling predict on a time series X to compute the outlier scores and flag the anomalies. We can also return the instance (timestep) level outlier score by setting return_instance_score to True.

is_outlier: boolean whether instances are above the threshold and therefore outlier instances. The array is of shape (timesteps,).
instance_score: contains instance level scores if return_instance_score equals True.

Examples

Sequence-to-Sequence (Seq2Seq)

source

Sequence-to-Sequence (Seq2Seq)

Overview

The Sequence-to-Sequence (Seq2Seq) outlier detector consists of 2 main building blocks: an encoder and a decoder. The encoder consists of a which processes the input sequence and initializes the decoder. The LSTM decoder then makes sequential predictions for the output sequence. In our case, the decoder aims to reconstruct the input sequence. If the input data cannot be reconstructed well, the reconstruction error is high and the data can be flagged as an outlier. The reconstruction error is measured as the mean squared error (MSE) between the input and the reconstructed instance.

Since even for normal data the reconstruction error can be state-dependent, we add an outlier threshold estimator network to the Seq2Seq model. This network takes in the hidden state of the decoder at each timestep and predicts the estimated reconstruction error for normal data. As a result, the outlier threshold is not static and becomes a function of the model state. This is similar to , but while they train the threshold estimator separately from the Seq2Seq model with a Support-Vector Regressor, we train a neural net regression network end-to-end with the Seq2Seq model.

The detector is first trained on a batch of unlabeled, but normal (inlier) data. Unsupervised training is desireable since labeled data is often scarce. The Seq2Seq outlier detector is suitable for both univariate and multivariate time series.

Usage

Initialize

Parameters:

n_features: number of features in the time series.
seq_len: sequence length fed into the Seq2Seq model.
threshold: threshold used for outlier detection. Can be a float or feature-wise array.

latent_dim: latent dimension of the encoder and decoder.
output_activation: activation used in the Dense output layer of the decoder.
beta: weight on the threshold estimation mean-squared error (MSE) loss term.

Initialized outlier detector example:

Fit

We then need to train the outlier detector. The following parameters can be specified:

X: univariate or multivariate time series array with preferably normal data used for training. Shape equals (batch, n_features) or (batch, seq_len, n_features).
loss_fn: loss function used for training. Defaults to the MSE loss.
optimizer: optimizer used for training. Defaults to with learning rate 1e-3.

It is often hard to find a good threshold value. If we have a batch of normal and outlier data and we know approximately the percentage of normal data in the batch, we can infer a suitable threshold. We can either set the threshold over both features combined or determine a feature-wise threshold. Here we opt for the feature-wise threshold. This is for instance useful when different features have different variance or sensitivity to outliers. The snippet assumes there are about 5% outliers in the first feature and 10% in the second:

Detect

We detect outliers by simply calling predict on a batch of instances X. Detection can be customized via the following parameters:

outlier_type: either 'instance' or 'feature'. If the outlier type equals 'instance', the outlier score at the instance level will be used to classify the instance as an outlier or not. If 'feature' is selected, outlier detection happens at the feature level. It is important to distinguish 2 use cases:
- X has shape (batch, n_features):

is_outlier: boolean whether instances or features are above the threshold and therefore outliers. If outlier_type equals 'instance', then the array is of shape (batch,). If it equals 'feature', then the array is of shape (batch, seq_len, n_features) or (batch, n_features), depending on the shape of X.
feature_score: contains feature level scores if return_feature_score equals True.

Examples

Isolation Forest outlier detection on KDD Cup ‘99 dataset

Method

Dataset

The outlier detector needs to detect computer network intrusions using TCP dump data for a local-area network (LAN) simulating a typical U.S. Air Force LAN. A connection is a sequence of TCP packets starting and ending at some well defined times, between which data flows to and from a source IP address to a target IP address under some well defined protocol. Each connection is labeled as either normal, or as an attack.

There are 4 types of attacks in the dataset:

DOS: denial-of-service, e.g. syn flood;
R2L: unauthorized access from a remote machine, e.g. guessing password;
U2R: unauthorized access to local superuser (root) privileges;
probing: surveillance and other probing, e.g., port scanning.

The dataset contains about 5 million connection records.

There are 3 types of features:

basic features of individual connections, e.g. duration of connection
content features within a connection, e.g. number of failed log in attempts
traffic features within a 2 second window, e.g. number of connections to the same host as the current connection

This notebook requires the seaborn package for visualization which can be installed via pip:

Load dataset

We only keep a number of continuous (18 out of 41) features.

Assume that a model is trained on normal instances of the dataset (not outliers) and standardization is applied:

Apply standardization:

Define outlier detector

We train an outlier detector from scratch:

The warning tells us we still need to set the outlier threshold. This can be done with the infer_threshold method. We need to pass a batch of instances and specify what percentage of those we consider to be normal via threshold_perc. Let's assume we have some data which we know contains around 5% outliers. The percentage of outliers can be set with perc_outlier in the create_outlier_batch function.

Let's save the outlier detector with updated threshold:

Detect outliers

We now generate a batch of data with 10% outliers and detect the outliers in the batch.

Predict outliers:

Display results

F1 score and confusion matrix:

Plot instance level outlier scores vs. the outlier threshold:

We can see that the isolation forest does not do a good job at detecting 1 type of outliers with an outlier score around 0. This makes inferring a good threshold without explicit knowledge about the outliers hard. Setting the threshold just below 0 would lead to significantly better detector performance for the outliers in the dataset. This is also reflected by the ROC curve:

Time series outlier detection with Spectral Residuals on synthetic data

Method

The Spectral Residual outlier detector is based on the paper and is suitable for unsupervised online anomaly detection in univariate time series data. The algorithm first computes the of the original data. Then it computes the spectral residual of the log amplitude of the transformed signal before applying the Inverse Fourier Transform to map the sequence back from the frequency to the time domain. This sequence is called the saliency map. The anomaly score is then computed as the relative difference between the saliency map values and their moving averages. If this score is above a threshold, the value at a specific timestep is flagged as an outlier. For more details, please check out the .

Methods

Offline

Chi-Squared

Overview

The drift detector applies feature-wise tests for the categorical features. For multivariate data, the obtained p-values for each feature are aggregated either via the

Kolmogorov-Smirnov

Overview

The drift detector applies feature-wise two-sample (K-S) tests. For multivariate data, the obtained p-values for each feature are aggregated either via the

Cramér-von Mises

Overview

The CVM drift detector is a non-parametric drift detector, which applies feature-wise two-sample

Fisher’s Exact Test

source

Fisher's Exact Test

Overview

The FET drift detector is a non-parametric drift detector. It applies Fisher's Exact Test (FET) to each feature, and is intended for application to , with binary univariate data consisting of either (True, False) or (0, 1). This detector is ideal for use in a supervised setting, monitoring drift in a model's instance level accuracy (i.e. correct prediction = 0, and incorrect prediction = 1).

The detector is primarily intended for univariate data, but can also be used in a multivariate setting. For multivariate data, the obtained p-values for each feature are aggregated either via the or the (FDR) correction. The Bonferroni correction is more conservative and controls for the probability of at least one false positive. The FDR correction on the other hand allows for an expected fraction of false positives to occur. As with other univariate detectors such as the detector, for high-dimensional data, we typically want to reduce the dimensionality before computing the feature-wise univariate FET tests and aggregating those via the chosen correction method. See for more guidance on this.

For the $j^{th}$ feature, the FET detector considers the 2x2 contingency table between the reference data $x_j^{ref}$ and test data $x_j$ for that feature:

True (1)

False (0)

where $N^{ref}_1$ represents the number of 1's in the reference data (for the $j^{th}$ feature), $N^{ref}_0$ the number of 0's, and so on. These values can be used to define an odds ratio:

The null hypothesis is $H_0: \widehat{OR}=1$. In other words, the proportion of 1's to 0's is unchanged between the test and reference distributions, such that the odds of 1's vs 0's is independent of whether the data is drawn from the reference or test distribution. The offline FET detector can perform one-sided or two-sided tests, with the alternative hypothesis set by the alternative keyword argument:

If alternative='greater', the alternative hypothesis is $H_a: \widehat{OR}>1$ i.e. proportion of 1's versus 0's has increased compared to the reference distribution.
If alternative='less', the alternative hypothesis is $H_a: \widehat{OR}<1$ i.e. the proportion of 1's versus 0's has decreased compared to the reference distribution.
If alternative='two-sided', the alternative hypothesis is $H_a: \widehat{OR} \ne 1$ i.e. the proportion of 1's versus 0's has changed compared to the reference distribution.

The p-value returned by the detector is then the probability of obtaining an odds ratio at least as extreme as that observed (in the direction specified by alternative), assuming the null hypothesis is true.

Usage

Initialize

Arguments:

x_ref: Data used as reference distribution. Note this should be the raw data, for example np.array([0, 0, 1, 0, 0, 0]), not the 2x2 contingency table.

Keyword arguments:

p_val: p-value used for significance of the FET test. If the FDR correction method is used, this corresponds to the acceptable q-value.
preprocess_at_init: Whether to already apply the (optional) preprocessing step to the reference data at initialization and store the preprocessed data. Dependent on the preprocessing step, this can reduce the computation time for the predict step significantly, especially when the reference dataset is large. Defaults to True. It is possible that it needs to be set to False if the preprocessing step requires statistics from both the reference and test data, such as the mean or standard deviation.
x_ref_preprocessed

Initialized drift detector example:

Detect Drift

We detect data drift by simply calling predict on a batch of instances x. We can return the feature-wise p-values before the multivariate correction by setting return_p_val to True. The drift can also be detected at the feature level by setting drift_type to 'feature'. No multivariate correction will take place since we return the output of n_features univariate tests. For drift detection on all the features combined with the correction, use 'batch'. return_p_val equal to True will also return the threshold used by the detector (either for the univariate case or after the multivariate correction).

is_drift: 1 if the sample tested has drifted from the reference data and 0 otherwise.
p_val: contains feature-level p-values if return_p_val equals True.
threshold: for feature-level drift detection the threshold equals the p-value used for the significance of the FET test. Otherwise the threshold after the multivariate correction (either

Examples

Mixed-type tabular data

source

Mixed-type tabular data

Overview

The drift detector applies feature-wise two-sample Kolmogorov-Smirnov (K-S) tests for the continuous numerical features and tests for the categorical features. For multivariate data, the obtained p-values for each feature are aggregated either via the or the (FDR) correction. The Bonferroni correction is more conservative and controls for the probability of at least one false positive. The FDR correction on the other hand allows for an expected fraction of false positives to occur. Similarly to the other drift detectors, a preprocessing steps could be applied, but the output features need to be categorical.

Usage

Initialize

Arguments:

x_ref: Data used as reference distribution.

Keyword arguments:

p_val: p-value used for significance of the K-S and Chi-Squared test across all features. If the FDR correction method is used, this corresponds to the acceptable q-value.
categories_per_feature: Dictionary with as keys the column indices of the categorical features and optionally as values the number of possible categorical values for that feature or a list with the possible values. If you know which features are categorical and simply want to infer the possible values of the categorical feature from the reference data you can pass a Dict[int, NoneType] such as {0: None, 3: None} if features 0 and 3 are categorical. If you also know how many categories are present for a given feature you could pass this in the categories_per_feature dict in the Dict[int, int] format, e.g. {0: 3, 3: 2}. If you pass N categories this will assume the possible values for the feature are [0, ..., N-1]. You can also explicitly pass the possible categories in the Dict[int, List[int]]

Initialized drift detector example:

Detect Drift

is_drift: 1 if the sample tested has drifted from the reference data and 0 otherwise.
p_val: contains feature-level p-values if return_p_val equals True.
threshold: for feature-level drift detection the threshold equals the p-value used for the significance of the K-S and Chi-Squared tests. Otherwise the threshold after the multivariate correction (either

Examples

Online

Online Fisher’s Exact Test

Online Fisher's Exact Test

Overview

The online detector is a non-parametric method for online drift detection. Like the

Examples

Adversarial Detection

Methods

Model Distillation

Model distillation

Overview

is a technique that is used to transfer knowledge from a large network to a smaller network. Typically, it consists of training a second model with a simplified architecture on soft targets (the output distributions or the logits) obtained from the original model.

Examples

Datasets

source

Overview

The package also contains functionality in alibi_detect.datasets to easily fetch a number of datasets for different modalities. For each dataset either the data and labels or a Bunch object with the data, labels and optional metadata are returned. Example:

Sequential Data and Time Series

Genome Dataset: fetch_genome

Bacteria genomics dataset for out-of-distribution detection, released as part of . From the original TL;DR: The dataset contains genomic sequences of 250 base pairs from 10 in-distribution bacteria classes for training, 60 OOD bacteria classes for validation, and another 60 different OOD bacteria classes for test. There are respectively 1, 7 and again 7 million sequences in the training, validation and test sets. For detailed info on the dataset check the .

ECG 5000: fetch_ecg

5000 ECG's, originally obtained from .

NAB: fetch_nab

Any univariate time series in a DataFrame from the . A list with the available time series can be retrieved using alibi_detect.datasets.get_list_nab().

Images

CIFAR-10-C: fetch_cifar10c

CIFAR-10-C () contains the test set of CIFAR-10, but corrupted and perturbed by various types of noise, blur, brightness etc. at different levels of severity, leading to a gradual decline in a classification model's performance trained on CIFAR-10. fetch_cifar10c allows you to pick any severity level or corruption type. The list with available corruption types can be retrieved with alibi_detect.datasets.corruption_types_cifar10c(). The dataset can be used in research on robustness and drift. The original data can be found . Example:

Adversarial CIFAR-10: fetch_attack

Load adversarial instances on a ResNet-56 classifier trained on CIFAR-10. Available attacks: ('cw') and ('slide'). Example:

Tabular

KDD Cup '99: fetch_kdd

Dataset with different types of computer network intrusions. fetch_kdd allows you to select a subset of network intrusions as targets or pick only specified features. The original data can be found .

Models

Models and/or building blocks that can be useful outside of outlier, adversarial or drift detection can be found under alibi_detect.models. Main implementations:

PixelCNN++: from alibi_detect.models.tensorflow import PixelCNN
Variational Autoencoder: from alibi_detect.models.tensorflow import VAE
Sequence-to-sequence model: from alibi_detect.models.tensorflow import Seq2Seq
ResNet: from alibi_detect.models.tensorflow import resnet

Pre-trained TensorFlow ResNet-20/32/44 models on CIFAR-10 can be found on our and can be fetched as follows:

Bibliography

[RA12] Gordon J. Ross and Niall M. Adams. Two nonparametric control charts for detecting arbitrary distribution changes. Journal of Quality Technology, 44(2):102–116, 2012. doi:10.1080/00224065.2012.11917887.
[RTA12] Gordon J. Ross, Dimitris K. Tasoulis, and Niall M. Adams. Sequential monitoring of a bernoulli sequence when the pre-change parameter is unknown. Computational Statistics, 28(2):463–479, March 2012. arXiv:1212.6020, doi:10.1007/s00180-012-0311-7.
[HHG20] Allison Marie Horst, Alison Presmanes Hill, and Kristen B Gorman. palmerpenguins: Palmer Archipelago (Antarctica) penguin data. 2020. R package version 0.1.0. doi:10.5281/zenodo.3960218.
[GBR+12] Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Schölkopf, and Alexander Smola. A kernel two-sample test. J. Mach. Learn. Res., 13(1):723–773, mar 2012. URL: https://dl.acm.org/doi/10.5555/2188385.2188410.

API Reference

alibi_detect.ad

alibi_detect.cd

alibi_detect.cd.keops

alibi_detect.cd.ks

`KSDrift`

Inherits from: BaseUnivariateDrift, BaseDetector, ABC, DriftConfigMixin

Detector Configuration Files

For advanced use cases, Alibi Detect features powerful configuration file based functionality. As shown below, Drift detectors can be specified with a configuration file named config.toml (adversarial and outlier detectors coming soon!), which can then be passed to {func}~alibi_detect.saving.load_detector:

import numpy as np
from alibi_detect.cd import MMDDrift

x_ref = np.load('detector_directory/x_ref.npy')
detector = MMDDrift(x_ref, p_val=0.05)


name = "MMDDrift"
x_ref = "x_ref.npy"
p_val = 0.05

from alibi_detect.saving import load_detector
filepath = 'detector_directory/'
detector = load_detector(filepath)

Compared to standard instantiation, config-driven instantiation has a number of advantages:

Human readable: The config.toml files are human-readable (and editable!), providing a readily accessible record of previously created detectors.
Flexible artefact specification: Artefacts such as datasets and models can be specified as locally serialized objects, or as runtime registered objects (see ). Multiple detectors can share the same artefacts, and they can be easily swapped.
Inbuilt validation: The {func}~alibi_detect.saving.load_detector function uses to validate detector configurations.

To get a general idea of the expected layout of a config file, see the . Alternatively, to obtain a fully populated config file for reference, users can run one of the and generate a config file by passing an instantiated detector to {func}~alibi_detect.saving.save_detector.

Configuration file layout

All detector configuration files follow a consistent layout, simplifying the process of writing simple config files by hand. For example, a {class}~alibi_detect.cd.KSDrift detector with a serialized function to preprocess reference and test data can be specified as:

The name field should always be the name of the detector, for example KSDrift or SpotTheDiffDrift. The remaining fields are the args/kwargs to pass to the detector (see the {mod}alibi_detect.cd docs for a full list of permissible args/kwargs for each detector). All config fields follow this convention, however as discussed in , some fields can be more complex than others.

Note:In the above example config.toml, x_ref and preprocess_fn are stored in detector_directory/, but this directory isn't included in the config file. This is because in the config file, relative directories are relative to the location of the config.toml file. Filepaths may be absolute, or include nested directories, but must be POSIX paths i.e. use / path separators instead of \.

Note:In the above example config.toml, x_ref and preprocess_fn are stored in detector_directory/, but this directory isn't incSometimes, fields representing kwargs need to be set to None. However, unspecified fields are set to a detector's default kwargs (or for , the defaults shown in the tables). To set fields as None, specify them as the string "None".

Specifying artefacts

When specifying a detector via a config.toml file, the locally stored reference data x_ref must be specified. In addition, many detectors also require (or allow) additional artefacts, such as kernels, functions and models. Depending on their type, artefacts can be specified in config.toml in a number of ways:

Local files: Simple functions and/or models can be specified as locally stored files, whilst data arrays are specified as locally stored numpy files.
Function/object registry: As discussed in , functions and other objects defined at runtime can be registered using {func}alibi_detect.saving.registry, allowing them to be specified in the config file without having to serialise them. For convenience a number of Alibi Detect functions such as {func}~alibi_detect.cd.tensorflow.preprocess.preprocess_drift are also pre-registered.
Dictionaries

The following table shows the allowable formats for all possible config file artefacts.

All Artefacts Table

Artefact dictionaries

Simple artefacts, for example a simple preprocessing function serialized in a dill file, can be specified directly: preprocess_fn = "function.dill". However, if more complex, they can be specified as an artefact dictionary:

config.toml (excerpt)

Here, the preprocess_fn field is a {class}~alibi_detect.saving.schemas.PreprocessConfig artefact dictionary. In this example, specifying the preprocess_fn function as a dictionary allows us to specify additional kwarg's to be passed to the function upon loading. This example also demonstrates the flexibility of the TOML format, with dictionaries able to be specified with {} brackets or by sections demarcated with [] brackets (see the for more details on the TOML format).

Other config fields in the {ref}all-artefacts-table table can be specified via artefact dictionaries in a similar way. For example, the model and proj fields can be set as TensorFlow or PyTorch models via the {class}~alibi_detect.saving.schemas.ModelConfig dictionary. Often an artefact dictionary may itself contain nested artefact dictionaries, as is the case in in the following example, where a preprocess_fn is specified with a TensorFlow model.

config.toml (excerpt)

Each artefact dictionary has an associated pydantic model which is used for . The for these pydantic models provides a description of the permissible fields for each artefact dictionary. For examples of how the artefact dictionaries can be used in practice, see {ref}examples.

Registering artefacts

Custom artefacts defined in Python code may be specified in the config file without the need to serialise them, by first adding them to the Alibi Detect artefact registry using the {mod}alibi_detect.saving.registry submodule. This submodule harnesses the library to allow functions to be registered with a decorator syntax:

Once the custom function has been registered, it can be specified in config.toml files via its reference string (with @ prepended), for example "@my_function.v1" in this case. Other objects, such as custom tensorflow or pytorch models, can also be registered by using the register function directly. For example, to register a tensorflow encoder model:

Examining registered artefacts

A registered object's metadata can be obtained with registry.find(), and all currently registered objects can be listed with registry.get_all(). For example, registry.find("my_function.v1") returns the following:

Pre-registered utility functions/objects

For convenience, Alibi Detect also pre-registers a number of commonly used utility functions and objects.

Function/Class

Registry reference*

Tensorflow

Pytorch

*For backend-specific functions/classes, [backend] should be replaced the desired backend e.g. tensorflow or pytorch.

These can be used in config.toml files. Of particular importance are the preprocess_drift utility functions, which allows models, tokenizers and embeddings to be easily specified for preprocessing, as demonstrated in the .

(examples)=

Example config files

Drift detection on text data

This example presents a configuration for the {class}~alibi_detect.cd.MMDDrift detector used in . The detector will pass the input text data through a preprocess_fn step consisting of a tokenizer, embedding and model. An model is included in order to reduce the dimensionality of the embedding space, which consists of a 768-dimensional vector for each instance. The config.toml is:

Validating config files

When {func}~alibi_detect.saving.load_detector is called, the {func}~alibi_detect.saving.validate_config utility function is used internally to validate the given detector configuration. This allows any problems with the configuration to be detected prior to sometimes time-consuming operations of loading artefacts and instantiating the detector. {func}~alibi_detect.saving.validate_config can also be used by devs working with Alibi Detect config dictionaries.

Under-the-hood, {func}~alibi_detect.saving.load_detector parses the config.toml file into a unresolved config dictionary. It then passes this dict through {func}~alibi_detect.saving.validate_config, to check for errors such as incorrectly named fields, and incorrect types. If working directly with config dictionaries, the same process can be done explicitly, for example:

This will return a ValidationError because p_val is expected to be float not a list, and bad_field isn't a recognised field for the MMDDrift detector:

Validating at this stage is useful as errors can be caught before the sometimes time-consuming operation of resolving the config dictionary, which involves loading each artefact in the dictionary ({func}~alibi_detect.saving.read_config and {func}~alibi_detect.saving.resolve_config can be used to manually read and resolve a config for debugging). The resolved config dictionary is then also passed through {func}~alibi_detect.saving.validate_config, and this second validation can also be done explicitly:

Note that since resolved=True, {func}~alibi_detect.saving.validate_config is now expecting x_ref to be a Numpy ndarray instead of a string. This second level of validation can be useful as it helps detect problems with loaded artefacts before attempting the sometimes time-consuming operation of instantiating the detector.

Context-aware drift detection on ECGs

Introduction

In this notebook we show how to detect drift on ECG data given a specific context using the context-aware MMD detector (Cobb and Van Looveren, 2022). Consider the following simple example: we have a heatbeat monitoring system which is trained on a wide variety of heartbeats sampled from people of all ages across a variety of activities (e.g. rest or running). Then we deploy the system to monitor individual people during certain activities. The distribution of the heartbeats monitored during deployment will then be drifting against the reference data which resembles the full training distribution, simply because only individual people in a specific setting are being tracked. However, this does not mean that the system is not working and requires re-training. We are instead interested in flagging drift given the relevant context such as the person's characteristics (e.g. age or medical history) and the activity. Traditional drift detectors cannot flexibly deal with this setting since they rely on the i.i.d. assumption when sampling the reference and test sets. The context-aware detector however allows us to pass this context to the detector and flag drift appropriately. More generally, the context-aware drift detector detects changes in the data distribution which cannot be attributed to a permissible change in the context variable. On top of that, the detector allows you to understand which subpopulations are present in both the reference and test data which provides deeper insights into the distribution underlying the test data.

Useful context (or conditioning) variables for the context-aware drift detector include but are not limited to:

Domain or application specific contexts such as the time of day or the activity (e.g. running or resting).
Conditioning on the relative prevalences of known subpopulations, such as the frequency of different types of heartbeats. It is important to note that while the relative frequency of each subpopulation (e.g. the different heartbeat types) might change, the distribution underlying each individual subpopulation (e.g. each specific type of heartbeat) cannot change.
Conditioning on model predictions. Assume we trained a classifier which detects arrhythmia, then we can provide the classifier model predictions as context and understand if, given the model prediction, the data comes from the same underlying distribution as the reference data or not.

The following settings will be showcased throughout the notebook:

A change in the prevalences of subpopulations (i.e. different types of heartbeats as determined by an unsupervised clustering model or an ECG classifier) which are also present in the reference data is observed. Contrary to traditional drift detection approaches, the context-aware detector does not flag drift as this change in frequency of various heartbeats is permissible given the context provided.
A change in the underlying distribution underlying one or more subpopulations takes place. While we allow changes in the prevalences of the subpopulations accounted for by the context variable, we do not allow changes of the subpopulations themselves. If for instance the ECGs are corrupted by noise on the sensor measurements, we want to flag drift.

We also show how to condition the detector on different context variables such as the ECG classifier model predictions, cluster membership by an unsupervised clustering algorithm and timestamps.

Under setting 1. we want our detector to be well-calibrated (a controlled False Positive Rate (FPR) and more generally a p-value which is uniformly distributed between 0 and 1) while under setting 2. we want our detector to be powerful and flag drift. Lastly, we show how the detector can help you to understand the connection between the reference and test data distributions better.

Data

The dataset contains 5000 ECG’s, originally obtained from Physionet from the , record chf07. The data has been pre-processed in 2 steps: first each heartbeat is extracted, and then each beat is made equal length via interpolation. The data is labeled and contains 5 classes. The first class $N$ which contains almost 60% of the observations is seen as normal while the others are supraventricular ectopic beats ($S$), ventricular ectopic beats ($V$), fusion beats ($F$) and unknown beats ($Q$).

Requirements

The notebook requires the torch and statsmodels packages to be installed, which can be done via pip:

Before we start let's fix the random seeds for reproducibility:

Load data

First we load the data, show the distribution across the ECG classes and visualise some ECGs from each class.

We can see that most heartbeats can be classified as normal, followed by the unknown class. We will now sample 500 heartbeats to train a simple ECG classifier. Importantly, we leave out the $F$ and $V$ classes which are used to detect drift. First we define a helper function to sample data.

We use a prop_train fraction of all samples to train the classifier and then remove instances from the $F$ and $V$ classes. The rest of the data is used by our drift detectors.

Train an ECG classifier

Now we define and train our classifier on the training set.

Let's evaluate out classifier on both the training and drift portions of the datasets.

Detector calibration under no change

We start with an example where no drift occurs and the reference and test data are both sampled randomly from all classes present in the reference data (classes 0, 1 and 3). Under this scenario, we expect no drift to be detected by either a normal MMD detector or by the context-aware MMD detector.

Before we can start using the context-aware drift detector, first we need to define our context variable. In our experiments we allow the relative prevalences of subpopulations (i.e. the relative frequency of different types of hearbeats also present in the reference data) to vary while the distributions underlying each of the subpopulations remain unchanged. To achieve this we condition on the prediction probabilities of the classifier we trained earlier to distinguish the different types of ECGs. We can do this because the prediction probabilities can account for the frequency of occurrence of each of the heartbeat types (be it imperfectly given our classifier makes the occasional mistake).

The below figure of the of a random sample from the uniform distribution U[0,1] against the obtained p-values from the vanilla and context-aware MMD detectors illustrate how well both detectors are calibrated. A perfectly calibrated detector should have a Q-Q plot which closely follows the diagonal. Only the middle plot in the grid shows the detector's p-values. The other plots correspond to n_runs p-values actually sampled from U[0,1] to contextualise how well the central plot follows the diagonal given the limited number of samples.

As expected we can see that both the normal MMD and the context-aware MMD detectors are well-calibrated.

Changing the relative subpopulation prevalences

We now focus our attention on a more realistic problem where the relative frequency of one or more subpopulations (i.e. types of hearbeats) is changing while the underlying subpopulation distribution stays the same. This would be the expected setting when we monitor the heartbeat of a specific person (e.g. only normal heartbeats) and we don't want to flag drift.

While the usual MMD detector only returns very low p-values (mostly 0), the context-aware MMD detector remains calibrated.

Changing the subpopulation distribution

In the following example we change the distribution of one or more of the underlying subpopulations (i.e. the different types of heartbeats). Notice that now we do want to flag drift since our context variable, which permits changes in relative subpopulation prevalences, can no longer explain the change in distribution.

We will again sample from the normal heartbeats, but now we will add random noise to a fraction of the extracted heartbeats to change the distribution. This could be the result of an error with some of the sensors. The perturbation is illustrated below:

As we can see from the Q-Q and power of the detector, the changes in the subpopulation are easily detected:

Changing the context variable

We now use the cluster membership probabilities of a Gaussian mixture model which is fit on the training instances as context variables instead of the model predictions. We will test both the calibration when the frequency of the subpopulations (the cluster memberships) changes as well as the power when the $F$ and $V$ heartbeats are included.

Interpretability of the context-aware detector

The test statistic $\hat{t}$ of the context-aware MMD detector can be formulated as follows: $\hat{t} = \langle K_{0,0}, W_{0,0} \rangle + \langle K_{1,1}, W_{1,1} \rangle -2\langle K_{0,1}, W_{0,1}\rangle$ where $0$ refers to the reference data, $1$ to the test data, and $W_{.,.}$ and $K_{.,.}$ are the weight and kernel matrices, respectively. The weight matrices $W_{.,.}$ allow us to focus on the distribution's subpopulations of interest. Reference instances which have similar contexts as the test data will have higher values for their entries in $W_{0,1}$ than instances with dissimilar contexts. We can therefore interpret $W_{0,1}$ as the coupling matrix between instances in the reference and the test sets. This allows us to investigate which subpopulations from the reference set are present and which are missing in the test data. If we also have a good understanding of the model performance on various subpopulations of the reference data, we could even try and use this coupling matrix to roughly proxy model performance on the unlabeled test instances. Note that in this case we would require labels from the reference data and make sure the reference instances come from the validation, not the training set.

In the following example we only pick 1 type of heartbeat (the normal one) to be present in the test set while 3 types are present in the reference set. We can then investigate via the coupling matrix whether the test statistic $\hat{t}$ focused on the right types of heartbeats in the reference data via $W_{0,1}$. More concretely, we can sum over the columns (the test instances) of $W_{0,1}$ and check which reference instances obtained the highest weights.

As expected no drift was detected since the test set only contains normal heartbeats. We now sort the weights of w_ref in descending order. We expect the top 400 entries to be fairly high and consistent since these represent the normal heartbeats in the reference set. Afterwards, the weight attribution to the other instances in the reference set should be low. The plot below confirms that this is indeed what happens.

Time conditioning

The dataset consists of nicely extracted and aligned ECGs of 140 data points for each observation. However in reality it is likely that we will continuously or periodically observe instances which are not nicely aligned. We could however assign a timestamp to the data (e.g. starting from a peak) and use time as the context variable. This is illustrated in the example below.

First we create a new dataset where we split each instance in slices of non-overlapping ECG segments. Each of the segments will have an associated timestamp as context variable. Then we can check the calibration under no change (besides the time-varying behaviour which is accounted for) as well as the power for ECG segments where we add incorrect time stamps to some of the segments.

Drift Detection

1. What is drift?

Although powerful, modern machine learning models can be sensitive. Seemingly subtle changes in a data distribution can destroy the performance of otherwise state-of-the art models, which can be especially problematic when ML models are deployed in production. Typically, ML models are tested on held out data in order to estimate their future performance. Crucially, this assumes that the process underlying the input data $\mathbf{X}$ and output data $\mathbf{Y}$ remains constant.

Drift is said to occur when the process underlying $\mathbf{X}$ and $\mathbf{Y}$ at test time differs from the process that generated the training data. In this case, we can no longer expect the model’s performance on test data to match that observed on held out training data. At test time we always observe features $\mathbf{X}$, and the ground truth then refers to a corresponding label $\mathbf{Y}$. If ground truths are available at test time, supervised drift detection can be performed, with the model’s predictive performance monitored directly. However, in many scenarios, such as the binary classification example below, ground truths are not available and unsupervised drift detection methods are required.

To explore the different types of drift, consider the common scenario where we deploy a model $f: \boldsymbol{x} \mapsto y$ on input data $\mathbf{X}$ and output data $\mathbf{Y}$, jointly distributed according to $P(\mathbf{X},\mathbf{Y})$. The model is trained on training data drawn from a distribution $P_{ref}(\mathbf{X},\mathbf{Y})$. Drift is said to have occurred when $P(\mathbf{X},\mathbf{Y}) \ne P_{ref}(\mathbf{X},\mathbf{Y})$. Writing the joint distribution as

we can classify drift under a number of types:

Covariate drift: Also referred to as input drift, this occurs when the distribution of the input data has shifted $P(\mathbf{X}) \ne P_{ref}(\mathbf{X})$, whilst $P(\mathbf{Y}|\mathbf{X})$ = $P_{ref}(\mathbf{Y}|\mathbf{X})$. This may result in the model giving unreliable predictions.
Prior drift: Also referred to as label drift, this occurs when the distribution of the outputs has shifted $P(\mathbf{Y}) \ne P_{ref}(\mathbf{Y})$, whilst $P(\mathbf{X}|\mathbf{Y})=P_{ref}(\mathbf{X}|\mathbf{Y})$. This can affect the model's decision boundary, as well as the model's performance metrics.
Concept drift: This occurs when the process generating $y$ from $x$ has changed, such that $P(\mathbf{Y}|\mathbf{X}) \ne P_{ref}(\mathbf{Y}|\mathbf{X})$. It is possible that the model might no longer give a suitable approximation of the true process.

Note that a change in one of the conditional probabilities $P(\mathbf{X}|\mathbf{Y})$ and $P(\mathbf{Y}|\mathbf{X})$ does not necessarily imply a change in the other. For example, consider the pneumonia prediction example from , whereby a classification model $f$ is trained to predict $y$, the occurrence (or not) of pneumonia, given a list of symptoms $\boldsymbol{x}$. During a pneumonia outbreak, $P(\mathbf{Y}|\mathbf{X})$ (e.g. pneumonia given cough) might rise, but the manifestations of the disease $P(\mathbf{X}|\mathbf{Y})$ might not change. In many cases, knowledge of underlying causal structure of the problem can be used to deduce that one of the conditionals will remain unchanged.

Below, the different types of drift are visualised for a simple two-dimensional classification problem. It is possible for a drift to fall under more than one category, for example the prior drift below also happens to be a case of covariate drift.

It is relatively easy to spot drift by eyeballing these figures here. However, the task becomes considerably harder for high-dimensional real problems, especially since real-time ground truths are not typically available. Some types of drift, such as prior and concept drift, are especially difficult to detect without access to ground truths. As a workaround proxies are required, for example a model’s predictions can be monitored to check for prior drift.

2. Detecting drift

offers a wide array of methods for detecting drift (see ), some of which are examined in the NeurIPS 2019 paper . Generally, these aim to determine whether the distribution $P(\mathbf{z})$ has drifted from a reference distribution $P_{ref}(\mathbf{z})$, where $\mathbf{z}$ may represent input data $\mathbf{X}$, true output data $\mathbf{Y}$, or some form of model output, depending on what type of drift we wish to detect.

Due to natural randomness in the process being modelled, we don’t necessarily expect observations $\mathbf{z}1,\dots,\mathbf{z}N$ drawn from $P(\mathbf{z})$ to be identical to $\mathbf{z}^{ref}1,\dots,\mathbf{z}^{ref}M$ drawn from $P{ref}(\mathbf{z})$. To decide whether differences between $P(\mathbf{z})$ and $P{ref}(\mathbf{z})$ are due to drift or just natural randomness in the data, statistical two-sample hypothesis testing is used, with the null hypothesis $P(\mathbf{z})=P{ref}(\mathbf{z})$. If the $p$-value obtained is below a given threshold, the null is rejected and the alternative hypothesis $P(\mathbf{z}) \ne P{ref}(\mathbf{z})$ is accepted, suggesting drift is occurring.

Since $\mathbf{z}$ is often high-dimensional (even a 200 x 200 greyscale image has 40k dimensions!), performing hypothesis testing in the full-space is often either computationally intractable, or unsuitable for the chosen statistical test. Instead, the pipeline below is often used, with dimension reduction as a pre-processing step.

:::{figure} images/drift_pipeline.png :align: center :alt: Drift detection pipeline

Figure inspired by Figure 1 in Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift. :::

Hypothesis testing

Hypothesis testing involves first choosing a test statistic $S(\mathbf{z})$, which is expected to be small if the null hypothesis $H_0$ is true, and large if the alternative $H_a$ is true. For observed data $\mathbf{z}$, $S(\mathbf{z})$ is computed, followed by a $p$-value $\hat{p} = P(\text{such an extreme } S(\mathbf{z}) | H_0)$. In other words, $\hat{p}$ represents the probability of such an extreme value of $S(\mathbf{z})$ occurring given that $H_0$ is true. When $\hat{p}\le \alpha$, results are said to be statistically significant, and the null $P(\mathbf{z})=P_{ref}(\mathbf{z})$ is rejected. Conveniently, the threshold $\alpha$ represents the desired false positive rate.

The test statistics available in can be broadly split into two categories; univariate and multivariate tests:

Univariate:
- (for categorical data)

When applied to multidimensional data with dimension $d$, the univariate tests are applied in a feature-wise manner. The obtained $p$-values for each feature are aggregated either via the or the (FDR) correction. The Bonferroni correction is more conservative and controls for the probability of at least one false positive. The FDR correction on the other hand allows for an expected fraction of false positives to occur. If the tests (i.e. each feature dimension) are independent, these corrections preserve the desired false positive rate (FPR). However, usually this is not the case, resulting in FPR's up to $d$-times lower than desired, which becomes especially problematic when $d$ is large. Additionally, since the univariate tests examine the feature-wise marginal distributions, they may miss drift in cases where the joint distribution over all $d$ features has changed, but the marginals have not. The multivariate tests avoid these problems, at the cost of greater complexity.

Dimension reduction

Given an input dataset $\mathbf{X}\in \mathbb{R}^{N\times d}$, where $N$ is the number of observations and $d$ the number of dimensions, the aim is to reduce the data dimensionality from $d$ to $K$, where $K\ll d$. A drift detector can then be applied to the lower dimensional data $\hat{\mathbf{X}}\in \mathbb{R}^{N\times K}$, where distances more meaningfully capture notions of similarity/dissimilarity between instances. Dimension reduction approaches can be broadly categorised under:

Linear projections
Non-linear projections
Feature maps (from ML model)
Model uncertainty

allows for a high degree of flexibility here, with a user’s chosen dimension reduction technique able to be incorporated into their chosen detector via the preprocess_fn argument (and sometimes preprocess_batch_fn and preprocess_at_init, depending on the detector). In the following sections, the three categories of techniques are briefly introduced. Alibi Detect offers the following functionality using either or backends and preprocessing utilities. For more details, see the .

Linear projections

This includes dimension reduction techniques such as and . These techniques involve using a transformation or projection matrix $\mathbf{R}$ to reduce the dimensionality of a given data matrix $\mathbf{X}$, such that $\hat{\mathbf{X}} = \mathbf{XR}$. A straightforward way to include such techniques as a pre-processing stage is to pass them to the detectors via the preprocess_fn argument, for example for the scikit-learn library’s PCA class:

:::{admonition} Note 1: Disjoint training and reference data sets

Astute readers may have noticed that in the code snippet above, the data X_train is used to “train” the PCA model, but the MMDDrift detector is initialised with X_ref. This is a subtle yet important point. If a detector’s preprocessor (a or other step) is trained on the reference data (X_ref), any over-fitting to this data may make the resulting detector overly sensitive to differences between the reference and test data sets.

To avoid an overly discriminative detector, it is customary to draw two disjoint datasets from $P_{ref}(\mathbf{z})$, a training set and a held-out reference set. The training data is used to train any input preprocessing steps, and the detector is then initialised on the reference set, and used to detect drift between the reference and test set. This also applies to the , which should be trained on the training set not the reference set. :::

Non-linear projections

A common strategy for obtaining non-linear dimension reducing representations is to use an autoencoder, but other can also be used. Autoencoders consist of an encoder function $\phi : \mathcal{X} \mapsto \mathcal{H}$ and a decoder function $\psi : \mathcal{H} \mapsto \mathcal{X}$, where the latent space $\mathcal{H}$ has lower dimensionality than the input space $\mathcal{X}$. The output of the encoder $\hat{\mathbf{X}} \in \mathcal{H}$ can then be monitored by the drift detector. Training involves learning both the encoding function $\phi$ and the decoding function $\psi$, in order to reduce the reconstruction loss, e.g. if MSE is used: $\phi, \psi = \text{arg} \min_{\phi, \psi}, \lVert \mathbf{X}-(\phi \circ \psi)\mathbf{X}\rVert^2$. However, untrained (randomly initialised) autoencoders can also be used. For an example, a pytorch autoencoder can be incorporated into a detector by packaging it as a callable function using {func}~alibi_detect.cd.pytorch.preprocess.preprocess_drift and {func}~functools.partial:

Feature maps

Following , feature maps can be extracted from existing pre-trained black-box models such as the image classifier shown below. Instead of using the latent space as the dimensionality-reducing representation, other layers of the model such as the softmax outputs or predicted class-labels can also be extracted and monitored. Since different layers yield different output dimensions, different hypothesis tests are required for each.

:::{figure} images/BBSD.png :align: center :alt: Black box shift detection

Figure inspired by this MNIST classification example from the timeserio package. :::

shows that extracting feature maps from existing models can be an effective technique, which is encouraging since this allows the user to repurpose existing black-box models for use as drift detectors. The syntax for incorporating existing models into drift detectors is similar to the previous autoencoder example, with the added step of using {class}~alibi_detect.cd.tensorflow.preprocess.HiddenOutput to select the model’s network layer to extract outputs from. The code snippet below is borrowed from , where the softmax layer of the well-known model is fed into an MMDDrift detector.

Model uncertainty

The -based drift detector uses the ML model of interest itself to detect drift. These detectors aim to directly detect drift that’s likely to affect the performance of a model of interest. The approach is to test for change in the number of instances falling into regions of the input space on which the model is uncertain in its predictions. For each instance in the reference set the detector obtains the model’s prediction and some associated notion of uncertainty. The same is done for the test set and if significant differences in uncertainty are detected (via a Kolmogorov-Smirnoff test) then drift is flagged. The model’s notion of uncertainty depends on the type of model. For a classifier this may be the entropy of the predicted label probabilities. For a regressor with dropout layers, can be used to provide a notion of uncertainty.

The model uncertainty-based detectors are classed under the dimension reduction category since a model's uncertainty is by definition one-dimensional. However, the syntax for the uncertainty-based detectors is different to the other detectors. Instead of passing a pre-processing step to a detector via a preprocess_fn (or similar) argument, the dimension reduction (in this case computing a notion of uncertainty) is performed internally by these detectors.

Input preprocessing

Dimension reduction is a common preprocessing task (e.g. for covariate drift detection on tabular or image data), but some modalities of data (e.g. text and graph data) require other forms of preprocessing in order for drift detection to be performed effectively.

Text data

When dealing with text data, performing drift detection on raw strings or tokenized data is not effective since they don’t represent the semantics of the input. Instead, we extract contextual embeddings from language transformer models and detect drift on those. This procedure has a significant impact on the type of drift we detect. Strictly speaking we are not detecting covariate/input drift anymore since the entire training procedure (objective function, training data etc) for the (pre)trained embeddings has an impact on the embeddings we extract.

:::{figure} images/BERT.png :align: center :alt: The DistilBERT language representation model

Figure based on Jay Alammar’s excellent to the BERT model :::

Alibi Detect contains functionality to leverage pre-trained embeddings from HuggingFace’s package. Popular models such as or (shown above) can be used, but Alibi Detect also allows you to easily use your own embeddings of choice. A subsequent dimension reduction step can also be applied if necessary, as is done in the example, where the 768-dimensional embeddings from the BERT model are passed through an untrained AutoEncoder to reduce their dimensionality. Alibi Detect allows various types of embeddings to be extracted from transformer models, using {class}~alibi_detect.models.tensorflow.embedding.TransformerEmbedding.

Graph data

In a similar manner to text data, graph data requires preprocessing before drift detection can be performed. This can be done by extracting graph embeddings from graph neural network (GNN) encoders, as shown below, and demonstrated in the example.

Simple example

For a simple example, we’ll use the to check for drift on the two-dimensional binary classification problem shown previously (see ). The MMD detector is a kernel-based method for multivariate two sample testing. Since the number of dimensions is already low, dimension reduction step is not necessary here here. For a more advanced example using the with dimension reduction, check out the example.

The true model/process is defined as:

where the slope $s$ is set as $s=-1$.

The reference distribution is defined as a mixture of two Normal distributions:

with the standard deviation set at $\sigma=0.8$, and the weights set to $\phi_1=\phi_2=0.5$. Reference data $\mathbf{X}^{ref}$ and training data $\mathbf{X}^{train}$ (see Note 1) can be generated by sampling from this distribution. The corresponding labels $\mathbf{Y}^{ref}$ and $\mathbf{Y}^{train}$ are obtained by evaluating true_model().

For a model, we choose the well-known decision tree classifier. As well as training the model, this is a good time to initialise the MMD detector with the held-out reference data $\mathbf{X}^{ref}$ by calling:

The significance threshold is set at $\alpha=0.05$, meaning the detector will flag results as drift detected when the computed $p$-value is less than this i.e. $\hat{p}< \alpha$.

No drift

Before introducing drift, we first examine the case where no drift is present. We resample from the same mixture of Gaussian distributions to generate test data $\mathbf{X}$. The individual data observations are different, but the underlying distributions are unchanged, hence no true drift is present.

Unsurprisingly, the model’s mean test accuracy is relatively high. To run the detector on test data the .predict() is used:

For the test statistic $S(\mathbf{X})$ (we write $S(\mathbf{X})$ instead of $S(\mathbf{z})$ since the detector is operating on input data), the MMD detector uses the to compute unbiased estimates of $\text{MMD}^2$. The $\text{MMD}$ is a distance-based measure between the two distributions $P$ and $P_{ref}$, based on the mean embeddings $\mu$ and $\mu_{ref}$ in a reproducing kernel Hilbert space $F$:

A $p$-value is then obtained via a on the estimates of $\text{MMD}^2$. As expected, since we are sampling from the reference distribution $P_{ref}(\mathbf{X})$, the detector’s prediction is 'is_drift':0 here, indicating that drift is not detected. More specifically, the detector’s $p$-value (p_val) is above the threshold of $\alpha=0.05$ (threshold), indicating that no statistically significant drift has been detected. The .predict() method also returns $\hat{S}(\mathbf{X})$ (distance_threshold), which is the threshold in terms of the test statistic $S(\mathbf{X})$ i.e. when $S(\mathbf{X})\ge \hat{S}(\mathbf{X})$ statistically significant drift has been detected.

Covariate and prior drift

To impose covariate drift, we apply a shift to the mean of one of the normal distributions:

The test data has drifted into a previously unseen region of feature space, and the model is now misclassifying a number of test observations. If true test labels are available, this is easily detectable by monitoring the test accuracy. However, labels are not always available at test time, in which case a drift detector monitoring the covariates comes in handy. In this case, the MMD detector successfully detects the covariate drift.

In a similar manner, a proxy for prior drift can be monitored by initialising a detector on labels from the reference set, and then feeding it a model’s predicted labels:

3. Learned drift detection

It can often be challenging to specify a test statistic $S(\mathbf{z})$ that is large when drift is present and small otherwise. Alibi Detect offers a number of learned detectors, which try to explicitly learn a test statistic which satisfies this property:

(erence)

These detectors can be highly effective, but require training hence potentially increasing data requirements and set-up time. Similarly to when training preprocessing steps, it is important that the learned detectors are trained on training data which is held-out from the reference data set (see Note 1). A brief overview of these detectors is given below. For more details, see the detectors’ respective pages.

Learned kernel

The uses a kernel $k(\mathbf{z},\mathbf{z}^{ref})$ to compute unbiased estimates of $\text{MMD}^2$. The user is free to provide their own kernel, but by default a is used. The drift detector () extends this approach by training a kernel to maximise an estimate of the resulting test power. The learned kernel is defined as

where $\Phi$ is a learnable projection, $k_a$ and $k_b$ are simple characteristic kernels (such as a Gaussian RBF), and $\epsilon>0$ is a small constant. By letting $\Phi$ be very flexible we can learn powerful kernels in this manner.

The figure below compares the use of a Gaussian and a learned kernel for identifying differences between two distributions $P$ and $P_{ref}$. The distributions are each equal mixtures of nine Gaussians with the same modes, but each component of $P_{ref}$ is an isotropic Gaussian, whereas the covariance of $P$ differs in each component. The Gaussian kernel (c) treats points isotropically throughout the space, based upon $\lVert \mathbf{z} - \mathbf{z}^{ref} \rVert$ only. The learned kernel (d) behaves differently in different regions of the space, adapting to local structure and therefore allowing better detection of differences between $P$ and $P_{ref}$.

:::{figure} images/deep_kernel.png :align: center :alt: Gaussian and deep kernels

Original image source: Liu et al., 2020. Captions modified to match notation used elsewhere on this page. :::

Classifier

The -based drift detector () attempts to detect drift by explicitly training a classifier to discriminate between data from the reference and test sets. The statistical test used depends on whether the classifier outputs probabilities or binarized (0 or 1) predictions, but the general idea is to determine whether the classifiers performance is statistically different from random chance. If the classifier can learn to discriminate better than randomly (in a generalisable manner) then drift must have occurred.

show that a classifier-based drift detector is actually a special case of the . An important difference is that to train a classifier we maximise its accuracy (or a cross-entropy proxy), while for a learned kernel we maximise the test power directly. Liu et al. show that the latter approach is empirically superior.

Spot-the-diff

The (erence) drift detector is an extension of the drift detector, where the classifier is specified in a manner that makes detections interpretable at the feature level when they occur. The detector is inspired by the work of but various major adaptations have been made.

As with the usual classifier-based approach, a portion of the available data is used to train a classifier that can discriminate reference instances from test instances. However, the spot-the-diff detector is specified such that when drift is detected, we can inspect the weights of the classifier to shine light on exactly which features of the data were used to distinguish reference from test samples, and therefore caused drift to be detected. The example demonstrates this capability.

4. Online drift detection

So far, we have discussed drift detection in an offline context, with the entire test set ${\mathbf{z}i}{i=1}^{N}$ compared to the reference dataset ${\mathbf{z}^{ref}i}{i=1}^{M}$. However, at test time, data sometimes arrives sequentially. Here it is desirable to detect drift in an online fashion, allowing us to respond as quickly as possible and limit the damage it might cause.

One approach is to perform a test for drift every $W$ time-steps, using the $W$ samples that have arrived since the last test. In other words, that is to compare ${\mathbf{z}i}{i=t-W+1}^{t}$ to ${\mathbf{z}^{ref}i}{i=1}^{M}$. Such a strategy could be implemented using any of the offline detectors implemented in alibi-detect, but being both sensitive to slight drift and responsive to severe drift is difficult. If the window size $W$ is too large the delay between consecutive statistical tests hampers responsiveness to severe drift, but an overly small window is unreliable. This is demonstrated below, where the offline is used to monitor drift in data $\mathbf{X}$ sampled from a normal distribution $\mathcal{N}\left(\mu,\sigma^2 \right)$ over time $t$, with the mean starting to drift from $\mu=0$ to $\mu=0.5$ at $t=40$.

Online drift detectors

An alternative strategy is to perform a test each time data arrives. However the usual offline methods are not applicable because the process for computing $p$-values is too expensive. Additionally, they don’t account for correlated test outcomes when using overlapping windows of test data, leading to miscalibrated detectors operating at an unknown False Positive Rate (FPR). Well-calibrated FPR’s are crucial for judging the significance of a drift detection. In the absence of calibration, drift detection can be useless since there is no way of knowing what fraction of detections are false positives. To tackle this problem, offers specialist online drift detectors:

These detectors leverage the calibration method introduced by in order to ensure they are well well-calibrated when used in a sequential manner. The detectors compute a test statistic $S(\mathbf{z})$ during the configuration phase. Then, at test time, the test statistic is updated sequentially at a low cost. When no drift has occurred the test statistic fluctuates around its expected value, and once drift occurs the test statistic starts to drift upwards. When it exceeds some preconfigured threshold value, drift is detected. The online detectors are constructed in a similar manner to the offline detectors, for example for the online MMD detector:

But, in addition to providing the detector with reference data, the expected run-time (see below), and size of the sliding window must also be defined. Another important difference is that the online detectors make predictions on single data instances:

This can be seen in the animation below, where the online detector considers each incoming observation/sample individually, instead of considering a batch of observations like the offline detectors.

Expected run-time and detection delay

Unlike offline detectors which require the specification of a threshold $p$-value, which is equivalent to a false positive rate (FPR), the online detectors in alibi-detect require the specification of an expected run-time (ERT) (an inverted FPR). This is the number of time-steps that we insist our detectors, on average, should run for in the absence of drift, before making a false detection.

Usually we would like the ERT to be large, however this results in insensitive detectors which are slow to respond when drift does occur. Hence, there is a tradeoff between the expected run time and the expected detection delay (the time taken for the detector to respond to drift in the data). To target the desired ERT, thresholds are configured during an initial configuration phase via simulation (n_bootstraps sets the number of boostrap simulations used here). This configuration process is only suitable when the amount of reference data is relatively large (ideally around an order of magnitude larger than the desired ERT). Configuration can be expensive (less so with a GPU), but allows the detector to operate at a low-cost at test time. For a more in-depth explanation, see .

Context-aware drift detection on news articles

Introduction

In this notebook we show how to detect drift on text data given a specific context using the context-aware MMD detector (Cobb and Van Looveren, 2022). Consider the following simple example: the upcoming elections result in an increase of political news articles compared to other topics such as sports or science. Given the context (the elections), it is however not surprising that we observe this uptick. Moreover, assume we have a machine learning model which is trained to classify news topics, and this model performs well on political articles. So given that we fully expect this uptick to occur given the context, and that our model performs fine on the political news articles, we do not want to flag this type of drift in the data. This setting corresponds more closely to many real-life settings than traditional drift detection where we make the assumption that both the reference and test data are i.i.d. samples from their underlying distributions.

In our news topics example, each different topic such as politics, sports or weather represents a subpopulation of the data. Our context-aware drift detector can then detect changes in the data distribution which cannot be attributed to a change in the relative prevalences of these subpopulations, which we deem permissible. As a cherry on the cake, the context-aware detector allows you to understand which subpopulations are present in both the reference and test data. This allows you to obtain deep insights into the distribution underlying the test data.

Useful context (or conditioning) variables for the context-aware drift detector include but are not limited to:

Domain or application specific contexts such as the time of day or the weather.
Conditioning on the relative prevalences of known subpopulations, such as the frequency of political articles. It is important to note that while the relative frequency of each subpopulation might change, the distribution underlying each subpopulation cannot change.
Conditioning on model predictions. Assume we trained a classifier which tries to figure out which news topic an article belongs to. Given our model predictions we then want to understand whether our test data follows the same underlying distribution as reference instances with similar model predictions. This conditioning would also be useful in case of trending news topics which cause the model prediction distribution to shift but not necessarily the distribution within each of the news topics.

The following settings will be illustrated throughout the notebook:

A change in the prevalences of subpopulations (i.e. news topics) relative to their prevalences in the training data. Contrary to traditional drift detection approaches, the context-aware detector does not flag drift as this change in frequency of news topics is permissible given the context provided (e.g. more political news articles around elections).
A change in the underlying distribution of one or more subpopulations takes place. While we allow changes in the prevalence of the subpopulations accounted for by the context variable, we do not allow changes of the subpopulations themselves. Let's assume that a newspaper usually has a certain tone (e.g. more conservative) when it comes to politics. If this tone changes (to less conservative) around elections (increased frequency of political news articles), then we want to flag it as drift since the change cannot be attributed to the context given to the detector.
A change in the distribution as we observe a previously unseen news topic

Under setting 1. we want our detector to be well-calibrated (a controlled False Positive Rate (FPR) and more generally a p-value which is uniformly distributed between 0 and 1) while under settings 2. and 3. we want our detector to be powerful and flag the drift. Lastly, we show how the detector can help you to understand the connection between the reference and test data distributions better.

Data

We use the which contains about 18,000 newsgroups post across 20 topics, including politics, science sports or religion.

Requirements

The notebook requires the umap-learn, torch, sentence-transformers, statsmodels, seaborn and datasets packages to be installed, which can be done via pip:

Before we start let's fix the random seeds for reproducibility:

Load data

First we load the data, show which classes (news topics) are present and what an instance looks like.

Let's take a look at an instance from the dataset:

Define models and train a classifier

We embed the news posts using pre-trained embeddings and optionally add a dimensionality reduction step with . UMAP also allows to leverage reference data labels.

We define respectively a generic clustering model using UMAP, a model to embed the text input using pre-trained SentenceTransformers embeddings, a text classifier and a utility function to place the data on the right device.

First we train a classifier on a small subset of the data. The aim of the classifier is to predict the news topic of each instance. Below we define a few simple training and evaluation functions.

We now split the data in 2 sets. The first set (x_train) we will use to train our text classifier, and the second set (x_drift) is held out to test our drift detector on.

Let's train our classifier. The classifier consists of a simple MLP head on top of a pre-trained SentenceTransformer model as the backbone. The SentenceTransformer remains frozen during training and only the MLP head is finetuned.

Detector calibration under no change

We start with an example where no drift occurs and the reference and test data are both sampled randomly from all news topics. Under this scenario, we expect no drift to be detected by either a normal MMD detector or by the context-aware MMD detector.

First we define some helper functions. The first one visualises the clustered text data while the second function samples disjoint reference and test sets with a specified number of instances per class (i.e. per news topic).

We first define the embedding model using the pre-trained SentenceTransformer embeddings and then embed both the reference and test sets.

By applying UMAP clustering on the SentenceTransformer embeddings, we can visually inspect the various news topic clusters. Note that we fit the clustering model on the held out data first, and then make predictions on the reference and test sets.

We can visually see that the reference and test set are made up of similar clusters of data, grouped by news topic. As a result, we would not expect drift to be flagged. If the data distribution did not change, we can expect the p-value distribution of our statistical test to be uniformly distributed between 0 and 1. So let's see if this assumption holds.

Importantly, first we need to define our context variable for the context-aware MMD detector. In our experiments we allow the relative prevalences of subpopulations to vary while the distributions underlying each of the subpopulations remain unchanged. To achieve this we condition on the prediction probabilities of the classifier we trained earlier to distinguish each of the 20 different news topics. We can do this because the prediction probabilities can account for the frequency of occurrence of each of the topics (be it imperfectly given our classifier makes the occasional mistake).

Before we set off our experiments, we embed all the instances in x_drift and compute all contexts c_drift so we don't have to call our transformer model every single pass in the for loop.

As expected we can see that both the normal MMD and the context-aware MMD detectors are well-calibrated.

Changing the relative subpopulation prevalence

We now focus our attention on a more realistic problem where the relative frequency of one or more subpopulations (i.e. news topics) is changing in a way which can be attributed to external events. Importantly, the distribution underlying each subpopulation (e.g. the distribution of hockey news itself) remains unchanged, only its frequency changes.

In our example we assume that the World Series and Stanley Cup coincide on the calendar leading to a spike in news articles on respectively baseball and hockey. Furthermore, there is not too much news on Mac or Windows since there are no new releases or products planned anytime soon.

While the context-aware detector remains well calibrated, the MMD detector consistently flags drift (low p-values). Note that this is the expected behaviour since the vanilla MMD detector cannot take any external context into account and correctly detects that the reference and test data do not follow the same underlying distribution.

We can also easily see this on the plot below where the p-values of the context-aware detector are uniformly distributed while the MMD detector's p-values are consistently close to 0. Note that we limited the y-axis range to make the plot easier to read.

Changing the subpopulation distribution

In the following example we change the distribution of one or more of the underlying subpopulations. Notice that now we do want to flag drift since our context variable, which permits changes in relative subpopulation prevalences, can no longer explain the change in distribution.

Imagine our news topic classification model is not as granular as before and instead of the 20 categories only predicts the 6 super classes, organised by subject matter:

Computers: comp.graphics; comp.os.ms-windows.misc; comp.sys.ibm.pc.hardware; comp.sys.mac.hardware; comp.windows.x
Recreation: rec.autos; rec.motorcycles; rec.sport.baseball; rec.sport.hockey
Science: sci.crypt; sci.electronics; sci.med; sci.space
Miscellaneous: misc.forsale

What if baseball and hockey become less popular and the distribution underlying the Recreation class changes? We will want to detect this as the change in distributions of the subpopulations (the 6 super classes) cannot be explained anymore by the context variable.

In order to reuse our pretrained classifier for the super classes, we add the following helper function to map the predictions on the super classes and return one-hot encoded predictions over the 6 super classes. Note that our context variable now changes from a probability distribution over the 20 news topics to a one-hot encoded representation over the 6 super classes.

We can see that the context-aware detector is powerful to detect changes in the distributions of the subpopulations.

Detect unseen topics

Next we illustrate the effectiveness of the context-aware detector to detect new topics which are not present in the reference data. Obviously we also want to flag drift in this case. As an example we introduce movie reviews in the test data.

Changing the context variable

So far we have conditioned the context-aware detector on the model predictions. There are however many other useful contexts possible. One such example would be to condition on the predictions of an unsupervised clustering algorithm. To facilitate this, we first apply kernel PCA on the embedding vectors, followed by a Gaussian mixture model which clusters the data into 6 classes (same as the super classes). We will test both the calibration under the null hypothesis (no distribution change) as well as the power when a new topic (movie reviews) is injected.

Next we change the number of instances in each cluster between the reference and test sets. Note that we do not alter the underlying distribution of each of the clusters, just the frequency.

Now we run the experiment and show the context-aware detector's calibration when changing the cluster frequencies. We also show how the usual MMD detector will consistently flag drift. Furthermore, we inject instances from the movie reviews dataset and illustrate that the context-aware detector remains powerful when the underlying cluster distribution changes (by including a previously unseen topic).

Interpretability of the context-aware detector

In the following example we only pick 2 classes to be present in the test set while all 20 are present in the reference set. We can then investigate via the coupling matrix whether the test statistic $\hat{t}$ focused on the right classes in the reference data via $W_{0,1}$. More concretely, we can sum over the columns (the test instances) of $W_{0,1}$ and check which reference instances obtained the highest weights.

Deployment

1. Dataset
2. Outlier detection with a variational autoencoder (VAE)
3. Adversarial detection by matching prediction probabilities
4. Drift detection with Kolmogorov-Smirnov

1. Dataset

consists of 60,000 32 by 32 RGB images equally distributed over 10 classes: airplane, automobile, bird, cat, deer, dog, frog, horse, ship and truck.

2. Outlier detection with a variational autoencoder (VAE)

Method

In a nutshell:

Train a VAE on normal data so it can reconstruct inliers well
If the VAE cannot reconstruct the incoming requests well? Outlier!

More resources on VAE: ,

Image source: https://lilianweng.github.io/lil-log/2018/08/12/from-autoencoder-to-beta-vae.html

Load detector or train from scratch

The pretrained outlier and adversarial detectors used in the notebook can be found . You can use the built-in fetch_detector function which saves the pre-trained models in a local directory filepath and loads the detector. Alternatively, you can train a detector from scratch:

Let's check whether the model manages to reconstruct the in-distribution training data:

Setting the threshold

Finding good threshold values can be tricky since they are typically not easy to interpret. The infer_threshold method helps finding a sensible value. We need to pass a batch of instances X and specify what percentage of those we consider to be normal via threshold_perc.

Create and detect outliers

We can create some outliers by applying a random noise mask to the original instances:

Deploy the detector

For this example we use the open source deployment platform and eventing based project which allows serverless components to be connected to event streams. The Seldon Core payload logger sends events containing model requests to the Knative broker which can farm these out to serverless components such as the outlier, drift or adversarial detection modules. Further eventing components can be added to feed off events produced by these components to send onwards to, for example, alerting or storage modules. This happens asynchronously.

We already configured a cluster on DigitalOcean with Seldon Core installed. The configuration steps to set everything up from scratch are detailed in .

First we get the IP address of the Istio Ingress Gateway. This assumes Istio is installed with a LoadBalancer.

We define some utility functions for the prediction of the deployed model.

Let's make a prediction on the original instance:

Let's check the message dumper for the output of the outlier detector:

We then make a prediction on the perturbed instance:

Although the prediction is still correct, the instance is clearly an outlier:

3. Adversarial detection by matching prediction probabilities

Method

The adversarial detector is based on . Usually, autoencoders are trained to find a transformation $T$ that reconstructs the input instance $x$ as accurately as possible with loss functions that are suited to capture the similarities between x and $x'$ such as the mean squared reconstruction error. The novelty of the adversarial autoencoder (AE) detector relies on the use of a classification model-dependent loss function based on a distance metric in the output space of the model to train the autoencoder network. Given a classification model $M$ we optimise the weights of the autoencoder such that the between the model predictions on $x$ and on $x'$ is minimised. Without the presence of a reconstruction loss term $x'$ simply tries to make sure that the prediction probabilities $M(x')$ and $M(x)$ match without caring about the proximity of $x'$ to $x$. As a result, $x'$ is allowed to live in different areas of the input feature space than $x$ with different decision boundary shapes with respect to the model $M$. The carefully crafted adversarial perturbation which is effective around x does not transfer to the new location of $x'$ in the feature space, and the attack is therefore neutralised. Training of the autoencoder is unsupervised since we only need access to the model prediction probabilities and the normal training instances. We do not require any knowledge about the underlying adversarial attack and the classifier weights are frozen during training.

The detector can be used as follows:

An adversarial score $S$ is computed. $S$ equals the K-L divergence between the model predictions on $x$ and $x'$.
If $S$ is above a threshold (explicitly defined or inferred from training data), the instance is flagged as adversarial.
For adversarial instances, the model $M$ uses the reconstructed instance $x'$ to make a prediction. If the adversarial score is below the threshold, the model makes a prediction on the original instance $x$.

This procedure is illustrated in the diagram below:

The method is very flexible and can also be used to detect common data corruptions and perturbations which negatively impact the model performance.

Utility functions

Rescale data

The ResNet classification model is trained on data standardized by instance:

Load pre-trained classifier

Check the predictions on the test:

Adversarial attack

We investigate both and attacks. You can simply load previously found adversarial instances on the pretrained ResNet-56 model. The attacks are generated by using :

We can verify that the accuracy of the classifier drops to almost $0$%:

Let's visualise some adversarial instances:

Load or train and evaluate the adversarial detector

We can again either fetch the pretrained detector from a or train one from scratch:

The detector first reconstructs the input instances which can be adversarial. The reconstructed input is then fed to the classifier to compute the adversarial score. If the score is above a threshold, the instance is classified as adversarial and the detector tries to correct the attack. Let's investigate what happens when we reconstruct attacked instances and make predictions on them:

Accuracy on attacked vs. reconstructed instances:

The detector restores the accuracy after the attacks from almost $0$% to well over $80$%! We can compute the adversarial scores and inspect some of the reconstructed instances:

The ROC curves and AUC values show the effectiveness of the adversarial score to detect adversarial instances:

The threshold for the adversarial score can be set via infer_threshold. We need to pass a batch of instances $X$ and specify what percentage of those we consider to be normal via threshold_perc. Assume we have only normal instances some of which the model has misclassified leading to a higher score if the reconstruction picked up features from the correct class or some might look adversarial in the first place. As a result, we set our threshold at $95$%:

The correct method of the detector executes the diagram in Figure 1. First the adversarial scores is computed. For instances where the score is above the threshold, the classifier prediction on the reconstructed instance is returned. Otherwise the original prediction is kept. The method returns a dictionary containing the metadata of the detector, whether the instances in the batch are adversarial (above the threshold) or not, the classifier predictions using the correction mechanism and both the original and reconstructed predictions. Let's illustrate this on a batch containing some adversarial (C&W) and original test set instances:

Let's check the model performance:

This can be improved with the correction mechanism:

There are a few other tricks highlighted in the (temperature scaling and hidden layer K-L divergence) and implemented in Alibi Detect which can further boost the adversarial detector's performance. Check for more details.

4. Drift detection with Kolmogorov-Smirnov

Method

The drift detector applies feature-wise two-sample (K-S) tests. For multivariate data, the obtained p-values for each feature are aggregated either via the or the (FDR) correction. The Bonferroni correction is more conservative and controls for the probability of at least one false positive. The FDR correction on the other hand allows for an expected fraction of false positives to occur.

For high-dimensional data, we typically want to reduce the dimensionality before computing the feature-wise univariate K-S tests and aggregating those via the chosen correction method. Following suggestions in , we incorporate Untrained AutoEncoders (UAE), black-box shift detection using the classifier's softmax outputs () and as out-of-the box preprocessing methods. Preprocessing methods which do not rely on the classifier will usually pick up drift in the input data, while BBSDs focuses on label shift. The which is part of the library can also be transformed into a drift detector picking up drift that reduces the performance of the classification model. We can therefore combine different preprocessing techniques to figure out if there is drift which hurts the model performance, and whether this drift can be classified as input drift or label shift.

Note that the library also has a drift detector based on the and contains as well.

Dataset

We will use the CIFAR-10-C dataset () to evaluate the drift detector. The instances in CIFAR-10-C come from the test set in CIFAR-10 but have been corrupted and perturbed by various types of noise, blur, brightness etc. at different levels of severity, leading to a gradual decline in the classification model performance. We also check for drift against the original test set with class imbalances.

We can select from the following corruption types at 5 severity levels:

Let's pick a subset of the corruptions at corruption level 5. Each corruption type consists of perturbations on all of the original test set images.

We split the original test set in a reference dataset and a dataset which should not be rejected under the H0 of the K-S test. We also split the corrupted data by corruption type:

We can visualise the same instance for each corruption type:

We can also verify that the performance of a ResNet-32 classification model on CIFAR-10 drops significantly on this perturbed dataset:

Given the drop in performance, it is important that we detect the harmful data drift!

Detect drift

We are trying to detect data drift on high-dimensional (32x32x3) data using an aggregation of univariate K-S tests. It therefore makes sense to apply dimensionality reduction first. Some dimensionality reduction methods also used in are readily available: UAE (Untrained AutoEncoder), BBSDs (black-box shift detection using the classifier's softmax outputs) and PCA (using scikit-learn).

Untrained AutoEncoder

First we try UAE:

Let's check whether the detector thinks drift occurred within the original test set:

As expected, no drift occurred. We can also inspect the feature-wise K-S statistics, threshold value and p-values for each univariate K-S test by (encoded) feature before the multivariate correction. Most of them are well above the $0.05$ threshold:

Let's now check the predictions on the perturbed data:

BBSDs

For BBSDs, we use the classifier's softmax outputs for black-box shift detection. This method is based on .

Here we use the output of the softmax layer to detect the drift, but other hidden layers can be extracted as well by setting 'layer' to the index of the desired hidden layer in the model:

There is again no drift on the original held out test set:

We compare this with the perturbed data:

For more functionality and examples, such as updating the reference data with reservoir sampling or picking another multivariate correction mechanism, check out .

Leveraging the adversarial detector for malicious drift detection

While monitoring covariate and predicted label shift is all very interesting and exciting, at the end of the day we are mainly interested in whether the drift actually hurt the model performance significantly. To this end, we can leverage the adversarial detector and measure univiariate drift on the adversarial scores!

Make drift predictions on the original test set and corrupted data:

We can therefore use the scores of the detector itself to quantify the harmfulness of the drift! We can generalise this to all the corruptions at each severity level in CIFAR-10-C.

On the plot below we show the mean values and standard deviations of the adversarial scores per severity level. The plot shows the mean adversarial scores (lhs) and ResNet-32 accuracies (rhs) for increasing data corruption severity levels. Level 0 corresponds to the original test set. Harmful scores are scores from instances which have been flipped from the correct to an incorrect prediction because of the corruption. Not harmful means that the prediction was unchanged after the corruption. The chart can be reproduced in .

Deploy

We can deploy the drift detector in a similar fashion as the . For a more detailed step-by-step overview of the deployment process, check .

The deployed drift detector accumulates requests until a predefined drift_batch_size is reached, in our case $5000$ which is defined in the and set in the . After $5000$ instances, the batch is cleared and fills up again.

We now run the same test on some corrupted data:

alibi_detect.cd.base

Constants

`has_pytorch`

bool(x) -> bool

Returns True when the argument x is true, False otherwise. The builtins True and False are the only two instances of the class bool. The class bool is a subclass of the class int, and cannot be subclassed.

`has_tensorflow`

bool(x) -> bool

`logger`

Instances of the Logger class represent a single logging channel. A "logging channel" indicates an area of an application. Exactly how an "area" is defined is up to the application developer. Since an application can have any number of areas, logging channels are identified by a unique string. Application areas can be nested (e.g. an area of "input processing" might include sub-areas "read CSV files", "read XLS files" and "read Gnumeric files"). To cater for this natural nesting, channel names are organized into a namespace hierarchy where levels are separated by periods, much like the Java or Python package namespace. So in the instance given above, channel names might be "input" for the upper level, and "input.csv", "input.xls" and "input.gnu" for the sub-levels. There is no arbitrary limit to the depth of nesting.

`BaseClassifierDrift`

Inherits from: BaseDetector, ABC

Constructor

Name

Type

Default

Description

Methods

`get_splits`

Split reference and test data in train and test folds used by the classifier.

Name

Type

Default

Description

Returns

Type: Union[Tuple[Union[numpy.ndarray, list], numpy.ndarray], Tuple[Union[numpy.ndarray, list], numpy.ndarray, Optional[List[Tuple[numpy.ndarray, numpy.ndarray]]]]]

`predict`

Predict whether a batch of data has drifted from the reference data.

Name

Type

Default

Description

Returns

Type: Dict[str, Dict[str, Union[str, int, float, Callable]]]

`preprocess`

Data preprocessing before computing the drift scores.

Name

Type

Default

Description

Returns

Type: Tuple[Union[numpy.ndarray, list], Union[numpy.ndarray, list]]

`score`

Name

Type

Default

Description

Returns

Type: Tuple[float, float, numpy.ndarray, numpy.ndarray, Union[numpy.ndarray, list], Union[numpy.ndarray, list]]

`test_probs`

Perform a statistical test of the probabilities predicted by the model against

what we'd expect under the no-change null.

Name

Type

Default

Description

Returns

Type: Tuple[float, float]

`BaseContextMMDDrift`

Inherits from: BaseDetector, ABC

Constructor

Name

Type

Default

Description

Methods

`predict`

Predict whether a batch of data has drifted from the reference data, given the provided context.

Name

Type

Default

Description

Returns

Type: Dict[Dict[str, str], Dict[str, Union[int, float]]]

`preprocess`

Data preprocessing before computing the drift scores.

Name

Type

Default

Description

Returns

Type: Tuple[numpy.ndarray, numpy.ndarray]

`score`

Name

Type

Default

Description

Returns

Type: Tuple[float, float, float, Tuple]

`BaseLSDDDrift`

Inherits from: BaseDetector, ABC

Constructor

Name

Type

Default

Description

Methods

`predict`

Predict whether a batch of data has drifted from the reference data.

Name

Type

Default

Description

Returns

Type: Dict[Dict[str, str], Dict[str, Union[int, float]]]

`preprocess`

Data preprocessing before computing the drift scores.

Name

Type

Default

Description

Returns

Type: Tuple[numpy.ndarray, numpy.ndarray]

`score`

Name

Type

Default

Description

Returns

Type: Tuple[float, float, float]

`BaseLearnedKernelDrift`

Inherits from: BaseDetector, ABC

Constructor

Name

Type

Default

Description

Methods

`get_splits`

Split reference and test data into two splits -- one of which to learn test locations

and parameters and one to use for tests.

Name

Type

Default

Description

Returns

Type: Tuple[Tuple[Union[numpy.ndarray, list], Union[numpy.ndarray, list]], Tuple[Union[numpy.ndarray, list], Union[numpy.ndarray, list]]]

`predict`

Predict whether a batch of data has drifted from the reference data.

Name

Type

Default

Description

Returns

Type: Dict[Dict[str, str], Dict[str, Union[int, float, Callable]]]

`preprocess`

Data preprocessing before computing the drift scores.

Name

Type

Default

Description

Returns

Type: Tuple[Union[numpy.ndarray, list], Union[numpy.ndarray, list]]

`score`

Name

Type

Default

Description

Returns

Type: Tuple[float, float, float]

`BaseMMDDrift`

Inherits from: BaseDetector, ABC

Constructor

Name

Type

Default

Description

Methods

`predict`

Predict whether a batch of data has drifted from the reference data.

Name

Type

Default

Description

Returns

Type: Dict[Dict[str, str], Dict[str, Union[int, float]]]

`preprocess`

Data preprocessing before computing the drift scores.

Name

Type

Default

Description

Returns

Type: Tuple[numpy.ndarray, numpy.ndarray]

`score`

Name

Type

Default

Description

Returns

Type: Tuple[float, float, float]

`BaseUnivariateDrift`

Inherits from: BaseDetector, ABC, DriftConfigMixin

Constructor

Name

Type

Default

Description

Methods

`feature_score`

Name

Type

Default

Description

Returns

Type: Tuple[numpy.ndarray, numpy.ndarray]

`predict`

Predict whether a batch of data has drifted from the reference data.

Name

Type

Default

Description

Returns

Type: Dict[Dict[str, str], Dict[str, Union[numpy.ndarray, int, float]]]

`preprocess`

Data preprocessing before computing the drift scores.

Name

Type

Default

Description

Returns

Type: Tuple[numpy.ndarray, numpy.ndarray]

`score`

Compute the feature-wise drift score which is the p-value of the

statistical test and the test statistic.

Name

Type

Default

Description

Returns

Type: Tuple[numpy.ndarray, numpy.ndarray]