Production image classifier with drift and outlier monitoring
Run these examples from the samples/examples/image_classifier
folder.
CIFAR10 Image Classification Production Deployment
![](https://docs.seldon.ai/~gitbook/image?url=https%3A%2F%2F1538183904-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252Fn8ONqKsfp9k7WHp5OWSM%252Fuploads%252Fgit-blob-ea00266f1ec5bdc7c606cd861eae5d610ba4b723%252Fdemo.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=548cde4f&sv=2)
We show an image classifier (CIFAR10) with associated outlier and drift detectors using a Pipeline.
The model is a tensorflow CIFAR10 image classfier
The outlier detector is created from the CIFAR10 VAE Outlier example.
The drift detector is created from the CIFAR10 KS Drift example
Model Training (optional for notebook)
To run local training run the training notebook.
import requests
import json
from typing import Dict, List
import numpy as np
import os
import tensorflow as tf
from alibi_detect.utils.perturbation import apply_mask
from alibi_detect.datasets import fetch_cifar10c
import matplotlib.pyplot as plt
tf.keras.backend.clear_session()
2023-06-30 15:39:28.732453: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2023-06-30 15:39:28.732465: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
train, test = tf.keras.datasets.cifar10.load_data()
X_train, y_train = train
X_test, y_test = test
X_train = X_train.astype('float32') / 255
X_test = X_test.astype('float32') / 255
print(X_train.shape, y_train.shape, X_test.shape, y_test.shape)
classes = (
"plane",
"car",
"bird",
"cat",
"deer",
"dog",
"frog",
"horse",
"ship",
"truck",
)
(50000, 32, 32, 3) (50000, 1) (10000, 32, 32, 3) (10000, 1)
outliers = []
for idx in range(0,X_train.shape[0]):
X_mask, mask = apply_mask(X_train[idx].reshape(1, 32, 32, 3),
mask_size=(14,14),
n_masks=1,
channels=[0,1,2],
mask_type='normal',
noise_distr=(0,1),
clip_rng=(0,1))
outliers.append(X_mask)
X_outliers = np.vstack(outliers)
X_outliers.shape
(50000, 32, 32, 3)
corruption = ['brightness']
X_corr, y_corr = fetch_cifar10c(corruption=corruption, severity=5, return_X_y=True)
X_corr = X_corr.astype('float32') / 255
reqJson = json.loads('{"inputs":[{"name":"input_1","data":[],"datatype":"FP32","shape":[]}]}')
url = "http://0.0.0.0:9000/v2/models/model/infer"
def infer(resourceName: str, batchSz: int, requestType: str):
if requestType == "outlier":
rows = X_outliers[0:0+batchSz]
elif requestType == "drift":
rows = X_corr[0:0+batchSz]
else:
rows = X_train[0:0+batchSz]
for i in range(batchSz):
show(rows[i])
reqJson["inputs"][0]["data"] = rows.flatten().tolist()
reqJson["inputs"][0]["shape"] = [batchSz, 32, 32, 3]
headers = {"Content-Type": "application/json", "seldon-model":resourceName}
response_raw = requests.post(url, json=reqJson, headers=headers)
print(response_raw)
print(response_raw.json())
def show(X):
plt.imshow(X.reshape(32, 32, 3))
plt.axis("off")
plt.show()
Pipeline
cat ../../models/cifar10.yaml
echo "---"
cat ../../models/cifar10-outlier-detect.yaml
echo "---"
cat ../../models/cifar10-drift-detect.yaml
apiVersion: mlops.seldon.io/v1alpha1
kind: Model
metadata:
name: cifar10
spec:
storageUri: "gs://seldon-models/triton/tf_cifar10"
requirements:
- tensorflow
---
apiVersion: mlops.seldon.io/v1alpha1
kind: Model
metadata:
name: cifar10-outlier
spec:
storageUri: "gs://seldon-models/scv2/examples/mlserver_1.3.5/cifar10/outlier-detector"
requirements:
- mlserver
- alibi-detect
---
apiVersion: mlops.seldon.io/v1alpha1
kind: Model
metadata:
name: cifar10-drift
spec:
storageUri: "gs://seldon-models/scv2/examples/mlserver_1.3.5/cifar10/drift-detector"
requirements:
- mlserver
- alibi-detect
seldon model load -f ../../models/cifar10.yaml
seldon model load -f ../../models/cifar10-outlier-detect.yaml
seldon model load -f ../../models/cifar10-drift-detect.yaml
{}
{}
{}
seldon model status cifar10 -w ModelAvailable | jq .
seldon model status cifar10-outlier -w ModelAvailable | jq .
seldon model status cifar10-drift -w ModelAvailable | jq .
{}
{}
{}
cat ../../pipelines/cifar10.yaml
apiVersion: mlops.seldon.io/v1alpha1
kind: Pipeline
metadata:
name: cifar10-production
spec:
steps:
- name: cifar10
- name: cifar10-outlier
- name: cifar10-drift
batch:
size: 20
output:
steps:
- cifar10
- cifar10-outlier.outputs.is_outlier
seldon pipeline load -f ../../pipelines/cifar10.yaml
seldon pipeline status cifar10-production -w PipelineReady | jq -M .
{
"pipelineName": "cifar10-production",
"versions": [
{
"pipeline": {
"name": "cifar10-production",
"uid": "cifeii2ufmbc73e5insg",
"version": 1,
"steps": [
{
"name": "cifar10"
},
{
"name": "cifar10-drift",
"batch": {
"size": 20
}
},
{
"name": "cifar10-outlier"
}
],
"output": {
"steps": [
"cifar10.outputs",
"cifar10-outlier.outputs.is_outlier"
]
},
"kubernetesMeta": {}
},
"state": {
"pipelineVersion": 1,
"status": "PipelineReady",
"reason": "created pipeline",
"lastChangeTimestamp": "2023-06-30T14:40:09.047429817Z",
"modelsReady": true
}
}
]
}
infer("cifar10-production.pipeline",20, "normal")
![](https://docs.seldon.ai/~gitbook/image?url=https%3A%2F%2F1538183904-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252Fn8ONqKsfp9k7WHp5OWSM%252Fuploads%252Fgit-blob-de6689a032d8089ab901f00c1877de6d60ba45cd%252Finfer_14_0.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=ed53476e&sv=2)
![](https://docs.seldon.ai/~gitbook/image?url=https%3A%2F%2F1538183904-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252Fn8ONqKsfp9k7WHp5OWSM%252Fuploads%252Fgit-blob-869a1020762b1b1523c9a409d6b169e1d74698df%252Finfer_14_1.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=3a027899&sv=2)
![](https://docs.seldon.ai/~gitbook/image?url=https%3A%2F%2F1538183904-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252Fn8ONqKsfp9k7WHp5OWSM%252Fuploads%252Fgit-blob-4e06a6d47f698174591c2037290669c01aefdddc%252Finfer_14_2.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=95ae6b45&sv=2)
![](https://docs.seldon.ai/~gitbook/image?url=https%3A%2F%2F1538183904-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252Fn8ONqKsfp9k7WHp5OWSM%252Fuploads%252Fgit-blob-0aa9b5fbbfeed904026f0536243335102c2d3070%252Finfer_14_3.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=b8fad626&sv=2)
![](https://docs.seldon.ai/~gitbook/image?url=https%3A%2F%2F1538183904-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252Fn8ONqKsfp9k7WHp5OWSM%252Fuploads%252Fgit-blob-25b0c4514103f89da9af735fc16dbcee37320e2f%252Finfer_14_4.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=f3a2b4b4&sv=2)
![](https://docs.seldon.ai/~gitbook/image?url=https%3A%2F%2F1538183904-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252Fn8ONqKsfp9k7WHp5OWSM%252Fuploads%252Fgit-blob-8b8d60a5e97ad5cb6c4c6aa88c93a2011b7357fe%252Finfer_14_5.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=5d79cc50&sv=2)
![](https://docs.seldon.ai/~gitbook/image?url=https%3A%2F%2F1538183904-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252Fn8ONqKsfp9k7WHp5OWSM%252Fuploads%252Fgit-blob-91f639d4e7f3305be89a005c75d11990fde76a54%252Finfer_14_6.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=8e53cc22&sv=2)
![](https://docs.seldon.ai/~gitbook/image?url=https%3A%2F%2F1538183904-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252Fn8ONqKsfp9k7WHp5OWSM%252Fuploads%252Fgit-blob-e227fd10f71c9959fe1c5d6d6718970d3e32adf8%252Finfer_14_7.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=37c949a3&sv=2)
![](https://docs.seldon.ai/~gitbook/image?url=https%3A%2F%2F1538183904-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252Fn8ONqKsfp9k7WHp5OWSM%252Fuploads%252Fgit-blob-5b8c00bf6b8a8a5a40f68c884ed4ddb68ebaa5ff%252Finfer_14_8.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=6a897422&sv=2)
![](https://docs.seldon.ai/~gitbook/image?url=https%3A%2F%2F1538183904-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252Fn8ONqKsfp9k7WHp5OWSM%252Fuploads%252Fgit-blob-7e28ea2337a7ec1c72f40c3ee58cec0c190e99f7%252Finfer_14_9.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=d9030bfc&sv=2)
![](https://docs.seldon.ai/~gitbook/image?url=https%3A%2F%2F1538183904-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252Fn8ONqKsfp9k7WHp5OWSM%252Fuploads%252Fgit-blob-9bd58b25f64fcbf40d478ffe9e63fdb53394b90c%252Finfer_14_10.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=fd181ed0&sv=2)
![](https://docs.seldon.ai/~gitbook/image?url=https%3A%2F%2F1538183904-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252Fn8ONqKsfp9k7WHp5OWSM%252Fuploads%252Fgit-blob-d510a4581be36ad2cc27d646c2d6c6cba019cdf4%252Finfer_14_11.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=a87fb3d0&sv=2)
![](https://docs.seldon.ai/~gitbook/image?url=https%3A%2F%2F1538183904-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252Fn8ONqKsfp9k7WHp5OWSM%252Fuploads%252Fgit-blob-8cfb478b1035a62be7f6079cf6c09acd5c8a886f%252Finfer_14_12.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=59f4e38f&sv=2)
![](https://docs.seldon.ai/~gitbook/image?url=https%3A%2F%2F1538183904-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252Fn8ONqKsfp9k7WHp5OWSM%252Fuploads%252Fgit-blob-7f08ecbcfe9a52bfbd48b4806613b0b18f3dc1d1%252Finfer_14_13.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=f3100593&sv=2)
![](https://docs.seldon.ai/~gitbook/image?url=https%3A%2F%2F1538183904-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252Fn8ONqKsfp9k7WHp5OWSM%252Fuploads%252Fgit-blob-d76a1bf4a627b65bc96eb4a4b3388fbc0623eb49%252Finfer_14_14.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=77f102b9&sv=2)
![](https://docs.seldon.ai/~gitbook/image?url=https%3A%2F%2F1538183904-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252Fn8ONqKsfp9k7WHp5OWSM%252Fuploads%252Fgit-blob-cec0f94e00790d4ae882f00cf19f0397cc3af09a%252Finfer_14_15.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=9d67306d&sv=2)
![](https://docs.seldon.ai/~gitbook/image?url=https%3A%2F%2F1538183904-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252Fn8ONqKsfp9k7WHp5OWSM%252Fuploads%252Fgit-blob-400c5372d5248ded57e135852704461aa0b1c9e0%252Finfer_14_16.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=9861b620&sv=2)
![](https://docs.seldon.ai/~gitbook/image?url=https%3A%2F%2F1538183904-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252Fn8ONqKsfp9k7WHp5OWSM%252Fuploads%252Fgit-blob-7787ea560423642c9fb65bbee7bb1d0640dc7b84%252Finfer_14_17.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=21a230de&sv=2)
![](https://docs.seldon.ai/~gitbook/image?url=https%3A%2F%2F1538183904-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252Fn8ONqKsfp9k7WHp5OWSM%252Fuploads%252Fgit-blob-58d829cd59f599ef7ed57f6012d44e05578f545a%252Finfer_14_18.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=ece7645a&sv=2)
![](https://docs.seldon.ai/~gitbook/image?url=https%3A%2F%2F1538183904-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252Fn8ONqKsfp9k7WHp5OWSM%252Fuploads%252Fgit-blob-05e12db341e861980feac35d81a01d7491d9fa48%252Finfer_14_19.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=d1ccd793&sv=2)
<Response [200]>
{'model_name': '', 'outputs': [{'data': [1.45001495e-08, 1.2525752e-09, 1.6298458e-07, 0.11529388, 1.7431412e-07, 6.1856604e-06, 0.8846994, 6.0739285e-09, 7.437921e-08, 4.7317337e-09, 1.26449e-06, 4.8814868e-09, 1.5153439e-09, 8.490656e-09, 5.5131194e-10, 1.1617216e-09, 5.7729294e-10, 2.8839776e-07, 0.0006149016, 0.99938357, 0.888746, 2.5331951e-06, 0.00012967695, 0.10531583, 2.4284174e-05, 6.3332986e-06, 0.0016261435, 1.13079e-05, 0.0013286703, 0.0028091935, 2.0993439e-06, 3.680449e-08, 0.0013269952, 2.1766558e-05, 0.99841356, 0.00015300694, 6.9472035e-06, 1.3277059e-05, 6.1860555e-05, 3.4072806e-07, 1.1205097e-05, 0.99997175, 1.9948227e-07, 6.9880834e-08, 3.3387135e-08, 5.2603138e-08, 3.0352305e-07, 4.3738982e-08, 5.3243946e-07, 1.5870584e-05, 0.0006525102, 0.013322109, 1.480307e-06, 0.9766325, 4.9847167e-05, 0.00058075984, 0.008405659, 5.2234273e-06, 0.00023390084, 0.000116047224, 1.6682397e-06, 5.7737526e-10, 0.9975605, 6.45564e-05, 0.002371972, 1.0392675e-07, 9.747962e-08, 1.4484569e-07, 8.762438e-07, 2.4758325e-08, 5.028761e-09, 6.856381e-11, 5.9932094e-12, 4.921233e-10, 1.471166e-07, 2.7940719e-06, 3.4563383e-09, 0.99999714, 5.9420524e-10, 9.445026e-11, 4.1854888e-05, 5.041549e-08, 8.0302314e-08, 1.2119854e-07, 6.781646e-09, 1.2616152e-08, 1.1878505e-08, 1.628573e-09, 0.9999578, 3.281738e-08, 0.08930307, 1.4065135e-07, 4.1117343e-07, 0.90898305, 8.933351e-07, 0.0015637449, 0.00013868928, 9.092981e-06, 4.8759745e-07, 4.3976044e-07, 0.00016094849, 3.5653954e-07, 0.0760521, 0.8927447, 0.0011777573, 0.00265573, 0.027189083, 4.1892267e-06, 1.329405e-05, 1.8564688e-06, 1.3373891e-06, 1.0251247e-07, 8.651912e-09, 4.458202e-06, 1.4646349e-05, 1.260957e-06, 1.046087e-08, 0.9998946, 8.332438e-05, 3.900894e-07, 6.53852e-05, 3.012202e-08, 1.0247197e-07, 1.8824371e-06, 0.0004958526, 3.533475e-05, 2.739997e-07, 0.99939275, 4.840305e-06, 3.5346695e-06, 0.0005518078, 3.1597017e-07, 0.99902296, 0.00031509742, 8.07886e-07, 1.6366084e-06, 2.795575e-06, 6.112367e-06, 9.817249e-05, 2.602709e-07, 0.0004561966, 5.360607e-06, 2.8656412e-05, 0.000116040654, 6.881144e-05, 8.844774e-06, 4.4655946e-05, 3.5564542e-05, 0.006564381, 0.9926715, 0.007300911, 1.766928e-06, 3.0520596e-07, 0.026906287, 1.3769699e-06, 0.00027539674, 5.583593e-06, 3.792553e-06, 0.0003876767, 0.9651169, 0.18114138, 2.8360228e-05, 0.00019927241, 0.007685872, 0.00014663498, 3.9361137e-05, 5.941682e-05, 7.36174e-05, 0.79936546, 0.01126067, 2.3992783e-11, 7.6336457e-16, 1.4644799e-15, 1, 2.4652159e-14, 1.1786078e-10, 1.9402116e-13, 4.2408636e-15, 1.209294e-15, 2.9042784e-15, 1.5366902e-08, 1.2476195e-09, 1.3560152e-07, 0.999997, 4.3113017e-11, 2.8163534e-08, 2.4494727e-06, 1.3122828e-10, 3.8081083e-07, 2.1628158e-11, 0.0004926238, 6.9424555e-06, 2.827196e-05, 0.92534137, 9.500486e-06, 0.00036133997, 0.072713904, 1.2831057e-07, 0.0010457055, 2.8514464e-07], 'name': 'fc10', 'shape': [20, 10], 'datatype': 'FP32'}, {'data': [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], 'name': 'is_outlier', 'shape': [1, 20], 'datatype': 'INT64', 'parameters': {'content_type': 'np'}}]}
seldon pipeline inspect cifar10-production.cifar10-drift.outputs.is_drift
seldon.default.model.cifar10-drift.outputs cifeij8fh5ss738i5bp0 {"name":"is_drift", "datatype":"INT64", "shape":["1", "1"], "parameters":{"content_type":{"stringParam":"np"}}, "contents":{"int64Contents":["0"]}}
infer("cifar10-production.pipeline",20, "drift")
![](https://docs.seldon.ai/~gitbook/image?url=https%3A%2F%2F1538183904-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252Fn8ONqKsfp9k7WHp5OWSM%252Fuploads%252Fgit-blob-6e170ef4c5309fb0fb374f860781f0c1a9743129%252Finfer_16_0.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=de0170fe&sv=2)
![](https://docs.seldon.ai/~gitbook/image?url=https%3A%2F%2F1538183904-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252Fn8ONqKsfp9k7WHp5OWSM%252Fuploads%252Fgit-blob-34d762c3516b8b95d2df08169358a73e06fa9955%252Finfer_16_1.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=f16cf9ac&sv=2)
![](https://docs.seldon.ai/~gitbook/image?url=https%3A%2F%2F1538183904-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252Fn8ONqKsfp9k7WHp5OWSM%252Fuploads%252Fgit-blob-c2a5ee6e64e7e8ef0875d05dce6feacd1f8f087d%252Finfer_16_2.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=7e92195a&sv=2)
![](https://docs.seldon.ai/~gitbook/image?url=https%3A%2F%2F1538183904-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252Fn8ONqKsfp9k7WHp5OWSM%252Fuploads%252Fgit-blob-b186464c36cf9c518ac94dd5166f2a7c73bcf9c9%252Finfer_16_3.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=e8c83cc1&sv=2)
![](https://docs.seldon.ai/~gitbook/image?url=https%3A%2F%2F1538183904-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252Fn8ONqKsfp9k7WHp5OWSM%252Fuploads%252Fgit-blob-530643f1f2d9f680066e41b15e94a0a5f163f397%252Finfer_16_4.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=a7323f19&sv=2)
![](https://docs.seldon.ai/~gitbook/image?url=https%3A%2F%2F1538183904-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252Fn8ONqKsfp9k7WHp5OWSM%252Fuploads%252Fgit-blob-70ccae38bb5b5615536ed336c49be90e8e9982a7%252Finfer_16_5.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=a7aab106&sv=2)
![](https://docs.seldon.ai/~gitbook/image?url=https%3A%2F%2F1538183904-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252Fn8ONqKsfp9k7WHp5OWSM%252Fuploads%252Fgit-blob-3d09f45efff57e2945e2a8d027b905d53153c6a3%252Finfer_16_6.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=c605adb2&sv=2)
![](https://docs.seldon.ai/~gitbook/image?url=https%3A%2F%2F1538183904-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252Fn8ONqKsfp9k7WHp5OWSM%252Fuploads%252Fgit-blob-b9ca7a7aa47e339c1008a4eaf4d229f1aa50c1c6%252Finfer_16_7.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=a0d2d8f9&sv=2)
![](https://docs.seldon.ai/~gitbook/image?url=https%3A%2F%2F1538183904-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252Fn8ONqKsfp9k7WHp5OWSM%252Fuploads%252Fgit-blob-d978a20389556ace2a5281e55f68f212e8ceb1c9%252Finfer_16_8.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=5e383b5f&sv=2)
![](https://docs.seldon.ai/~gitbook/image?url=https%3A%2F%2F1538183904-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252Fn8ONqKsfp9k7WHp5OWSM%252Fuploads%252Fgit-blob-a662f3752549e57fab82df49214b94248ad55c11%252Finfer_16_9.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=17c70964&sv=2)
![](https://docs.seldon.ai/~gitbook/image?url=https%3A%2F%2F1538183904-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252Fn8ONqKsfp9k7WHp5OWSM%252Fuploads%252Fgit-blob-fe904ff7b005170dee9f395d285b4f1a0e589574%252Finfer_16_10.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=dc86a109&sv=2)
![](https://docs.seldon.ai/~gitbook/image?url=https%3A%2F%2F1538183904-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252Fn8ONqKsfp9k7WHp5OWSM%252Fuploads%252Fgit-blob-79d3db57418045f10509841aa924e30754b6efa8%252Finfer_16_11.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=1aec70ca&sv=2)
![](https://docs.seldon.ai/~gitbook/image?url=https%3A%2F%2F1538183904-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252Fn8ONqKsfp9k7WHp5OWSM%252Fuploads%252Fgit-blob-4f397d151f28990351a74718e917f5bdcb23dbd9%252Finfer_16_12.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=8fcdc7bc&sv=2)
![](https://docs.seldon.ai/~gitbook/image?url=https%3A%2F%2F1538183904-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252Fn8ONqKsfp9k7WHp5OWSM%252Fuploads%252Fgit-blob-3a1457846a0184be600c2fd031905a638db0027f%252Finfer_16_13.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=d684ee0a&sv=2)
![](https://docs.seldon.ai/~gitbook/image?url=https%3A%2F%2F1538183904-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252Fn8ONqKsfp9k7WHp5OWSM%252Fuploads%252Fgit-blob-97a1e8e013c11ab4828e5c07a2cb5b8a85d82594%252Finfer_16_14.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=ac091958&sv=2)
![](https://docs.seldon.ai/~gitbook/image?url=https%3A%2F%2F1538183904-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252Fn8ONqKsfp9k7WHp5OWSM%252Fuploads%252Fgit-blob-1d043b9aa03d1836b06e1299d5deff3598a455d5%252Finfer_16_15.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=a2c1cf2b&sv=2)
![](https://docs.seldon.ai/~gitbook/image?url=https%3A%2F%2F1538183904-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252Fn8ONqKsfp9k7WHp5OWSM%252Fuploads%252Fgit-blob-838a9e3f1a8c2451b662adf255427957bdd4a967%252Finfer_16_16.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=62f5a845&sv=2)
![](https://docs.seldon.ai/~gitbook/image?url=https%3A%2F%2F1538183904-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252Fn8ONqKsfp9k7WHp5OWSM%252Fuploads%252Fgit-blob-ed9ca0616c7696023d6b4c4faed22f45da5388b1%252Finfer_16_17.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=23f84cbd&sv=2)
![](https://docs.seldon.ai/~gitbook/image?url=https%3A%2F%2F1538183904-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252Fn8ONqKsfp9k7WHp5OWSM%252Fuploads%252Fgit-blob-3b220edde5096c43751e0db25da33078326a287f%252Finfer_16_18.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=8d1ea32e&sv=2)
![](https://docs.seldon.ai/~gitbook/image?url=https%3A%2F%2F1538183904-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252Fn8ONqKsfp9k7WHp5OWSM%252Fuploads%252Fgit-blob-1443fbf450179c7fbbe18677ac80839f865413de%252Finfer_16_19.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=bc3756fa&sv=2)
<Response [200]>
{'model_name': '', 'outputs': [{'data': [8.080701e-09, 2.3025173e-12, 2.2681688e-09, 1, 4.1828953e-11, 4.48467e-09, 3.216822e-08, 2.8404365e-13, 5.217064e-09, 3.3497323e-13, 0.96965235, 4.7030144e-06, 1.6964266e-07, 1.7355454e-05, 2.6667e-06, 1.9505828e-06, 1.1363079e-07, 3.3352034e-08, 0.030320557, 1.7086056e-07, 0.03725602, 6.8623276e-06, 7.5557014e-05, 0.00018132397, 2.2838503e-05, 0.000110639296, 2.3732607e-06, 2.1210687e-06, 0.9623351, 7.131072e-06, 0.999079, 4.207448e-09, 1.5788535e-08, 2.723756e-08, 2.6555508e-11, 2.1526697e-10, 2.7599315e-10, 2.0737433e-10, 0.0009210062, 3.0885383e-09, 6.665241e-07, 1.7765576e-09, 1.4911559e-07, 0.9765331, 1.9476123e-07, 2.8244015e-06, 0.023463126, 5.8030287e-09, 3.243206e-09, 1.12179785e-08, 4.4123663e-06, 4.7628927e-09, 1.1727273e-08, 0.9761534, 1.1409252e-08, 8.922882e-05, 0.023752932, 3.1563903e-08, 2.7916305e-09, 8.7746266e-10, 1.0166265e-05, 0.999703, 4.5408615e-05, 0.00022673907, 1.7365853e-07, 1.0147362e-06, 6.253448e-06, 2.9711526e-07, 7.811687e-07, 6.183683e-06, 0.86618125, 5.47548e-07, 0.00038408802, 0.013155022, 3.6916779e-06, 0.0006137024, 0.11965008, 3.6425424e-06, 6.7638084e-06, 1.2372367e-06, 1.9545263e-05, 1.1281859e-13, 1.6811868e-14, 0.9999777, 1.9805435e-11, 2.7563674e-06, 2.9651657e-09, 1.1363432e-12, 2.9902746e-13, 1.220973e-12, 2.9895918e-05, 3.4964305e-07, 1.1331837e-08, 1.7012125e-06, 3.6088227e-07, 3.035954e-08, 2.2102333e-06, 1.7414077e-08, 0.9999455, 1.9921794e-05, 0.9999999, 5.3446598e-11, 6.3188843e-10, 1.0956511e-07, 1.1538642e-10, 8.113561e-10, 4.7179572e-08, 1.4544753e-11, 5.490219e-08, 1.3347151e-10, 1.5363307e-07, 6.604881e-09, 2.424105e-10, 9.963063e-09, 3.9349533e-09, 1.5709017e-09, 7.705774e-10, 4.8085802e-08, 1.8885139e-05, 0.9999809, 7.147243e-08, 3.143131e-13, 2.1447092e-13, 0.00042652222, 6.945973e-12, 0.9995734, 6.174434e-09, 4.1128205e-11, 3.4031404e-13, 8.573159e-15, 1.2226405e-09, 2.3768018e-10, 2.822187e-07, 8.016278e-08, 4.0692296e-08, 6.8023346e-06, 2.3926754e-07, 0.9999925, 6.652648e-09, 7.743497e-09, 7.6360675e-06, 5.9386625e-09, 1.5675019e-09, 2.136716e-07, 1.3074002e-06, 3.700079e-10, 1.0984521e-09, 6.2138824e-08, 0.9609078, 0.03908287, 0.0008332255, 7.696685e-08, 2.4428939e-09, 7.186676e-05, 1.4520063e-09, 1.4521317e-08, 1.09093e-06, 1.2531165e-10, 0.9990938, 5.798501e-09, 5.785368e-05, 3.82365e-09, 7.404351e-08, 0.008338481, 8.048078e-10, 0.99157715, 1.1663455e-05, 1.4583546e-05, 8.3543476e-08, 3.274394e-08, 2.4682688e-05, 1.3951502e-09, 1.0260489e-08, 0.9998845, 1.9418138e-08, 8.667954e-07, 2.1851054e-07, 8.917964e-05, 4.4437223e-07, 1.1292918e-07, 4.5302792e-07, 5.631744e-08, 2.9086214e-08, 3.1013877e-07, 7.695681e-09, 2.1452344e-09, 1.1493902e-08, 6.1980093e-10, 0.99999917, 1.1436694e-08, 2.42685e-05, 8.557389e-08, 0.024081504, 0.0073837163, 4.8152968e-05, 5.128531e-07, 0.9684405, 9.630179e-08, 2.1060101e-05, 1.901065e-07], 'name': 'fc10', 'shape': [20, 10], 'datatype': 'FP32'}, {'data': [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], 'name': 'is_outlier', 'shape': [1, 20], 'datatype': 'INT64', 'parameters': {'content_type': 'np'}}]}
seldon pipeline inspect cifar10-production.cifar10-drift.outputs.is_drift
seldon.default.model.cifar10-drift.outputs cifeimgfh5ss738i5bpg {"name":"is_drift", "datatype":"INT64", "shape":["1", "1"], "parameters":{"content_type":{"stringParam":"np"}}, "contents":{"int64Contents":["1"]}}
infer("cifar10-production.pipeline",1, "outlier")
![](https://docs.seldon.ai/~gitbook/image?url=https%3A%2F%2F1538183904-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252Fn8ONqKsfp9k7WHp5OWSM%252Fuploads%252Fgit-blob-c83c7f42238be8167475d9ac36f5fb79466628f7%252Finfer_18_0.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=3f4d13a&sv=2)
<Response [200]>
{'model_name': '', 'outputs': [{'data': [6.3606867e-06, 0.0006106364, 0.0054279356, 0.6536454, 1.4738829e-05, 2.6104701e-06, 0.3397848, 1.3538776e-05, 0.0004458526, 4.807229e-05], 'name': 'fc10', 'shape': [1, 10], 'datatype': 'FP32'}, {'data': [1], 'name': 'is_outlier', 'shape': [1, 1], 'datatype': 'INT64', 'parameters': {'content_type': 'np'}}]}
infer("cifar10-production.pipeline",1, "ok")
![](https://docs.seldon.ai/~gitbook/image?url=https%3A%2F%2F1538183904-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252Fn8ONqKsfp9k7WHp5OWSM%252Fuploads%252Fgit-blob-de6689a032d8089ab901f00c1877de6d60ba45cd%252Finfer_19_0.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=e20cc655&sv=2)
<Response [200]>
{'model_name': '', 'outputs': [{'data': [1.45001495e-08, 1.2525752e-09, 1.6298458e-07, 0.11529388, 1.7431412e-07, 6.1856604e-06, 0.8846994, 6.0739285e-09, 7.43792e-08, 4.7317337e-09], 'name': 'fc10', 'shape': [1, 10], 'datatype': 'FP32'}, {'data': [0], 'name': 'is_outlier', 'shape': [1, 1], 'datatype': 'INT64', 'parameters': {'content_type': 'np'}}]}
Use the seldon CLI to look at the outputs from the CIFAR10 model. It will decode the Triton binary outputs for us.
seldon pipeline inspect cifar10-production.cifar10.outputs
seldon.default.model.cifar10.outputs cifeiq8fh5ss738i5bqg {"modelName":"cifar10_1", "modelVersion":"1", "outputs":[{"name":"fc10", "datatype":"FP32", "shape":["1", "10"], "contents":{"fp32Contents":[1.45001495e-8, 1.2525752e-9, 1.6298458e-7, 0.11529388, 1.7431412e-7, 0.0000061856604, 0.8846994, 6.0739285e-9, 7.43792e-8, 4.7317337e-9]}}]}
seldon pipeline unload cifar10-production
seldon model unload cifar10
seldon model unload cifar10-outlier
seldon model unload cifar10-drift
PreviousHuggingface speech to sentiment with explanations pipelineNextProduction income classifier with drift, outlier and explanations
Last updated
Was this helpful?