Canary Promotion

This demo helps you learn about:

• Deploying a pretrained sklearn iris model

• Load testing the model

• Observing requests and metrics

• Deploying a canary XGBoost model

• Load testing the canary model

• Observing requests and metrics for both models

• Promoting the canary model

Iris Model

Iris is the genus of flower which contains three species, namely setosa, versicolor, and virginica. This demo is based on iris classification model based on flower properties like sepal length, sepal width, petal length, and petal width. The species are also the classes that are used for the classification.

Create a Seldon ML Pipeline

1. In the Overview page, select Create new deployment.

2. In the Deployment Creation Wizard, enter the deployment details as follows:

   • Name: iris-classifier

   • Namespace: seldon

   • Type: Seldon ML Pipeline

   

   Deployment details
3. Configure the default predictor as follows:

   • Runtime: Scikit Learn

   • Model URI: gs://seldon-models/scv2/samples/mlserver_1.6.0/iris-sklearn

   • Model Project: default

   • Storage Secret: (leave blank/none)

   

   Default predictor spec
4. Click Next for the remaining steps, then click Launch.

Afer your deployment is launched successfully, the status of the deployment reads Available in the Overview page.

Start Load Test

1. Click the deployment that you created. The Dashboard page for the deployment is displayed.

2. In the Requests Monitor section, click Start Load Test to start a load test.

3. In the Load Test Wizard configure the following details:

   • Connections(total): 1

   • Load Parameter: Duration(seconds)

   • Value: 120

   • Json payload:

     {
         "inputs": [
             {
                 "name": "predict",
                 "data": [
                     0.38606369295833043,
                     0.006894049558299753,
                     0.6104082981607108,
                     0.3958954239450676
                 ],
                 "datatype": "FP64",
                 "shape": [
                     1,
                     4
                 ]
             }
         ]
     }

   

   Load test

After you click Run Test it spawns a Kubernetes job that send continuous prediction requests for the specified seconds to the SKLearn model in the deployment.

Observe requests and metrics

After the load test has started, you can monitor the upcoming requests, their responses and metrics in the Requests page the deployment. If this doesn't work, consult the request logging docs section for debugging.

logs

You can also see core metrics in the Dashboard page.

metrics

Deploy a Canary model

The next step is to create an XGBoost canary model.

1. In the Dashboard of the deployment and click Add Canary.

2. In the Canary Configuration Wizard, configure the default predictor as follows:

   • Runtime: XGBoost

   • Model URI: gs://seldon-models/xgboost/iris

   • Model Project: default

   • Storage Secret: (leave blank/none)

   • Canary Traffic Percentage: 10

   

   Default predictor spec
3. Click Next for the remaining steps in the wizard and then click Launch.

When the canary model is launched successfully, the deployment remains in an Available status.

This creates a new canary deployment with the XGBoost model and 10% of the traffic is sent to this deployment.

Note: The deployment status represents the status of the main model. If the canary model is not successfully launched, click the warning icon to see the error message.

Load test the canary model

This time, create a new load test with the canary model running and observe the requests and metrics for both models. You can use either the same Json payload from the previous load test or construct a new one with different values or number of predictions.

Important: Remember that roughly 10% of the traffic will be sent to the canary model. If, however, the canary model is not available, all the traffic will be sent to the main model.

Observe requests and metrics for both models

After the load test has started, you can monitor the upcoming requests, their responses and metrics in the Requests page the deployment. If this doesn't work, consult the request logging docs section for debugging.

In order to see the requests for the canary model, you need to select the iris-classifier-canary predictor and the related Node in the Node Selector filter of the Requests page, as shown on the screenshot. Make a note that the number of requests for the canary model and it is 10% of the total number of requests, as was specified in the canary deployment.

logs

You can also see core metrics such as Live Requests, and Resource Monitor for both models in the Dashboard page.

metrics

Promote the Canary model

After you evaluated the performace of the canary model you can promote the canary model as a main model.

1. In the Dashboard page of the deployment and click Promote Canary.

2. In the Promote Canary dialog, click Confirm to promote the canary model to the main model.

When the canary model is promoted successfully, the deployment remains in an Available status.

metrics