Edit

Experiment version examples

This notebook will show how we can update running experiments.

Test change candidate for a model

We will use three SKlearn Iris classification models to illustrate experiment updates.

Load all models.

seldon model load -f ./models/sklearn1.yaml
seldon model load -f ./models/sklearn2.yaml
seldon model load -f ./models/sklearn3.yaml
{}
{}
{}

seldon model status iris -w ModelAvailable
seldon model status iris2 -w ModelAvailable
seldon model status iris3 -w ModelAvailable
{}
{}
{}

Let's call all three models individually first.

We will start an experiment to change the iris endpoint to split traffic with the iris2 model.

Now when we call the iris model we should see a roughly 50/50 split between the two models.

Now we update the experiment to change to a split with the iris3 model.

Now we should see a split with the iris3 model.

Now the experiment has been stopped we check everything as before.

Test change default model in an experiment

Here we test changing the model we want to split traffic on. We will use three SKlearn Iris classification models to illustrate.

Let's call all three models to verify initial conditions.

Now we start an experiment to change calls to the iris model to split with the iris2 model.

Run a set of calls and record which route the traffic took. There should be roughly a 50/50 split.

Now let's change the model we want to experiment to modify to the iris3 model. Splitting between that and iris2.

Let's check the iris model is now as before but the iris3 model has traffic split.

Finally, let's check now the experiment has stopped as is as at the start.

PreviousLocal experimentsNextTritonclient examples

Last updated

Was this helpful?