Artifact versions

Note: The Seldon CLI allows you to view information about underlying Seldon resources and make changes to them through the scheduler in non-Kubernetes environments. However, it cannot modify underlying manifests within a Kubernetes cluster. Therefore, using the Seldon CLI for control plane operations in a Kubernetes environment is not recommended. For more details, see Seldon CLI.

Seldon V2 Kubernetes Multi Version Artifact Examples

We have a Triton model that has two version folders

Model 1 adds 10 to input, Model 2 multiples by 10 the input. The structure of the artifact repo is shown below:

config.pbtxt
1/model.py <add 10>
2/model.py <mul 10>

import os
os.environ["NAMESPACE"] = "seldon-mesh"

MESH_IP=!kubectl get svc seldon-mesh -n ${NAMESPACE} -o jsonpath='{.status.loadBalancer.ingress[0].ip}'
MESH_IP=MESH_IP[0]
import os
os.environ['MESH_IP'] = MESH_IP
MESH_IP

'172.19.255.1'

Model

cat ./models/multi-version-1.yaml

apiVersion: mlops.seldon.io/v1alpha1
kind: Model
metadata:
  name: math
spec:
  storageUri: "gs://seldon-models/scv2/samples/triton_23-03/multi-version"
  artifactVersion: 1
  requirements:
  - triton
  - python

kubectl apply -f ./models/multi-version-1.yaml -n ${NAMESPACE}

model.mlops.seldon.io/math created

kubectl wait --for condition=ready --timeout=300s model --all -n ${NAMESPACE}

model.mlops.seldon.io/math condition met

seldon model infer math --inference-mode grpc --inference-host ${MESH_IP}:80 \
  '{"model_name":"math","inputs":[{"name":"INPUT","contents":{"fp32_contents":[1,2,3,4]},"datatype":"FP32","shape":[4]}]}' | jq -M .

{
  "modelName": "math_1",
  "modelVersion": "1",
  "outputs": [
    {
      "name": "OUTPUT",
      "datatype": "FP32",
      "shape": [
        "4"
      ],
      "contents": {
        "fp32Contents": [
          11,
          12,
          13,
          14
        ]
      }
    }
  ]
}

cat ./models/multi-version-2.yaml

apiVersion: mlops.seldon.io/v1alpha1
kind: Model
metadata:
  name: math
spec:
  storageUri: "gs://seldon-models/scv2/samples/triton_23-03/multi-version"
  artifactVersion: 2
  requirements:
  - triton
  - python

kubectl apply -f ./models/multi-version-2.yaml -n ${NAMESPACE}

model.mlops.seldon.io/math configured

kubectl wait --for condition=ready --timeout=300s model --all -n ${NAMESPACE}

model.mlops.seldon.io/math condition met

seldon model infer math --inference-mode grpc --inference-host ${MESH_IP}:80 \
  '{"model_name":"math","inputs":[{"name":"INPUT","contents":{"fp32_contents":[1,2,3,4]},"datatype":"FP32","shape":[4]}]}' | jq -M .

{
  "modelName": "math_2",
  "modelVersion": "1",
  "outputs": [
    {
      "name": "OUTPUT",
      "datatype": "FP32",
      "shape": [
        "4"
      ],
      "contents": {
        "fp32Contents": [
          10,
          20,
          30,
          40
        ]
      }
    }
  ]
}

kubectl delete -f ./models/multi-version-1.yaml -n ${NAMESPACE}

model.mlops.seldon.io "math" deleted