RClone Storage Initializer - testing new secret format

In this documentation page we provide an example upgrade path from kfserving-based to rclone-based storage initializer. This is required due to the fact that secret format expected by these two storage initializers is different.

Storage initializers are used by Seldon's pre-packaged model servers to download models binaries. As it is explained in the SC 1.8 upgrading notes the seldonio/rclone-storage-initializer became default storage initializer in v1.8.0.

In this tutorial we will show how to upgrade your configuration to new Storage Initializer with focus on getting the new format of a required secret right.

Read more:

• Prepackaged Model Servers documentation page

• SC 1.8 upgrading notes

• Example upgrade path to use rclone-based storage initializer globally

Prerequisites

• A kubernetes cluster with kubectl configured

• mc client

• curl

Steps in this tutorial

• Copy iris model from GCS into in-cluster minio and configure old-style storage initializer secret

• Deploy SKlearn Pre-Packaged server using kfserving storage initializer

• Discuss upgrading procedure and tips how to test new secret format

• Deploy Pre-packaged model server using rclone storage initializer

Setup Seldon Core

Use the setup notebook to Setup Cluster with Ambassador Ingress and Install Seldon Core.

Setup MinIO

Use the provided notebook to install Minio in your cluster and configure mc CLI tool.

Copy iris model into local MinIO

%%bash
mc config host add gcs https://storage.googleapis.com "" "" 

mc mb minio-seldon/sklearn/iris/ -p
mc cp gcs/seldon-models/sklearn/iris/model.joblib minio-seldon/sklearn/iris/
mc cp gcs/seldon-models/sklearn/iris/metadata.yaml minio-seldon/sklearn/iris/

%%bash
mc ls minio-seldon/sklearn/iris/

Deploy SKLearn Server with kfserving-storage-initializer

First we deploy the model using kfserving-storage-initializer. This is using the default Storage Initializer for pre Seldon Core v1.8.0.

%%writefile sklearn-iris-kfserving.yaml

apiVersion: v1
kind: Secret
metadata:
  name: seldon-kfserving-secret
type: Opaque
stringData:
  AWS_ACCESS_KEY_ID: minioadmin
  AWS_SECRET_ACCESS_KEY: minioadmin
  AWS_ENDPOINT_URL: http://minio.minio-system.svc.cluster.local:9000
  USE_SSL: "false"
    
---
    
apiVersion: machinelearning.seldon.io/v1
kind: SeldonDeployment
metadata:
  name: sklearn-iris-kfserving
spec:
  predictors:
  - name: default
    replicas: 1
    graph:
      name: classifier
      implementation: SKLEARN_SERVER
      modelUri: s3://sklearn/iris
      envSecretRefName: seldon-kfserving-secret
      storageInitializerImage: kfserving/storage-initializer:v0.6.1

Overwriting sklearn-iris-kfserving.yaml

!kubectl apply -f sklearn-iris-kfserving.yaml

secret/seldon-kfserving-secret configured
seldondeployment.machinelearning.seldon.io/sklearn-iris-kfserving configured

!kubectl rollout status deploy/$(kubectl get deploy -l seldon-deployment-id=sklearn-iris-kfserving -o jsonpath='{.items[0].metadata.name}')

%%bash
curl -s -X POST -H 'Content-Type: application/json' \
    -d '{"data":{"ndarray":[[5.964, 4.006, 2.081, 1.031]]}}' \
    http://localhost:8003/seldon/seldon/sklearn-iris-kfserving/api/v1.0/predictions  | jq .

Preparing rclone-compatible secret

The rclone-based storage initializer expects one to define a new secret. General documentation credentials hadling can be found here with constantly updated examples of tested configurations.

If we do not have yet an example for Cloud Storage solution that you are using, please, consult the relevant page on RClone documentation.

Preparing seldon-rclone-secret

Knowing format of required format of the secret we can create it now

%%writefile seldon-rclone-secret.yaml
apiVersion: v1
kind: Secret
metadata:
  name: seldon-rclone-secret
type: Opaque
stringData:
  RCLONE_CONFIG_S3_TYPE: s3
  RCLONE_CONFIG_S3_PROVIDER: minio
  RCLONE_CONFIG_S3_ENV_AUTH: "false"
  RCLONE_CONFIG_S3_ACCESS_KEY_ID: minioadmin
  RCLONE_CONFIG_S3_SECRET_ACCESS_KEY: minioadmin
  RCLONE_CONFIG_S3_ENDPOINT: http://minio.minio-system.svc.cluster.local:9000

!kubectl apply -f seldon-rclone-secret.yaml

Testing seldon-rclone-secret

Before deploying SKLearn server one can test directly using the rclone-storage-initializer image

%%writefile rclone-pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: rclone-pod
spec:
  containers:
  - name: rclone
    image: seldonio/rclone-storage-initializer:1.19.0-dev
    command: [ "/bin/sh", "-c", "--", "sleep 3600"]
    envFrom:
    - secretRef:
        name: seldon-rclone-secret

!kubectl apply -f rclone-pod.yaml

! kubectl exec -it rclone-pod -- rclone ls s3:sklearn

! kubectl exec -it rclone-pod -- rclone copy s3:sklearn .

! kubectl exec -it rclone-pod -- sh -c "ls iris/"

Once we tested that secret format is correct we can delete the pod

!kubectl delete -f rclone-pod.yaml

Deploy SKLearn Server with rclone-storage-initializer

%%writefile sklearn-iris-rclone.yaml

apiVersion: machinelearning.seldon.io/v1
kind: SeldonDeployment
metadata:
  name: sklearn-iris-rclone
spec:
  predictors:
  - name: default
    replicas: 1
    graph:
      name: classifier
      implementation: SKLEARN_SERVER
      modelUri: s3://sklearn/iris
      envSecretRefName: seldon-rclone-secret
      storageInitializerImage: seldonio/rclone-storage-initializer:1.19.0-dev

!kubectl apply -f sklearn-iris-rclone.yaml

!kubectl rollout status deploy/$(kubectl get deploy -l seldon-deployment-id=sklearn-iris-rclone -o jsonpath='{.items[0].metadata.name}')

%%bash
curl -s -X POST -H 'Content-Type: application/json' \
    -d '{"data":{"ndarray":[[5.964, 4.006, 2.081, 1.031]]}}' \
    http://localhost:8003/seldon/seldon/sklearn-iris-rclone/api/v1.0/predictions  | jq .

Cleanup

%%bash
kubectl delete -f sklearn-iris-rclone.yaml
kubectl delete -f sklearn-iris-kfserving.yaml