Edit

Batch Processing with Argo Workflows and S3 / Minio

In this notebook we will dive into how you can run batch processing with Argo Workflows and Seldon Core.

Dependencies:

  • Seldon core installed as per the docs with an ingress

  • Minio running in your cluster to use as local (s3) object storage

  • Argo Workfklows installed in cluster (and argo CLI for commands)

Setup

Install Seldon Core

Use the notebook to set-up Seldon Core with Ambassador or Istio Ingress.

Note: If running with KIND you need to make sure do follow these steps as workaround to the /.../docker.sock known issue.

Set up Minio in your cluster

Use the notebook to set-up Minio in your cluster.

Create rclone configuration

In this example, our workflow stages responsible for pulling / pushing data to in-cluster MinIO S3 storage will use rclone CLI. In order to configure the CLI we will create a following secret:

%%writefile rclone-config.yaml
apiVersion: v1
kind: Secret
metadata:
  name: rclone-config-secret
type: Opaque
stringData:
  rclone.conf: |
    [cluster-minio]
    type = s3
    provider = minio
    env_auth = false
    access_key_id = minioadmin
    secret_access_key = minioadmin
    endpoint = http://minio.minio-system.svc.cluster.local:9000

Install Argo Workflows

You can follow the instructions from the official Argo Workflows Documentation.

You also need to make sure that argo has permissions to create seldon deployments - for this you can create a role:

A service account:

And a binding

Create some input for our model

We will create a file that will contain the inputs that will be sent to our model

Check the contents of the file

Upload the file to our minio

Create Argo Workflow

In order to create our argo workflow we have made it simple so you can leverage the power of the helm charts.

Before we dive into the contents of the full helm chart, let's first give it a try with some of the settings.

We will run a batch job that will set up a Seldon Deployment with 10 replicas and 100 batch client workers to send requests.

Check output in object store

We can now visualise the output that we obtained in the object store.

First we can check that the file is present:

Now we can output the contents of the file created using the mc head command.

PreviousTraining Outlier Detector for CIFAR10 with PoetryNextBatch Processing with Argo Workflows and HDFS

Last updated

Was this helpful?