Edit

Kubeflow Seldon E2E Pipeline

In this example we showcase how to build re-usable components to build an ML pipeline that can be trained and deployed at scale.

We will automate content moderation on the Reddit comments in /r/science building a machine learning NLP model with the following components:

completed-pipeline-deploy

This tutorial will break down in the following sections:

  1. Test and build all our reusable pipeline steps

  2. Use Kubeflow to Train the Pipeline and Deploy to Seldon

  3. Test Seldon Deployed ML REST Endpoints

  4. Visualise Seldon's Production ML Pipelines

Before you start

Make sure you have the following components set-up and running in your Kubernetes cluster:

Let's get started! 🚀🔥 We will be building the end-to-end pipeline below:

kubeflow-seldon-nlp-full

1) Test and build all our reusable pipeline steps

We will start by building each of the components in our ML pipeline.

kubeflow-seldon-nlp-reusable-components

Let's first have a look at our clean_text step:

Like in this step, all of the other steps can be found in the pipeline/pipeline_steps/ folder, and all have the following structure:

  • pipeline_step.py which exposes the functionality through a CLI

  • Transformer.py which transforms the data accordingly

  • requirements.txt which states the python dependencies to run

  • build_image.sh which uses s2i to build the image with one line

Let's check out the CLI for clean_text

The pipeline_step CLI is the entry point for the kubeflow image as it will be able to pass any relevant parameters

This is actually a very simple file, as we are using the click library to define the commands:

The Transformer is where the data munging and transformation stage comes in, which will be wrapped by the container and exposed through the Seldon Engine to ensure our pipeline can be used in production.

Seldon provides multiple different features, such as abilities to send custom metrics, pre-process / post-process data and more. In this example we will only be exposing the predict step.

If you want to understand how the CLI pipeline talks to each other, have a look at the end to end test in pipeline/pipeline_tests/:

To build the image we provide a build script in each of the steps that contains the instructions:

The only thing you need to make sure is that Seldon knows how to wrap the right model and file.

This can be achieved with the s2i/environment file.

As you can see, here we just tell it we want it to use our Transformer.py file:

Once this is defined, the only thing we need to do is to run the build_image.sh for all the reusable components.

Here we show the manual way to do it:

3) Train our NLP Pipeline through the Kubeflow UI

We can access the Kubeflow dashboard to train our ML pipeline via http://localhost/_/pipeline-dashboard

If you can't edit this, you need to make sure that the ambassador gateway service is accessible:

In my case, I need to change the kind from NodePort into LoadBalancer which can be done with the following command:

Now that I've changed it to a loadbalancer, it has allocated the external IP as my localhost so I can access it at http://localhost/_/pipeline-dashboard

If this was successful, you should be able to access the dashboard kf-pipeline-dashboard

Define the pipeline

Now we want to generate the pipeline. For this we can use the DSL provided by kubeflow to define the actual steps required.

The pipeline will look as follows:

kf-seldon-nlp-ml-pipelines

Breaking down the code

As you can see in the DSL, we have the ContainerOp - each of those is a step in the Kubeflow pipeline.

At the end we can see the seldondeploy step which basically deploys the trained pipeline

The definition of the SeldonDeployment graph is provided in the deploy_pipeline/seldon_production_pipeline.yaml file.

The seldondeployment file defines our production execution graph using the same reusable components.

Seldon Production pipeline contents

If we look at the file we'll be using to deploy our pipeline, we can see that it has the following key points:

  1. Reusable components definitions as containerSpecs: cleantext, spacytokenizer, tfidfvectorizer & lrclassifier

  2. DAG (directed acyclic graph) definition for REST pipeline: cleantext -> spacytokenizer -> tfidfvectorizer -> lrclassifier

This graph in our production deployment looks as follows:

kf-seldon-npl-pipelines-deploy

Generate the pipeline files to upload to Kubeflow

To generate the pipeline we just have to run the pipeline file, which will output the tar.gz file that will be uploaded.

Run the pipeline

You can access the Kubeflow Pipelines UI by forwarding the port with the following command:

The UI should now be accessible via http://localhost:8000.

We now need to upload the resulting nlp_pipeline.py.tar.gz file generated.

This can be done through the "Upload PIpeline" button in the UI.

Once it's uploaded, we want to create and trigger a run! You should now be able to see how each step is executed:

running-pipeline

Inspecting the data created in the Persistent Volume

The pipeline saves the output of the pipeline together with the trained model in the persistent volume claim.

The persistent volume claim is the same name as the argo workflow:

Our workflow is there! So we can actually access it by running

And we can use good old sed to insert this workflow name in our PVC-Access controler which we can use to inspect the contents of the volume:

We just need to apply this container with our kubectl command, and we can use it to inspect the mounted folder:

Now we can run an ls command to see what's inside:

5) Test Deployed ML REST Endpoints

Now that it's running we have a production ML text pipeline that we can Query using REST and GRPC

First we can check if our Seldon deployment is running with

We will need the Seldon Pipeline Deployment name to reach the API, so we can get it using:

Now we can interact with our API in two ways:

  1. Using CURL or any client like PostMan

  2. Using the Python SeldonClient

Using CURL from the terminal

When using CURL, the only thing we need to provide is the data in JSON format, as well as the url, which is of the format:

Using the SeldonClient

We can also use the Python SeldonClient to interact with the pipeline we just deployed

6) Visualise Seldon's Production ML Pipelines

We can visualise the performance using the SeldonAnalytics package, which we can deploy using:

In my case, similar to what I did with Ambassador, I need to make sure the service is a LoadBalancer instead of a NodePort

Now we can access it at the port provided, in my case it is http://localhost:32445/d/3swM2iGWz/prediction-analytics?refresh=5s&orgId=1

(initial username is admin and password is password, which will be requested to be changed on the first login)

Generate a bunch of requests and visualise:

You now have a full end-to-end training and production NLP pipeline

seldon-analytics
PreviousTriton ExamplesNextH2O Java MoJo

Last updated

Was this helpful?