Edit

Alibaba Cloud Tensorflow Deep MNIST

In this example we will deploy a tensorflow MNIST model in the Alibaba Cloud Container Service for Kubernetes.

This tutorial will break down in the following sections:

  1. Train a tensorflow model to predict mnist locally

  2. Containerise the tensorflow model with our docker utility

  3. Test model locally with docker

  4. Set-up and configure Alibaba Cloud environment

  5. Deploy your model and visualise requests

Let's get started! 🚀🔥

Dependencies:

  • Helm v3.0.0+

  • kubectl v1.14+

  • Python 3.6+

  • Python DEV requirements

1) Train a tensorflow model to predict mnist locally

We will load the mnist images, together with their labels, and then train a tensorflow model to predict the right labels

2) Containerise the tensorflow model with our docker utility

Create a wrapper file that exposes the functionality through a predict function:

Define the dependencies for the wrapper in the requirements.txt:

You need to make sure that you have added the .s2i/environment configuration file in this folder with the following content:

Now we can build a docker image named "deep-mnist" with the tag 0.1

3) Test model locally with docker

We first run the docker image we just created as a container called "mnist_predictor"

Send some random features that conform to the contract

4) Set-up and configure Alibaba Cloud environment

4.1 create a managed kubernetes cluster

We need to first create a cluster in Alibaba Cloud - you should follow the following instructions (make sure you expose the cluster with an elastic IP by checking the tickbox): https://www.alibabacloud.com/help/doc-detail/95108.htm

You should follow up the instructions but the finished cluster should look as follows:

k8s-dashboard-2

4.2 Copy the kubectl configuration to access the cluster

Once you have the cluster created, you will be able to use your local kubectl by copying the configuration details on the overview page, and copy it to your ~/.kube/config

4.3 Create an Alibaba Container Registry to push the image

Finally we need to create a container registry repository by following this guide: https://www.alibabacloud.com/blog/up-and-running-with-alibaba-cloud-container-registry_593765

Setup Seldon Core

Use the setup notebook to Setup Cluster with Ambassador Ingress and Install Seldon Core. Instructions also online.

Finally we install the Seldon Analytics Package

4.5 Push docker image

We'll now make sure the image is accessible within the Kubernetes cluster by pushing it to the repo that we created in step 4.3. This should look as follows in your dashboard:

k8s-dashboard-1

To push the image we first tag it

And then we push it

5 - Deploy your model and visualise requests

IMPORTANT: Make sure you replace the URL for your repo in the format of:

  • registry-intl.[REPO].aliyuncs.com/[REGISTRY]/[REPO]:0.1

Run the deployment in your cluster

And let's check that it's been created.

Test the model

We'll use a random example from our dataset

png

First we need to find the URL that we'll use

You need to add it to the script in the next block

We can now add the URL above to send our request:

Let's see the predictions for each label

It seems that it correctly predicted the number 7

Finally let's visualise the metrics that seldon provides out of the box

For this we can access the URL with the command below, it will request an admin and password which by default are set to the following:

  • Username: admin

  • Password: admin

You will be able to access it at http://[URL]/d/ejAHFXIWz/prediction-analytics?orgId=1

The metrics include requests per second, as well as latency. You are able to add your own custom metrics, and try out other more complex deployments by following these guides

graf
PreviousGKE with GPU Tensorflow Deep MNISTNextTriton GPT2 Example Azure

Last updated

Was this helpful?