Edit

Azure AKS Tensorflow Deep MNIST

In this example we will deploy a tensorflow MNIST model in the Azure Kubernetes Service (AKS).

This tutorial will break down in the following sections:

  1. Train a tensorflow model to predict mnist locally

  2. Containerise the tensorflow model with our docker utility

  3. Send some data to the docker model to test it

  4. Install and configure Azure tools to interact with your cluster

  5. Use the Azure tools to create and setup AKS cluster with Seldon

  6. Push and run docker image through the Azure Container Registry

  7. Test our Elastic Kubernetes deployment by sending some data

Let's get started! 🚀🔥

Dependencies:

  • Helm v3.0.0+

  • A Kubernetes cluster running v1.13 or above (minkube / docker-for-windows work well if enough RAM)

  • kubectl v1.14+

  • az CLI v2.0.66+

  • Python 3.6+

  • Python DEV requirements

1) Train a tensorflow model to predict mnist locally

We will load the mnist images, together with their labels, and then train a tensorflow model to predict the right labels

2) Containerise the tensorflow model with our docker utility

First you need to make sure that you have added the .s2i/environment configuration file in this folder with the following content:

Now we can build a docker image named "deep-mnist" with the tag 0.1

3) Send some data to the docker model to test it

We first run the docker image we just created as a container called "mnist_predictor"

Send some random features that conform to the contract

4) Install and configure Azure tools

First we install the azure cli - follow specific instructions at https://docs.microsoft.com/en-us/cli/azure/install-azure-cli?view=azure-cli-latest

Configure the azure CLI so it can talk to your server

(if you are getting issues, make sure you have the permmissions to create clusters)

You must run this through a terminal and follow the instructions:

Once you are logged in, we can create our cluster. Run the following command, it may take a while so feel free to get a ☕.

Once it's created we can authenticate our local kubectl to make sure we can talk to the azure cluster:

And now we can check that this has been successful by making sure that our kubectl context is pointing to the cluster:

Setup Seldon Core

Use the setup notebook to Setup Cluster with Ambassador Ingress and Install Seldon Core. Instructions also online.

Push docker image

In order for the EKS seldon deployment to access the image we just built, we need to push it to the Azure Container Registry (ACR) - you can check if it's been successfully created in the dashboard https://portal.azure.com/#blade/HubsExtension/BrowseResourceBlade/resourceType/Microsoft.ContainerRegistry%2Fregistries

If you have any issues please follow the official Azure documentation: https://docs.microsoft.com/en-us/azure/container-registry/container-registry-get-started-azure-cli

First we create a registry

Make sure you keep the loginServer value in the output dictionary as we'll use it below.

Make sure your local docker instance has access to the registry

Now prepare docker image

We need to first tag the docker image before we can push it.

NOTE: if you named your registry different make sure you change the value of seldoncontainerregistry.azurecr.io

And push the image

NOTE: if you named your registry different make sure you change the value of seldoncontainerregistry.azurecr.io

Running the Model

We will now run the model. As you can see we have a placeholder "REPLACE_FOR_IMAGE_AND_TAG", which we'll replace to point to our registry.

Let's first have a look at the file we'll be using to trigger the model:

Now let's trigger seldon to run the model.

Run the deployment in your cluster

NOTE: In order for this to work you need to make sure that your cluster has the permissions to pull the images. You can do this by:

  1. Go into the Azure Container Registry

  2. Select the SeldonContainerRegistry you created

  3. Click on "Add a role assignment"

  4. Select the AcrPull role

  5. Select service principle

  6. Find the SeldonCluster

  7. Wait until the role has been added

We basically have a yaml file, where we want to replace the value "REPLACE_FOR_IMAGE_AND_TAG" for the image you pushed

And let's check that it's been created.

You should see an image called "deep-mnist-single-model...".

We'll wait until STATUS changes from "ContainerCreating" to "Running"

Test the model

Now we can test the model, let's first find out what is the URL that we'll have to use:

We'll use a random example from our dataset

We can now add the URL above to send our request:

Let's visualise the probability for each label

It seems that it correctly predicted the number 7

PreviousAWS EKS Tensorflow Deep MNISTNextGKE with GPU Tensorflow Deep MNIST

Last updated

Was this helpful?