Edit

AWS EKS Tensorflow Deep MNIST

In this example we will deploy a tensorflow MNIST model in Amazon Web Services' Elastic Kubernetes Service (EKS).

This tutorial will break down in the following sections:

  1. Train a tensorflow model to predict mnist locally

  2. Containerise the tensorflow model with our docker utility

  3. Send some data to the docker model to test it

  4. Install and configure AWS tools to interact with AWS

  5. Use the AWS tools to create and setup EKS cluster with Seldon

  6. Push and run docker image through the AWS Container Registry

  7. Test our Elastic Kubernetes deployment by sending some data

Let's get started! 🚀🔥

Dependencies:

  • Helm v3.0.0+

  • A Kubernetes cluster running v1.13 or above (minkube / docker-for-windows work well if enough RAM)

  • kubectl v1.14+

  • EKS CLI v0.1.32

  • AWS Cli v1.16.163

  • Python 3.6+

  • Python DEV requirements

1) Train a tensorflow model to predict mnist locally

We will load the mnist images, together with their labels, and then train a tensorflow model to predict the right labels

2) Containerise the tensorflow model with our docker utility

First you need to make sure that you have added the .s2i/environment configuration file in this folder with the following content:

Now we can build a docker image named "deep-mnist" with the tag 0.1

3) Send some data to the docker model to test it

We first run the docker image we just created as a container called "mnist_predictor"

Send some random features that conform to the contract

png

4) Install and configure AWS tools to interact with AWS

First we install the awscli

Configure aws so it can talk to your server

(if you are getting issues, make sure you have the permmissions to create clusters)

Install EKCTL

IMPORTANT: These instructions are for linux Please follow the official installation of ekctl at: https://docs.aws.amazon.com/eks/latest/userguide/getting-started-eksctl.html

5) Use the AWS tools to create and setup EKS cluster with Seldon

In this example we will create a cluster with 2 nodes, with a minimum of 1 and a max of 3. You can tweak this accordingly.

If you want to check the status of the deployment you can go to AWS CloudFormation or to the EKS dashboard.

It will take 10-15 minutes (so feel free to go grab a ☕).

IMPORTANT: If you get errors in this step it is most probably IAM role access requirements, which requires you to discuss with your administrator.

Configure local kubectl

We want to now configure our local Kubectl so we can actually reach the cluster we've just created

And we can check if the context has been added to kubectl config (contexts are basically the different k8s cluster connections) You should be able to see the context as "...aws:eks:eu-west-1:27...". If it's not activated you can activate that context with kubectlt config set-context <CONTEXT_NAME>

Setup Seldon Core

Use the setup notebook to Setup Cluster with Ambassador Ingress and Install Seldon Core. Instructions also online.

Push docker image

In order for the EKS seldon deployment to access the image we just built, we need to push it to the Elastic Container Registry (ECR).

If you have any issues please follow the official AWS documentation: https://docs.aws.amazon.com/AmazonECR/latest/userguide/what-is-ecr.html

First we create a registry

You can run the following command, and then see the result at https://us-west-2.console.aws.amazon.com/ecr/repositories?#

Now prepare docker image

We need to first tag the docker image before we can push it

We now login to aws through docker so we can access the repository

And push the image

Make sure you add your AWS Account ID

Running the Model

We will now run the model.

Let's first have a look at the file we'll be using to trigger the model:

Now let's trigger seldon to run the model.

We basically have a yaml file, where we want to replace the value "REPLACE_FOR_IMAGE_AND_TAG" for the image you pushed

And let's check that it's been created.

You should see an image called "deep-mnist-single-model...".

We'll wait until STATUS changes from "ContainerCreating" to "Running"

Test the model

Now we can test the model, let's first find out what is the URL that we'll have to use:

We'll use a random example from our dataset

We can now add the URL above to send our request:

Let's visualise the probability for each label

It seems that it correctly predicted the number 7

PreviousAdvanced CPP Buildsystem OverrideNextAzure AKS Tensorflow Deep MNIST

Last updated

Was this helpful?