Deployment Models

Deployment models for Seldon Enterprise Platform

When it comes to deploying Seldon Enterprise Platform for a production workload, there are different deployment models that can apply depending on your use case. For example, you can consider the following questions:

• Do you want to support multiple environments (e.g. production, staging, etc.)? Will these be split across multiple clusters? Or across different namespaces?

• Are you planning on supporting multiple tenants in your infrastructure? How will you manage permissions for each one of them?

This section will showcase some of the most common deployment models to set up Seldon Enterprise Platform in your cluster.

Multiple environments

When you are managing machine learning models, a common use case is to test them first on some kind of test environment, isolated from production workloads. These test environments are usually known as staging, qa, pre-prod, etc.

In some cases, these environments will form a deployment (or continuous delivery) pipeline, where each environment is considered more stable than the previous one:

Dev >... > Staging > Production

Seldon Enterprise Platform allows you to have multiple environments by separating them as namespaces or clusters.

Regardless of the approach you choose, this doesn't restrict your options when it comes to syncing your models with a GitOps repository. In fact, you can still have:

• Single GitOps repository, with one folder per environment.

• Multiple GitOps repositories, with one repository per environment.

Namespace environments

The official Kubernetes docs define namespaces as virtual clusters. Therefore, it makes sense to treat each separate environment as a separate namespace. Following the example above, that would mean creating a dev, staging and production namespaces.

By default, Seldon Enterprise Platform will treat each namespace as a separate environment. Therefore, this is the suggested approach.

Out of the box, this gives you easy control over:

• Where is each model deployed (on which environments).

• Who is authorised to view or create models.

• Visibility over all your models across every environment from a single instance of Seldon Enterprise Platform.

Cluster environments

When you are dealing with multiple environments, it makes sense to keep them completely isolated of each other. That way, any cluster-wide effects will only affect one of them.

Using separate Kubernetes clusters for each of your environments allows you to achieve a higher-level of separation than namespaces. However, this separation also comes with a few caveats:

• All your infrastructure resources will need to be duplicated and managed across every environment (e.g. auth, ingress, logging, metrics, ...).

• Operations spanning multiple environments become more complicated (e.g. promotion of models between different environments).

This approach is usually recommended to test new versions of cluster-wide components at the infrastructure level. For example, a new version of Seldon Enterprise Platform or a new ingress system. However, due to the caveats listed above, we don't suggest this approach to segregate different versions of your models.

Having said so, you can still leverage the GitOps paradigm to communicate between different clusters.