Multi-tenant Environment
Guidance for multi-tenant clusters
Multi-tenant Guidance
We suggest running Seldon Enterprise Platform in cluster-wide mode. If the full stack of suggested components is used then this makes it simpler to scale by adding new namespaces dynamically for new departments/users.
If you are able to use cluster-wide permissions, you don't need to read this guide. Use one of the other install types.
The below is guidance on restricting the use of cluster-wide permissions.
Optional Components
Istio
Istio currently has very limited support for multi-tenancy.
There are alternative flavours of Istio. OpenShift Service Mesh has some multi-tenancy support.
An alternative is Ambassador, which can be installed namespaced.
If a non-default ingress setup is used then the requestForm and curlForm sections in the install-values.yaml file need to be configured with templates applicable to the ingress URL rules being used.
ArgoCD
From our testing, ArgoCD does require read permissions at a cluster-wide level on all objects.
Write permissions can however be entirely namespaced. Using this means configuring roles for each new namespace, as explained in the ArgoCD GitHub issue.
ArgoCD is only required in order to use GitOps. Seldon Enterprise Platform can be used without GitOps.
If using the Production Installation with GitOps then this script should be consulted to adapt the steps for a multi-tenant cluster. The main differences are:
The ArgoCD installation is different as it uses different roles from the manifests that ArgoCD publishes.
For each new namespace added to GitOps, we have to add roles for ArgoCD to be able to sync it.
For each new namespace a new instance of Seldon Core is installed, dedicated to that namespace.
For each new namespace new roles are added for Seldon Enterprise Platform to be able to operate on that namespace.
The details of these differences can be found in the script.
Seldon Core
Seldon Core supports a namespaced install with a singleNamespace setting in its Helm chart. There is an example in seldon-deploy-install/sd-setup/gitops-setup.sh.
MinIO
We suggest installing with the MinIO Helm chart. By default we setup one instance per cluster but that could be configured and at the time of writing ClusterRoles are used only on OpenShift
Argo Workflows
We suggest using Argo Workflows default installation which is cluster-wide but namespaced installations are possible.
Note that each Seldon namespace that runs batch jobs would need the ability to start Argo Workflows. Batch jobs are optional.
Knative
At the time of writing neither Knative Serving nor Knative Eventing seems to officially support multi-tenancy. See below.
Configuring Seldon Enterprise Platform
Some functionality depends on Knative. If Knative is not used then this affects the request logger configuration. (Not using Knative also means the outlier and drift detector demos won't work for Seldon Core v1 runtimes.)
Seldon Core v1's installation instructions contain a default URL that points to a Knative broker in the seldon-logs namespace (executor.requestLogger.defaultEndpoint). This can be changed to point to seldon-request-logger.seldon-logs (or your request logger location).
Detector component wizards and the Data Science metrics component wizard do need Knative.
The Seldon Enterprise Platform Helm chart has a rbac.clusterWide setting. There are two options from here.
Option 1 - One Platform, per-Namespace RBAC (recommended)
This is reduced RBAC rather than true multi-tenant. In this option, you will install a single instance of Seldon Enterprise Platform and use granular RBAC to allow it to control resources in other namespaces. Assuming that you would want to have your Seldon deployments in namespaces dev & prod you would want to set following Helm values:
Option 2 - Platform Instance Per Namespace
This is full multi-tenant using only namespaced roles.
If rbac.readNamespaces is false then a Enterprise Platform per namespace should be used. Each Enterprise Platform then looks only at its own namespace.
The request logger namespace is configurable. This could be a dedicated namespace per Enterprise Platform instance or the same namespace as Enterprise Platform.
Last updated