Production Environment

Install Core 2 in a production Kubernetes environment.

Prerequisites

Set up and connect to a Kubernetes cluster running version 1.27 or later. For instructions on connecting to your Kubernetes cluster, refer to the documentation provided by your cloud provider.
Install kubectl, the Kubernetes command-line tool.
Install Helm, the package manager for Kubernetes.

To use Seldon Core 2 in a production environment:

Seldon publishes the Helm charts that are required to install Seldon Core 2. For more information about the Helm charts and the related dependencies, see Helm charts and Dependencies.

Creating Namespaces

Create a namespace to contain the main components of Seldon Core 2. For example, create the namespace seldon-mesh:
```
kubectl create ns seldon-mesh || echo "Namespace seldon-mesh already exists"
```
Create a namespace to contain the components related to monitoring. For example, create the namespace seldon-monitoring:
```
kubectl create ns seldon-monitoring || echo "Namespace seldon-monitoring already exists"
```

Installing Seldon Core 2

Add and update the Helm charts seldon-charts to the repository.

helm repo add seldon-charts https://seldonio.github.io/helm-charts/
helm repo update seldon-charts

Install custom resource definitions for Seldon Core 2.

helm upgrade seldon-core-v2-crds seldon-charts/seldon-core-v2-crds \
--namespace default \
--install

Install Seldon Core 2 operator in the seldon-mesh namespace.
```
 helm upgrade seldon-core-v2-setup seldon-charts/seldon-core-v2-setup \
 --namespace seldon-mesh --set controller.clusterwide=true \
 --install
```
This configuration installs the Seldon Core 2 operator across an entire Kubernetes cluster. To perform cluster-wide operations, create ClusterRoles and ensure your user has the necessary permissions during deployment. With cluster-wide operations, you can create SeldonRuntimes in any namespace.
With cluster-wide installation, you can specify the namespaces to watch by setting controller.watchNamespaces to a comma-separated list of namespaces (e.g., {ns1, ns2}). This allows the Seldon Core 2 operator to monitor and manage resources in those namespaces.
You can also install multiple operators in different namespaces, and configure them to watch a disjoint set of namespaces. For example, you can install two operators in op-ns1 and op-ns2, and configure them to watch ns1, ns2 and ns3, ns4, respectively, using the following commands:
```
for ns in op-ns1 op-ns2 ns1 ns2 ns3 ns4; do kubectl create ns "$ns"; done
```
We now install the first operator in op-ns1 and configure it to watch ns1 and ns2:
```
helm upgrade seldon-core-v2-setup seldon-charts/seldon-core-v2-setup \
--namespace op-ns1 \
--set controller.clusterwide=true \
--set "controller.watchNamespaces={ns1,ns2}" \
--install
```
Next, we install the second operator in op-ns2 and configure it to watch ns3 and ns4:
```
helm upgrade seldon-core-v2-setup seldon-charts/seldon-core-v2-setup \
--namespace op-ns2 \
--set controller.clusterwide=true \
--set "controller.watchNamespaces={ns3,ns4}" \
--set controller.skipClusterRoleCreation=true \
--install
```
Note that the second operator is installed with skipClusterRoleCreation=true to avoid re-creating the ClusterRole and ClusterRoleBinding that were created by the first operator.
Finally, you can configure the installation to deploy the Seldon Core 2 operator in a specific namespace so that it control resources in the provided namespace. To do this, set controller.clusterwide to false.

Install Seldon Core 2 runtimes in the namespace seldon-mesh.

helm upgrade seldon-core-v2-runtime seldon-charts/seldon-core-v2-runtime \
--namespace seldon-mesh \
--install

Install Seldon Core 2 servers in the namespace seldon-mesh. Two example servers named mlserver-0, and triton-0 are installed so that you can load the models to these servers after installation.
```
 helm upgrade seldon-core-v2-servers seldon-charts/seldon-core-v2-servers \
 --namespace seldon-mesh \
 --install
```

Check Seldon Core 2 operator, runtimes, servers, and CRDS are installed in the namespace seldon-mesh:

 kubectl get pods -n seldon-mesh

The output should be similar to this:

NAME                                            READY   STATUS             RESTARTS      AGE
hodometer-749d7c6875-4d4vw                      1/1     Running            0             4m33s
mlserver-0                                      3/3     Running            0             4m10s
seldon-dataflow-engine-7b98c76d67-v2ztq         0/1     CrashLoopBackOff   5 (49s ago)   4m33s
seldon-envoy-bb99f6c6b-4mpjd                    1/1     Running            0             4m33s
seldon-modelgateway-5c76c7695b-bhfj5            1/1     Running            0             4m34s
seldon-pipelinegateway-584c7d95c-bs8c9          1/1     Running            0             4m34s
seldon-scheduler-0                              1/1     Running            0             4m34s
seldon-v2-controller-manager-5dd676c7b7-xq5sm   1/1     Running            0             4m52s
triton-0                                        2/3     Running            0             4m10s

Note: Pods with names starting with seldon-dataflow-engine, seldon-pipelinegateway, and seldon-modelgateway may generate log errors until they successfully connect to Kafka. This occurs because Kafka is not yet fully integrated with Seldon Core 2.

Next steps

You can integrate Seldon Core 2 with Kafka that is self-hosted or a managed Kafka.

Additional Resources

PreviousSelf-hosted Kafka NextKafka Integration

Last updated 1 month ago

Was this helpful?