Production Environment

Install Core 2 in a production Kubernetes environment.

Prerequisites

• Set up and connect to a Kubernetes cluster running version 1.27 or later. For instructions on connecting to your Kubernetes cluster, refer to the documentation provided by your cloud provider.

• Install kubectl, the Kubernetes command-line tool.

• Install Helm, the package manager for Kubernetes.

To use Seldon Core 2 in a production environment:

1. Create namespaces

2. Install Seldon Core 2

Seldon publishes the Helm charts that are required to install Seldon Core 2. For more information about the Helm charts and the related dependencies, see Helm charts and Dependencies.

Creating Namespaces

• Create a namespace to contain the main components of Seldon Core 2. For example, create the namespace seldon-mesh:

  kubectl create ns seldon-mesh || echo "Namespace seldon-mesh already exists"

• Create a namespace to contain the components related to monitoring. For example, create the namespace seldon-monitoring:

  kubectl create ns seldon-monitoring || echo "Namespace seldon-monitoring already exists"

Installing Seldon Core 2

1. Add and update the Helm charts seldon-charts to the repository.

   helm repo add seldon-charts https://seldonio.github.io/helm-charts/
   helm repo update seldon-charts

2. Install custom resource definitions for Seldon Core 2.

   helm upgrade seldon-core-v2-crds seldon-charts/seldon-core-v2-crds \
   --namespace default \
   --install 

3. Install Seldon Core 2 operator in the seldon-mesh namespace.

    helm upgrade seldon-core-v2-setup seldon-charts/seldon-core-v2-setup \
    --namespace seldon-mesh --set controller.clusterwide=true \
    --install

   This configuration installs the Seldon Core 2 operator across an entire Kubernetes cluster. To perform cluster-wide operations, create ClusterRoles and ensure your user has the necessary permissions during deployment. With cluster-wide operations, you can create SeldonRuntimes in any namespace.

   You can configure the installation to deploy the Seldon Core 2 operator in a specific namespace so that it control resources in the provided namespace. To do this, set controller.clusterwide to false.

4. Install Seldon Core 2 runtimes in the namespace seldon-mesh.

   helm upgrade seldon-core-v2-runtime seldon-charts/seldon-core-v2-runtime \
   --namespace seldon-mesh \
   --install

5. Install Seldon Core 2 servers in the namespace seldon-mesh. Two example servers named mlserver-0, and triton-0 are installed so that you can load the models to these servers after installation.

    helm upgrade seldon-core-v2-servers seldon-charts/seldon-core-v2-servers \
    --namespace seldon-mesh \
    --install

6. Check Seldon Core 2 operator, runtimes, servers, and CRDS are installed in the namespace seldon-mesh:

    kubectl get pods -n seldon-mesh

   The output should be similar to this:

   NAME                                            READY   STATUS             RESTARTS      AGE
   hodometer-749d7c6875-4d4vw                      1/1     Running            0             4m33s
   mlserver-0                                      3/3     Running            0             4m10s
   seldon-dataflow-engine-7b98c76d67-v2ztq         0/1     CrashLoopBackOff   5 (49s ago)   4m33s
   seldon-envoy-bb99f6c6b-4mpjd                    1/1     Running            0             4m33s
   seldon-modelgateway-5c76c7695b-bhfj5            1/1     Running            0             4m34s
   seldon-pipelinegateway-584c7d95c-bs8c9          1/1     Running            0             4m34s
   seldon-scheduler-0                              1/1     Running            0             4m34s
   seldon-v2-controller-manager-5dd676c7b7-xq5sm   1/1     Running            0             4m52s
   triton-0                                        2/3     Running            0             4m10s

Note: Pods with names starting with seldon-dataflow-engine, seldon-pipelinegateway, and seldon-modelgateway may generate log errors until they successfully connect to Kafka. This occurs because Kafka is not yet fully integrated with Seldon Core 2.

Next steps

You can integrate Seldon Core 2 with Kafka that is self-hosted or a managed Kafka.

Additional Resources

• Seldon Enterprise Documentation

• GKE Documentation

• AWS Documentation

• Azure Documentation