Install Seldon Core 2 in a local learning environment.
You can install Seldon Core 2 on your local computer that is running a Kubernetes cluster using kind.
Seldon publishes the Helm charts that are required to install Seldon Core 2. For more information about the Helm charts and the related dependencies,see Helm charts and Dependencies.
Install a Kubernetes cluster that is running version 1.27 or later.
Install , the Kubernetes command-line tool.
Install , the package manager for Kubernetes or , the automation tool used for provisioning, configuration management, and application deployment.
Create a namespace to contain the main components of Seldon Core 2. For example, create the seldon-mesh namespace.
Add and update the Helm charts, seldon-charts, to the repository.
Install Custom resource definitions for Seldon Core 2.
If you installed Seldon Core 2 using Helm, you need to complete the installation of other components in the following order:
seldon-mesh namespace.This configuration installs the Seldon Core 2 operator across an entire Kubernetes cluster. To perform cluster-wide operations, create ClusterRoles and ensure your user has the necessary permissions during deployment. With cluster-wide operations, you can create SeldonRuntimes in any namespace.
You can configure the installation to deploy the Seldon Core 2 operator in a specific namespace so that it control resources in the provided namespace. To do this, set controller.clusterwide to false.
Install Seldon Core 2 runtimes in the seldon-mesh namespace.
helm upgrade seldon-core-v2-runtime seldon-charts/seldon-core-v2-runtime \
--namespace seldon-mesh \
--installInstall Seldon Core 2 servers in the seldon-mesh namespace. Two example servers named mlserver-0, and triton-0 are installed so that you can load the models to these servers after installation.
helm upgrade seldon-core-v2-servers seldon-charts/seldon-core-v2-servers \
--namespace seldon-mesh \
--installCheck Seldon Core 2 operator, runtimes, servers, and CRDS are installed in the seldon-mesh namespace. It might take a couple of minutes for all the Pods to be ready. To check the status of the Pods in real time use this command: kubectl get pods -w -n seldon-mesh.
kubectl get pods -n seldon-meshThe output should be similar to this:
NAME READY STATUS RESTARTS AGE
hodometer-749d7c6875-4d4vw 1/1 Running 0 4m33s
mlserver-0 3/3 Running 0 4m10s
seldon-dataflow-engine-7b98c76d67-v2ztq 0/1 CrashLoopBackOff 5 (49s ago) 4m33s
seldon-envoy-bb99f6c6b-4mpjd 1/1 Running 0 4m33s
seldon-modelgateway-5c76c7695b-bhfj5 1/1 Running 0 4m34s
seldon-pipelinegateway-584c7d95c-bs8c9 1/1 Running 0 4m34s
seldon-scheduler-0 1/1 Running 0 4m34s
seldon-v2-controller-manager-5dd676c7b7-xq5sm 1/1 Running 0 4m52s
triton-0 2/3 Running 0 4m10sYou can install Seldon Core 2 and its components using Ansible in one of the following methods:
To install Seldon Core 2 into a new local kind Kubernetes cluster, you can use the seldon-all playbook with a single command:
This creates a kind cluster and installs ecosystem dependencies such kafka,
Prometheus, OpenTelemetry, and Jaeger as well as all the seldon-specific components.
The seldon components are installed using helm-charts from the current git
checkout (../k8s/helm-charts/).
Internally this runs, in order, the following playbooks:
kind-cluster.yaml
setup-ecosystem.yaml
setup-seldon.yaml
You may pass any of the additonal variables which are configurable for those playbooks to seldon-all.
For example:
Running the playbooks individually gives you more control over what and when it runs. For example, if you want to install into an existing k8s cluster.
Create a kind cluster.
Setup ecosystem.
Seldon runs by default in the seldon-mesh namespace and a Jaeger pod and OpenTelemetry collector are installed in the namespace.
To install in a different <mynamespace> namespace:
Install Seldon Core 2 in the ansible/ folder.
kubectl create ns seldon-mesh || echo "Namespace seldon-mesh already exists"helm repo add seldon-charts https://seldonio.github.io/helm-charts/
helm repo update seldon-chartshelm upgrade seldon-core-v2-crds seldon-charts/seldon-core-v2-crds \
--namespace default \
--install helm upgrade seldon-core-v2-setup seldon-charts/seldon-core-v2-setup \
--namespace seldon-mesh --set controller.clusterwide=true \
--install<mynamespace>.ansible-playbook playbooks/seldon-all.yamlansible-playbook playbooks/seldon-all.yaml -e seldon_mesh_namespace=my-seldon-mesh -e install_prometheus=no -e @playbooks/vars/set-custom-images.yamlansible-playbook playbooks/kind-cluster.yamlansible-playbook playbooks/setup-ecosystem.yamlansible-playbook playbooks/setup-ecosystem.yaml -e seldon_mesh_namespace=<mynamespace>ansible-playbook playbooks/setup-seldon.yamlansible-playbook playbooks/setup-seldon.yaml -e seldon_mesh_namespace=<mynamespace>Learn how to set up a self-hosted Kafka cluster for Seldon Core in development and learning environments.
You can run Kafka in the same Kubernetes cluster that hosts the Seldon Core 2. Seldon recommends using the Strimzi operator for Kafka installation and maintenance. For more details about configuring Kafka with Seldon Core 2 see the Configuration section.
Integrating self-hosted Kafka with Seldon Core 2 includes these steps:
Strimzi provides a Kubernetes Operator to deploy and manage Kafka clusters. First, we need to install the Strimzi Operator in your Kubernetes cluster.
Create a namespace where you want to install Kafka. For example the namespace seldon-mesh:
Install Strimzi.
Install Strimzi Operator.
This deploys the Strimzi Operator in the seldon-mesh namespace. After the Strimzi Operator is running, you can create a Kafka cluster by applying a Kafka custom resource definition.
Apply the Kafka cluster configuration.
Create a YAML file named kafka-nodepool.yaml to create a nodepool for the kafka cluster.
Apply the Kafka node pool configuration.
Check the status of the Kafka Pods to ensure they are running properly:
You should see multiple Pods for Kafka, and Strimzi operators running.
Error The Pod that begins with the name seldon-dataflow-engine does not show the status as Running.
One of the possible reasons could be that the DNS resolution for the service failed.
Solution
Check the logs of the Pod <seldon-dataflow-engine>:
In the output check if a message reads:
Verify the name in the metadata for the kafka.yaml and kafka-nodepool.yaml. It should read seldon
When the SeldonRuntime is installed in a namespace a ConfigMap is created with the settings for Kafka configuration. Update the ConfigMap only if you need to customize the configurations.
Verify that the ConfigMap resource named seldon-kafka that is created in the namespace seldon-mesh:
You should the ConfigMaps for Kafka, Zookeeper, Strimzi operators, and others.
View the configuration of the the ConfigMap named seldon-kafka.
You should see an output simialr to this:
After you integrated Seldon Core 2 with Kafka, you need to that adds an abstraction layer for traffic routing by receiving traffic from outside the Kubernetes platform and load balancing it to Pods running within the Kubernetes cluster.
To customize the settings you can add and modify the Kafka configuration using Helm, for example to add compression for producers.
Create a YAML file to specify the compression configuration for Seldon Core 2 runtime. For example, create the values-runtime-kafka-compression.yaml file. Use your preferred text editor to create and save the file with the following content:
Change to the directory that contains the values-runtime-kafka-compression.yaml file and then install Seldon Core 2 runtime in the namespace seldon-mesh.
If you are using a shared Kafka cluster with other applications, it is advisable to isolate topic names and consumer group IDs from other cluster users to prevent naming conflicts. This can be achieved by configuring the following two settings:
topicPrefix: set a prefix for all topics
consumerGroupIdPrefix: set a prefix for all consumer groups
Here's an example of how to configure topic name and consumer group ID isolation during a Helm installation for an application named myorg:
After you installed Seldon Core 2, and Kafka using Helm, you need to complete .
Create a YAML file to specify the initial configuration.
Note: This configuration sets up a Kafka cluster with version 3.9.0. Ensure that you review the the supported versions of Kafka and update the version in the kafka.yaml file as needed. For more configuration examples, see this strimzi-kafka-operator.
Use your preferred text editor to create and save the file as kafka.yaml with the following content:
Check the name of the Kafka services in the namespace:
Restart the Pod:
kubectl get svc -n seldon-meshkubectl delete pod <seldon-dataflow-engine> -n seldon-mesh kubectl create namespace seldon-mesh || echo "namespace seldon-mesh exists"helm repo add strimzi https://strimzi.io/charts/
helm repo updatehelm install strimzi-kafka-operator strimzi/strimzi-kafka-operator --namespace seldon-meshapiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
name: seldon
namespace: seldon-mesh
annotations:
strimzi.io/node-pools: enabled
strimzi.io/kraft: enabled
spec:
kafka:
replicas: 3
version: 3.9.0
listeners:
- name: plain
port: 9092
tls: false
type: internal
- name: tls
port: 9093
tls: true
type: internal
config:
processMode: kraft
auto.create.topics.enable: true
default.replication.factor: 1
inter.broker.protocol.version: 3.7
min.insync.replicas: 1
offsets.topic.replication.factor: 1
transaction.state.log.min.isr: 1
transaction.state.log.replication.factor: 1
entityOperator: nullkubectl apply -f kafka.yaml -n seldon-meshapiVersion: kafka.strimzi.io/v1beta2
kind: KafkaNodePool
metadata:
name: kafka
namespace: seldon-mesh
labels:
strimzi.io/cluster: seldon
spec:
replicas: 3
roles:
- broker
- controller
resources:
requests:
cpu: '500m'
memory: '2Gi'
limits:
memory: '2Gi'
template:
pod:
tmpDirSizeLimit: 1Gi
storage:
type: jbod
volumes:
- id: 0
type: ephemeral
sizeLimit: 500Mi
kraftMetadata: shared
- id: 1
type: persistent-claim
size: 10Gi
deleteClaim: falsekubectl apply -f kafka-nodepool.yaml -n seldon-meshkubectl get pods -n seldon-mesh```bash
NAME READY STATUS RESTARTS AGE
hodometer-5489f768bf-9xnmd 1/1 Running 0 25m
mlserver-0 3/3 Running 0 24m
seldon-dataflow-engine-75f9bf6d8f-2blgt 1/1 Running 5 (23m ago) 25m
seldon-envoy-7c764cc88-xg24l 1/1 Running 0 25m
seldon-kafka-0 1/1 Running 0 21m
seldon-kafka-1 1/1 Running 0 21m
seldon-kafka-2 1/1 Running 0 21m
seldon-modelgateway-54d457794-x4nzq 1/1 Running 0 25m
seldon-pipelinegateway-6957c5f9dc-6blx6 1/1 Running 0 25m
seldon-scheduler-0 1/1 Running 0 25m
seldon-v2-controller-manager-7b5df98677-4jbpp 1/1 Running 0 25m
strimzi-cluster-operator-66b5ff8bbb-qnr4l 1/1 Running 0 23m
triton-0 3/3 Running 0 24m
```kubectl logs <seldon-dataflow-engine> -n seldon-meshWARN [main] org.apache.kafka.clients.ClientUtils : Couldn't resolve server seldon-kafka-bootstrap.seldon-mesh:9092 from bootstrap.servers as DNS resolution failed for seldon-kafka-bootstrap.seldon-meshkubectl get configmaps -n seldon-meshNAME DATA AGE
kube-root-ca.crt 1 38m
seldon-agent 1 30m
seldon-kafka 1 30m
seldon-kafka-0 6 26m
seldon-kafka-1 6 26m
seldon-kafka-2 6 26m
seldon-manager-config 1 30m
seldon-tracing 4 30m
strimzi-cluster-operator 1 28mkubectl get configmap seldon-kafka -n seldon-mesh -o yamlapiVersion: v1
data:
kafka.json: '{"bootstrap.servers":"seldon-kafka-bootstrap.seldon-mesh:9092","consumer":{"auto.offset.reset":"earliest","message.max.bytes":"1000000000","session.timeout.ms":"6000","topic.metadata.propagation.max.ms":"300000"},"producer":{"linger.ms":"0","message.max.bytes":"1000000000"},"topicPrefix":"seldon"}'
kind: ConfigMap
metadata:
creationTimestamp: "2024-12-05T07:12:57Z"
name: seldon-kafka
namespace: seldon-mesh
ownerReferences:
- apiVersion: mlops.seldon.io/v1alpha1
blockOwnerDeletion: true
controller: true
kind: SeldonRuntime
name: seldon
uid: 9e724536-2487-487b-9250-8bcd57fc52bb
resourceVersion: "778"
uid: 5c041e69-f36b-4f14-8f0d-c8790003cb3ehelm upgrade seldon-core-v2-runtime seldon-charts/seldon-core-v2-runtime \
--namespace seldon-mesh \
-f values-runtime-kafka-compression.yaml \
--installhelm upgrade --install seldon-core-v2-setup seldon-charts/seldon-core-v2-setup \
--namespace seldon-mesh \
--set controller.clusterwide=true \
--set kafka.topicPrefix=myorg \
--set kafka.consumerGroupIdPrefix=myorgUnexpected error with integration github-files: Integration is not installed on this space
Unexpected error with integration github-files: Integration is not installed on this space