Seldon Core 2

Seldon Core 2 provides a state of the art solution for machine learning inference.

Prerequisites

Set up and connect to a Kubernetes cluster running version 1.23 or later. For instructions on connecting to your Kubernetes cluster, refer to the documentation provided by your cloud provider.
Install kubectl, the Kubernetes command-line tool.
Install Helm, the package manager for Kubernetes.

To use Seldon Core 2 in a production environment:

Creating Namespaces

Create a namespace to contain the main components of Seldon. For example, create the namespace seldon-system:
```
kubectl create ns seldon-system || echo "Namespace seldon-system already exists"
```
Create a namespace to contain Kafka. For example, create the namespace kafka:
```
kubectl create ns kafka || echo "Namespace kafka already exists"
```
Create a namespace to contain the components related to request logging. For example, create the namespace seldon-logs:
```
kubectl create ns seldon-logs || echo "Namespace seldon-logs already exists"
```
Create a namespace that is accessible by Seldon Enterprise Platform, Seldon Core 2 runtime that defines core components required in each model, and Seldon Core 2 pre-configured servers to host the models. For example, create the namespace seldon:
```
kubectl create ns seldon || echo "Namespace seldon already exists"
```
Annotate the namespace seldon so that it is accessible in the Seldon Enterprise Platform UI:
```
kubectl label ns seldon seldon.restricted=false --overwrite=true
```

Installing Seldon Core 2

Add and update the Helm charts seldon-charts to the repository.

helm repo add seldon-charts https://seldonio.github.io/helm-charts/
helm repo update seldon-charts

Install Custom resource definitions for Seldon Core 2.

helm upgrade seldon-core-v2-crds seldon-charts/seldon-core-v2-crds \
--version 2.8.5 \
--namespace default \
--install

Create a YAML file to specify the initial configuration for Seldon Core 2 operator. For example, create the components-values.yaml file. Use your preferred text editor to create and save the file with the following content:

controller:
  clusterwide: true

dataflow:
  resources:
    cpu: 500m

envoy:
  service:
    type: ClusterIP

kafka:
  bootstrap: seldon-kafka-bootstrap.kafka:9092
  topics:
    numPartitions: 4

opentelemetry:
  enable: false

scheduler:
  service:
    type: ClusterIP

serverConfig:
  mlserver:
    resources:
      cpu: 1
      memory: 2Gi

  triton:
    resources:
      cpu: 1
      memory: 2Gi

serviceGRPCPrefix: "http2-"

This configuration installs the Seldon Core 2 Operator to work across an entire Kubernetes cluster. If you wish to install the operator in a specific namespace instead, set clusterwide to false in the components-values.yaml file.

Change to the directory that contains the components-values.yaml file and then install Seldon Core 2 operator in the namespace seldon-system.

 helm upgrade seldon-core-v2-components seldon-charts/seldon-core-v2-setup \
 --version 2.8.5 \
 -f components-values.yaml \
 --namespace seldon-system \
 --install

Install Seldon Core 2 runtimes in the namespace seldon.
```
helm upgrade seldon-core-v2-runtime seldon-charts/seldon-core-v2-runtime \
--version 2.8.5 \
--namespace seldon \
--install
```
One of the runtime components installed in this step is the Dataflow Engine. It requires a running Kafka instance to function properly. Ensure that you have a Kafka bootstrap server running at seldon-kafka-bootstrap.kafka:9092, as specified in the YAML file in step 3. If this is not the case, you can still proceed with the next step and resolve this issue in a later step, when you install Kafka.
Create a YAML file to specify the initial configuration for Seldon Core 2 servers. For example, create the servers-values.yaml file. Use your preferred text editor to create and save the file with the following content:
```
mlserver:
 replicas: 1

triton:
  replicas: 1
```

Change to the directory that contains the servers-values.yaml file and then install Seldon Core 2 servers in the namespace seldon.

 helm upgrade seldon-core-v2-servers seldon-charts/seldon-core-v2-servers \
 --version 2.8.5 \
 -f servers-values.yaml \
 --namespace seldon \
 --install

Additional Resources

PreviousProduction Environment NextKafka Integration

Last updated 4 days ago