Seldon Core 2

Seldon Core 2 provides a state of the art solution for machine learning inference.

Prerequisites

  • Set up and connect to a Kubernetes cluster running version 1.23 or later. For instructions on connecting to your Kubernetes cluster, refer to the documentation provided by your cloud provider.

  • Install kubectl, the Kubernetes command-line tool.

  • Install Helm, the package manager for Kubernetes.

To use Seldon Core 2 in a production environment:

  1. Create namespaces

  2. Install Seldon Core 2

Creating Namespaces

  • Create a namespace to contain the main components of Seldon. For example, create the namespace seldon-system:

    kubectl create ns seldon-system || echo "Namespace seldon-system already exists"

  • Create a namespace to contain Kafka. For example, create the namespace kafka:

    kubectl create ns kafka || echo "Namespace kafka already exists"

  • Create a namespace to contain the components related to request logging. For example, create the namespace seldon-logs:

    kubectl create ns seldon-logs || echo "Namespace seldon-logs already exists"

  • Create a namespace that is accessible by Seldon Enterprise Platform, Seldon Core 2 runtime that defines core components required in each model, and Seldon Core 2 pre-configured servers to host the models. For example, create the namespace seldon:

    kubectl create ns seldon || echo "Namespace seldon already exists"

  • Annotate the namespace seldon so that it is accessible in the Seldon Enterprise Platform UI:

    kubectl label ns seldon seldon.restricted=false --overwrite=true

Installing Seldon Core 2

  1. Add and update the Helm charts seldon-charts to the repository.

  2. Install Custom resource definitions for Seldon Core 2.

  3. Create a YAML file to specify the initial configuration for Seldon Core 2 operator. For example, create the components-values.yaml file. Use your preferred text editor to create and save the file with the following content:

    This configuration installs the Seldon Core 2 Operator to work across an entire Kubernetes cluster. If you wish to install the operator in a specific namespace instead, set clusterwide to false in the components-values.yaml file.

  4. Change to the directory that contains the components-values.yaml file and then install Seldon Core 2 operator in the namespace seldon-system.

  5. Install Seldon Core 2 runtimes in the namespace seldon.

    One of the runtime components installed in this step is the Dataflow Engine. It requires a running Kafka instance to function properly. Ensure that you have a Kafka bootstrap server running at seldon-kafka-bootstrap.kafka:9092, as specified in the YAML file in step 3. If this is not the case, you can still proceed with the next step and resolve this issue in a later step, when you install Kafka.

  6. Create a YAML file to specify the initial configuration for Seldon Core 2 servers. For example, create the servers-values.yaml file. Use your preferred text editor to create and save the file with the following content:

  7. Change to the directory that contains the servers-values.yaml file and then install Seldon Core 2 servers in the namespace seldon.

Additional Resources

PreviousProduction EnvironmentNextKafka Integration

Last updated

Was this helpful?