# Seldon Core 2

Seldon Core 2 provides a state of the art solution for machine learning inference.

### Prerequisites

* Set up and connect to a Kubernetes cluster running version 1.23 or later. For instructions on connecting to your Kubernetes cluster, refer to the documentation provided by your cloud provider.
* Install [kubectl](https://kubernetes.io/docs/tasks/tools/#kubectl), the Kubernetes command-line tool.
* Install [Helm](https://helm.sh/docs/intro/install/), the package manager for Kubernetes.

To use Seldon Core 2 in a production environment:

1. [Create namespaces](#creating-namespaces)
2. [Install Seldon Core 2](#installing-seldon-core-2)

### Creating Namespaces

* Create a namespace to contain the main components of Seldon. For example, create the namespace `seldon-system`:

  ```bash
  kubectl create ns seldon-system || echo "Namespace seldon-system already exists"
  ```
* Create a namespace to contain Kafka. For example, create the namespace `kafka`:

  ```bash
  kubectl create ns kafka || echo "Namespace kafka already exists"
  ```
* Create a namespace to contain the components related to request logging. For example, create the namespace `seldon-logs`:

  ```bash
  kubectl create ns seldon-logs || echo "Namespace seldon-logs already exists"
  ```
* Create a namespace that is accessible by Seldon Enterprise Platform, Seldon Core 2 runtime that defines core components required in each model, and Seldon Core 2 pre-configured servers to host the models. For example, create the namespace `seldon`:

  ```bash
  kubectl create ns seldon || echo "Namespace seldon already exists"
  ```
* Annotate the namespace `seldon` so that it is accessible in the Seldon Enterprise Platform UI:

  ```bash
  kubectl label ns seldon seldon.restricted=false --overwrite=true
  ```

### Installing Seldon Core 2

1. Add and update the Helm charts `seldon-charts` to the repository.

   ```bash
   helm repo add seldon-charts https://seldonio.github.io/helm-charts/
   helm repo update seldon-charts
   ```
2. Install Custom resource definitions for Seldon Core 2.

   ```bash
   helm upgrade seldon-core-v2-crds seldon-charts/seldon-core-v2-crds \
   --version 2.8.5 \
   --namespace default \
   --install 
   ```
3. Create a YAML file to specify the initial configuration for Seldon Core 2 operator. For example, create the `components-values.yaml` file. Use your preferred text editor to create and save the file with the following content:

   ```yaml
   controller:
     clusterwide: true

   dataflow:
     resources:
       cpu: 500m

   envoy:
     service:
       type: ClusterIP

   kafka:
     bootstrap: seldon-kafka-bootstrap.kafka:9092
     topics:
       numPartitions: 4

   opentelemetry:
     enable: false

   scheduler:
     service:
       type: ClusterIP

   serverConfig:
     mlserver:
       resources:
         cpu: 1
         memory: 2Gi

     triton:
       resources:
         cpu: 1
         memory: 2Gi

   serviceGRPCPrefix: "http2-"
   ```

   This configuration installs the Seldon Core 2 Operator to work across an entire Kubernetes cluster. If you wish to install the operator in a specific namespace instead, set `clusterwide` to `false` in the `components-values.yaml` file.
4. Change to the directory that contains the `components-values.yaml` file and then install Seldon Core 2 operator in the namespace `seldon-system`.

   ```bash
    helm upgrade seldon-core-v2-components seldon-charts/seldon-core-v2-setup \
    --version 2.8.5 \
    -f components-values.yaml \
    --namespace seldon-system \
    --install
   ```
5. Install Seldon Core 2 runtimes in the namespace `seldon`.

   ```bash
   helm upgrade seldon-core-v2-runtime seldon-charts/seldon-core-v2-runtime \
   --version 2.8.5 \
   --namespace seldon \
   --install
   ```

   <div data-gb-custom-block data-tag="hint" data-style="warning" class="hint hint-warning"><p>One of the runtime components installed in this step is the <code>Dataflow Engine</code>. It requires a running Kafka instance to function properly. Ensure that you have a Kafka bootstrap server running at <code>seldon-kafka-bootstrap.kafka:9092</code>, as specified in the YAML file in step 3. If this is not the case, you can still proceed with the next step and resolve this issue in a later step, when you <a href="/pages/buUHGNpfYJ24aaKUWhgP">install Kafka</a>.</p></div>
6. Create a YAML file to specify the initial configuration for Seldon Core 2 servers. For example, create the `servers-values.yaml` file. Use your preferred text editor to create and save the file with the following content:

   ```yaml
   mlserver:
    replicas: 1

   triton:
     replicas: 1
   ```
7. Change to the directory that contains the `servers-values.yaml` file and then install Seldon Core 2 servers in the namespace `seldon`.

   ```bash
    helm upgrade seldon-core-v2-servers seldon-charts/seldon-core-v2-servers \
    --version 2.8.5 \
    -f servers-values.yaml \
    --namespace seldon \
    --install
   ```

### Additional Resources

* [Seldon Core 2 Documentation](https://docs.seldon.ai/seldon-core-2)
* [GKE Documentation](https://cloud.google.com/kubernetes-engine/docs)
* [AWS Documentation](https://docs.aws.amazon.com/)
* [Azure Documentation](https://learn.microsoft.com/azure/)


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.seldon.ai/seldon-enterprise-platform/production-environment/seldon-core-2.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.