All pages
Powered by GitBook
1 of 3

Loading...

Loading...

Loading...

Self-hosted Kafka

You can run Kafka in the same Kubernetes cluster that hosts the Seldon Enterprise Platform. We recommend using the Strimzi operator for Kafka installation and maintenance.

Note: These instructions help you quickly set up a Kafka cluster. For production grade installation consult or use one of .

Integrating self-hosted Kafka with Seldon Core 2 includes these steps:

  1. Install Kafka

  2. Configure Seldon Core 2

Installing Kafka in a Kubernetes cluster

Strimzi provides a Kubernetes Operator to deploy and manage Kafka clusters. First, we need to install the Strimzi Operator in your Kubernetes cluster.

  1. Create a namespace where you want to install Kafka. For example the name space kafka:

  2. Install Strimzi.

  3. Install Strimzi Operator.

    This deploys the Strimzi Operator in the

To integrate Kafka with Seldon Core 2:

  1. Update the configuration for Kafka to integrate with Seldon Core 2 Operator. For example, update the components-values.yaml file. Use your preferred text editor to update and save the file with the following content:

  2. Change to the directory that contains the components-values.yaml file and then install Seldon Core 2 operator in the namespace seldon-system.

kafka
namespace.
  • Next, you need to create a Kafka cluster by saving the following YAML configuration to a file named kafka.yaml and applying it with kubectl apply -f kafka.yaml:

    This will set up a Kafka cluster with version 3.7.0. Ensure that you have reviewed the supported versions of Kafka and updated the version in the kafka.yaml file as needed.

  • Check the status of the Kafka pods to ensure they are running properly:

    You should see multiple pods for Kafka, Zookeeper, and Strimzi operators running.

  • Configuring Seldon Core 2

    Strimzi documentation
    managed solutions
     apiVersion: kafka.strimzi.io/v1beta2
     kind: Kafka
     metadata:
       name: seldon
       namespace: kafka
     spec:
       kafka:
         replicas: 3
         version: 3.7.0
         config:
           auto.create.topics.enable: true
           default.replication.factor: 1
           inter.broker.protocol.version: 3.7
           min.insync.replicas: 1
           offsets.topic.replication.factor: 1
           transaction.state.log.min.isr: 1
           transaction.state.log.replication.factor: 1
         listeners:
         - name: plain
           port: 9092
           tls: false
           type: internal
         storage:
           type: ephemeral
       zookeeper:
         replicas: 1
         storage:
           type: ephemeral
    kubectl get pods -n kafka
    NAME                                       READY   STATUS    RESTARTS   AGE
    seldon-kafka-0                             1/1     Running   0          2d1h
    seldon-kafka-1                             1/1     Running   0          2d1h
    seldon-kafka-2                             1/1     Running   0          2d1h
    seldon-zookeeper-0                         1/1     Running   0          2d1h
    strimzi-cluster-operator-58ff6ccf5-pqfjn   1/1     Running   0          2d1h
    kubectl create namespace kafka || echo "namespace kafka exists"
    helm repo add strimzi https://strimzi.io/charts/
    helm repo update
    helm install strimzi-kafka-operator strimzi/strimzi-kafka-operator --namespace kafka
    controller:
      clusterwide: true
    
    dataflow:
      resources:
        cpu: 500m
    
    envoy:
      service:
        type: ClusterIP
    
    kafka:
      bootstrap: seldon-kafka-bootstrap.kafka:9092
      topicPrefix: seldon
      debug:
      consumer:
        autoOffsetReset: earliest
        sessionTimeoutMs: 6000
        topicMetadataRefreshIntervalMs: 1000
        topicMetadataPropagationMaxMs: 300000
        messageMaxBytes: 1000000000
      producer:
        lingerMs: 0
        messageMaxBytes: 1000000000
      topics:
        replicationFactor: 1
        numPartitions: 1
    
    opentelemetry:
      enable: false
    
    scheduler:
      service:
        type: ClusterIP
    
    serverConfig:
      mlserver:
        resources:
          cpu: 1
          memory: 2Gi
    
      triton:
        resources:
          cpu: 1
          memory: 2Gi
    
    serviceGRPCPrefix: "http2-"
     helm upgrade seldon-core-v2-components seldon-charts/seldon-core-v2-setup \
     --version 2.8.5 \
     -f components-values.yaml \
     --namespace seldon-system \
     --install

    Self-hosted PostgreSQL

    Installing PostgreSQL in the same Kubernetes cluster that hosts the Seldon Enterprise Platform.You can run PostgreSQL in the same Kubernetes cluster that hosts the Seldon Enterprise Platform. We recommend using the Zalando PostgreSQL operator for managing PostgreSQL installation and maintenance. Refer to their installation matrix for details about the supported PostgreSQL releases.

    1. Install Seldon Enterprise Platform.

    2. Install Ingress Controller.

    3. Install .

    1. Clone the Zalando operator repository in your computer.

    2. Change to the postgres-operatordirectory.

    3. Create a namespace where you want to install PostgreSQL. For example the name space postgres:

    1. Get the Pod that is running Seldon Enterprise Platform in the cluster and save it as $POD_NAME.

    1. You can use port-forwarding to access your application locally.

    1. Open your browser and navigate to http://127.0.0.1:8000/seldon-deploy/

    You may now explore the feature in Seldon Enterprise Platform.

    Install PostgreSQL using the Helm charts.

    After a successful installation, you should see::

  • To install a minimal PostgreSQL setup in the Kubernetes cluster, execute the following:

    If you would like to install a more complex setup with additional users, databases, replicas, and others see the official documentation of Zalando operator.

  • Verify if the postgres-operator Pod is running.

  • Create the required secret using the auto-generated password:

    After a successful configuration, you should see: secret/metadata-postgres configured

  • View the password that you created to access PostgreSQL database:

  • After the PostgreSQL database and secrets with credentials are ready, add the following to your install-values.yamlfile.

  • Change to the directory that contains the install-values.yaml file and then upgrade the Seldon Enterprise Platform installation in the namespace seldon-system.

  • Check the status of the installation seldon-enterprise-seldon-deploy.

    When the installation is complete you should see this:

  • Access Seldon Enterprise Platform.

  • to access Seldon Enterprise Platform.
    1. Find the IP address of the Seldon Enterprise Platform instance running with Istio:

    1. Open your browser and navigate to http://$ISTIO_INGRESS/seldon-deploy/ to access Seldon Enterprise Platform. Where $ISTIO_INGRESS is the IP address of Seldon Enterprise Platform.

    Installing PostgreSQL in a Kubernetes cluster

    Next

    Git
    Model Catalog
    helm install postgres-operator ./charts/postgres-operator --namespace   postgres
    NAME: postgres-operator 
    LAST DEPLOYED: Tue Aug 13 15:22:02 2024
    NAMESPACE: postgres
    STATUS: deployed
    REVISION: 1
    TEST SUITE: None
    NOTES: To verify that postgres-operator has started, run:
    kubectl --namespace=postgres get pods -l "app.kubernetes.io/name=postgres-operator"
    cat << EOF | kubectl apply -f -
    apiVersion: "acid.zalan.do/v1"
    kind: postgresql
    metadata:
      name: seldon-metadata-storage
      namespace: postgres
    spec:
      teamId: "seldon"
      volume:
        size: 5Gi
      numberOfInstances: 2
      users:
        seldon:  # database owner
        - superuser
        - createdb
      databases:
        metadata: seldon  # dbname: owner
      postgresql:
        version: "15"
    EOF
    kubectl --namespace=postgres get pods -l "app.kubernetes.io/name=postgres-operator"
    kubectl get secret seldon.seldon-metadata-storage.credentials.postgresql.acid.zalan.do -n postgres -o 'jsonpath={.data.password}' | base64 -d > db_pass
    kubectl create secret generic -n seldon-system metadata-postgres \
      --from-literal=user=seldon \
      --from-file=password=./db_pass \
      --from-literal=host=seldon-metadata-storage.postgres.svc.cluster.local \
      --from-literal=port=5432 \
      --from-literal=dbname=metadata \
      --from-literal=sslmode=require \
      --dry-run=client -o yaml \
      | kubectl apply -n seldon-system -f -
    rm db_pass
    kubectl get secret metadata-postgres -n seldon-system -o 'jsonpath={.data.password}' | base64 -d  
    metadata:
      pg:
       enabled: true
       secret: metadata-postgres
       clientTLSSecret: "postgres-client-certs"  # Optional, only needed for SSL verification
    helm upgrade seldon-enterprise seldon-charts/seldon-deploy --namespace seldon-system -f install-values.yaml --version 2.4.0 --install
    kubectl rollout status deployment/seldon-enterprise-seldon-deploy -n seldon-system
    deployment "seldon-enterprise-seldon-deploy" successfully rolled out
    ISTIO_INGRESS=$(kubectl get svc -n istio-system istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
    ISTIO_INGRESS+=$(kubectl get svc -n istio-system istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
    
    echo "Seldon Enterprise Platform: http://$ISTIO_INGRESS/seldon-deploy/"
    
    git clone https://github.com/zalando/postgres-operator.git
    cd postgres-operator
    kubectl create namespace postgres || echo "namespace postgres exists"
    export POD_NAME=$(kubectl get pods --namespace seldon-system -l "app.kubernetes.io/name=seldon-deploy,app.kubernetes.io/instance=seldon-enterprise" -o jsonpath="{.items[0].metadata.name}")
    kubectl port-forward $POD_NAME 8000:8000 --namespace seldon-system

    Learning Environment

    Installing Seldon Enterprise Platform in a learning environment.

    You can install Seldon Enterprise Platform on your local computer that is running a Kubernetes cluster using kind.

    Note: These instructions guide you through installing the Seldon Enterprise Platform on a local Kubernetes cluster, focusing on ease of learning. Ensure your cluster is running on hardware with at least 32GB of RAM. For installing Seldon Enterprise Platform in a production environment, see.

    To install Seldon Enterprise Platform and Seldon Core 2 ecosystem components:

    1. Create namespaces

    2. Install Seldon Core 2

    • Install a Kubernetes cluster that is running version 1.23 or later.

    • Install , the Kubernetes command-line tool.

    • Install , the package manager for Kubernetes.

    • Create a namespace to contain the main components of Seldon. For example, create the namespace seldon-system:

    • Create a namespace to contain Kafka. For example, create the namespace kafka:

    • Create a namespace to contain the components related to request logging. For example, create the namespace seldon-logs

    1. Add and update the Helm charts seldon-charts to the repository.

    2. Install Custom resource definitions for Seldon Core 2.

    3. Create a YAML file to specify the initial configuration for Seldon Core 2 Operator. For example, create the components-values.yaml file. Use your preferred text editor to create and save the file with the following content:

    1. Install Knative Eventing core components in the Kubernetes cluster.

    2. Create a YAML file to specify the initial configuration. For example, create the install-values.yaml file. Use your preferred text editor to create and save the file with the following content:

    3. Change to the directory that contains the install-values.yaml file and then install Seldon Enterprise Platform in the namespace seldon-system

    To explore the features of Seldon Enterprise Platform, you need to complete the installation of other components in the following order:

    Seldon Enterprise Platform license key. You can reach out to Seldon support team to get a trial license key.
    :
  • Create a namespace that is accessible by Seldon Enterprise Platform, Seldon Core 2 runtime that defines core components required in each model, and Seldon Core 2 pre-configured servers to host the models. For example, create the namespace seldon:

    kubectl create ns seldon || echo "Namespace seldon already exists"
  • Annotate the namespace seldon so that it is accessible in the Seldon Enterprise Platform UI:

    kubectl label ns seldon seldon.restricted=false --overwrite=true
  • Change to the directory that contains the components-values.yaml file and then install Seldon Core 2 operator in the namespace seldon-system.

     helm upgrade seldon-core-v2-components seldon-charts/seldon-core-v2-setup \
     --version 2.8.5 \
     -f components-values.yaml \
     --namespace seldon-system \
     --install
  • Install Seldon Core 2 runtimes in the namespace seldon.

    helm upgrade seldon-core-v2-runtime seldon-charts/seldon-core-v2-runtime \
    --version 2.8.5 \
    --namespace seldon \
    --install
  • Create a YAML file to specify the initial configuration for Seldon Core 2 servers. For example, create the servers-values.yaml file. Use your preferred text editor to create and save the file with the following content:

    mlserver:
     replicas: 1
    
    triton:
      replicas: 1
  • Change to the directory that contains the servers-values.yaml file and then install Seldon Core 2 servers in the namespace seldon.

     helm upgrade seldon-core-v2-servers seldon-charts/seldon-core-v2-servers \
     --version 2.8.5 \
     -f servers-values.yaml \
     --namespace seldon \
     --install
  • .

    When the installation is successful, you should see this:

  • Check the status of the installation seldon-enterprise-seldon-deploy.

    kubectl rollout status deployment/seldon-enterprise-seldon-deploy -n seldon-system

    When the installation is complete you should see this:

    deployment "seldon-enterprise-seldon-deploy" successfully rolled out
  • Get the Pod that is running Seldon Enterprise Platform in the cluster and save it as $POD_NAME.

    export POD_NAME=$(kubectl get pods --namespace seldon-system -l "app.kubernetes.io/name=seldon-deploy,app.kubernetes.io/instance=seldon-enterprise" -o jsonpath="{.items[0].metadata.name}")
  • You can use port-forwarding to access your application.

    kubectl port-forward $POD_NAME 8000:8000 --namespace seldon-system
  • Open your browser and navigate to http://127.0.0.1:8000/seldon-deploy/ to access Seldon Enterprise Platform with Seldon Core 2 resources, operator, runtimes and servers.

    Seldon Enterprise Platform
  • Apply the trial license and click Activate.

  • kubectl create ns seldon-system || echo "Namespace seldon-system already exists"
    kubectl create ns kafka || echo "Namespace kafka already exists"
    helm repo add seldon-charts https://seldonio.github.io/helm-charts/
    helm repo update seldon-charts
    helm upgrade seldon-core-v2-crds seldon-charts/seldon-core-v2-crds \
    --version 2.8.5 \
    --namespace default \
    --install 
    controller:
      clusterwide: true
    
    dataflow:
      resources:
        cpu: 500m
    
    envoy:
      service:
        type: ClusterIP
    
    kafka:
      bootstrap: seldon-kafka-bootstrap.seldon-mesh:9092
      topics:
        numPartitions: 4
    
    opentelemetry:
      enable: false
    
    scheduler:
      service:
        type: ClusterIP
    
    serverConfig:
      mlserver:
        resources:
          cpu: 1
          memory: 2Gi
    
      triton:
        resources:
          cpu: 1
          memory: 2Gi
    
    serviceGRPCPrefix: "http2-"
    kubectl apply -f https://github.com/knative/eventing/releases/download/knative-v1.12.2/eventing-crds.yaml
    kubectl apply -f https://github.com/knative/eventing/releases/download/knative-v1.12.2/eventing-core.yaml
    kubectl get crds | grep eventing.knative.dev
    image:
      image: seldonio/seldon-deploy-server:2.4.0
    
    rbac:
     opa:
       enabled:  false
     nsLabelsAuth:
       enabled:  true
    
    virtualService:
      create: false
    
    requestLogger:
      kafka_consumer:
        enabled: true
        bootstrap_servers: seldon-kafka-bootstrap.kafka.svc.cluster.local:9092
        group_id: metronome
        auto_offset_reset: earliest
    
    gitops:
      argocd:
        enabled: false
    
    enableAppAuth: false
    
    elasticsearch:
      basicAuth: false
    
    seldon:
      enabled: false
      knativeEnabled: false
    
    seldonCoreV2:
      enabled: true

    Prerequisites

    Creating Namespaces

    Installing Seldon Core 2

    Installating Seldon Enterprise Platform on a Kubernetes cluster

    Note: These configurations do not enable features such as Model Catalog, Monitoring and Alerting, Request logging, GitOps and others.

    Note: After confirming that port-forwarding to the application is configured, you can open your browser and navigate to http://127.0.0.1:8000/seldon-deploy/ to access the Seldon Enterprise Platform deployed in the Kubernetes cluster at any time.

    Next

    Additional Resources

    Install Seldon Enterprise Platform
    kubectl
    Helm
    Integrating with Kafka
    Installing Istio
    Installing PostgreSQL
    Installing Prometheus
    Seldon Core Documentation
    Seldon Enterprise Documentation
    kind
    cluster requirements
    kubectl create ns seldon-logs || echo "Namespace seldon-logs already exists"
    helm install seldon-enterprise seldon-charts/seldon-deploy --namespace seldon-system  -f install-values.yaml --version 2.4.0
    NAME: seldon-enterprise
    LAST DEPLOYED: Fri Jul 26 16:43:44 2024
    NAMESPACE: seldon-system
    STATUS: deployed
    REVISION: 1
    NOTES:
    1. Get the application URL by running these commands:
      export POD_NAME=$(kubectl get pods --namespace seldon-system -l "app.kubernetes.io/name=seldon-deploy,app.kubernetes.io/instance=seldon-enterprise" -o jsonpath="{.items[0].metadata.name}")
      echo "Visit http://127.0.0.1:8000/seldon-deploy/ to use your application"
      kubectl port-forward $POD_NAME 8000:8000 --namespace seldon-system