Managed Kafka

Guide to integrating managed Kafka services (Confluent Cloud, Amazon MSK, Azure Event Hub) with Seldon Core 2, including security configurations and authentication setup.

Seldon recommends a managed Kafka service for production installation. You can integrate and secure your managed Kafka Seldon Core 2.

Some of the Managed Kafka services that are tested are:

  • Confluent Cloud (security: SASL/PLAIN)

  • Confluent Cloud (security: SASL/OAUTHBEARER)

  • Amazon MSK (security: mTLS)

  • Amazon MSK (security: SASL/SCRAM)

  • Azure Event Hub (security: SASL/PLAIN)

Integrating managed Kafka services

These instructions outline the necessary configurations to integrate managed Kafka services with Seldon Core 2.

  1. Examples configurations

  2. Configuring with Seldon Core 2

Securing managed Kafka services

You can secure Seldon Core 2 integration with managed Kafka services using:

Kafka Encryption (TLS)

In production settings, always set up TLS encryption with Kafka. This ensures that neither the credentials nor the payloads are transported in plaintext.

Note: TLS encryption involves only single-sided TLS. This means that the contents are encrypted and sent to the server, but the client won't send any form of certificate. Therefore, it does not take care of authenticating the client. Client authentication can be configured through mutual TLS (mTLS) or SASL mechanism, which are covered in the Kafka Authentication section .

When TLS is enabled, the client needs to know the root CA certificate used to create the server's certificate. This is used to validate the certificate sent back by the Kafka server.

  1. Create a certificate named ca.crt that is encoded as a PEM certificate. It is important that the certificate is saved as ca.crt. Otherwise, Seldon Core 2 may not be able to find the certificate. Within the cluster, you can provide the server's root CA certificate through a secret. For example, a secret named kafka-broker-tls with a certificate.

kubectl create secret generic kafka-broker-tls -n seldon-mesh --from-file ./ca.crt

Kafka Authentication

In production environments, Kafka clusters often require authentication, especially when using managed Kafka solutions. Therefore, when installing Seldon Core 2 components, it is crucial to provide the correct credentials for a secure connection to Kafka.

The type of authentication used with Kafka varies depending on the setup but typically includes one of the following:

  • Simple Authentication and Security Layer (SASL): Requires a username and password.

  • Mutual TLS (mTLS): Involves using SSL certificates as credentials.

  • OAuth 2.0: Uses the client credential flow to acquire a JWT token.

These credentials are stored as Kubernetes secrets within the cluster. When setting up Seldon Core 2 you must create the appropriate secret in the correct format and update the components-values.yaml, and install-values files respectively.

When you use SASL as the authentication mechanism for Kafka, the credentials consist of a username and password pair. The password is supplied through a secret.

Note:

  • Ensure that the field used for the password within the secret is named password. Otherwise, Seldon Core 2 may not be able to find the correct password.

  • This password must be present in seldon-logs namespace and every namespace containing Seldon Core 2 runtime.

  1. Create a password for Seldon Core 2 in the namespace seldon-mesh.

    kubectl create secret generic kafka-sasl-secret --from-literal password=<kafka-password> -n seldon-mesh

Values in Seldon Core 2

In Seldon Core 2 you need to specify these values in values.yaml

  • security.kafka.sasl.mechanism - SASL security mechanism, e.g. SCRAM-SHA-512

  • security.kafka.sasl.client.username - Kafka username

  • security.kafka.sasl.client.secret - Created secret with password

  • security.kafka.ssl.client.brokerValidationSecret - Certificate Authority of Kafka Brokers

The resulting set of values to include in values.yaml is similar to:

security:
  kafka:
    protocol: SASL_SSL
    sasl:
      mechanism: SCRAM-SHA-512
      client:
        username: <kafka-username>      # TODO: Replace with your Kafka username
        secret: kafka-sasl-secret       # NOTE: Secret name from previous step
    ssl:
      client:
        secret:                                   # NOTE: Leave empty
        brokerValidationSecret: kafka-broker-tls  # NOTE: Optional

The security.kafka.ssl.client.brokerValidationSecret field is optional. Leave it empty if your brokers use well known Certificate Authority such as Let's Encrypt.

Example configurations for managed Kafka services

Here are some examples to create secrets for managed Kafka services such as Azure Event Hub, Confluent Cloud(SASL), Confluent Cloud(OAuth2.0).

Prerequisites:

  • You must use at least the Standard tier for your Event Hub namespace because the Basic tier does not support the Kafka protocol.

  • Seldon Core 2 creates two Kafka topics for each model and pipeline, plus one global topic for errors. This results in a total number of topics calculated as: 2 x (number of models + number of pipelines) + 1. This topic count is likely to exceed the limit of the Standard tier in Azure Event Hub. For more information, see quota information.

Creating a namespace and obtaining the connection string

These are the steps that you need to perform in Azure Portal.

  1. Create an Azure Event Hub namespace. You need to have an Azure Event Hub namespace. Follow the Azure quickstart documentation to create one. Note: You do not need to create individual Event Hubs (topics) as Seldon Core 2 automatically creates all necessary topics.

  2. Connection string for Kafka Integration. To connect to the Azure Event Hub using the Kafka API, you need to obtain Kafka endpoint and Connection string. For more information, see Get an Event Hubs connection string

    Note: Ensure you get the Connection string at the namespace level, as it is needed to dynamically create new topics. The format of the Connection string should be:

    Endpoint=sb://<namespace>.servicebus.windows.net/;SharedAccessKeyName=XXXXXX;SharedAccessKey=XXXXXX

Creating secrets for Seldon Core 2 These are the steps that you need to perform in the Kubernetes cluster that run Seldon Core 2 to store the SASL password.

  1. Create a secret named azure-kafka-secret for Seldon Core 2 in the namespace seldon. In the following command make sure to replace <password> with a password of your choice and <namespace> with the namespace form Azure Event Hub.

kubectl create secret generic azure-kafka-secret --from-literal=<password>="Endpoint=sb://<namespace>.servicebus.windows.net/;SharedAccessKeyName=XXXXXX;SharedAccessKey=XXXXXX" -n seldon

  1. Create a secret named azure-kafka-secret for Seldon Core 2 in the namespace seldon-system. In the following command make sure to replace <password> with a password of your choice and <namespace> with the namespace form Azure Event Hub.

kubectl create secret generic azure-kafka-secret --from-literal=<password>="Endpoint=sb://<namespace>.servicebus.windows.net/;SharedAccessKeyName=XXXXXX;SharedAccessKey=XXXXXX" -n seldon-system

Configuring Seldon Core 2

To integrate Kafka with Seldon Core 2.

  1. Update the initial configuration.

Note: In these configurations you may need:

  • to tweak the values for replicationFactor and numPartitions that best suits your cluster configuration.

  • set the value for username as $ConnectionString this is not a variable.

  • replace <namespace> with the namespace in Azure Event Hub.

Update the initial configuration for Seldon Core 2 in the components-values.yaml file. Use your preferred text editor to update and save the file with the following content:

controller:
  clusterwide: true

dataflow:
  resources:
    cpu: 500m

envoy:
  service:
    type: ClusterIP

kafka:
  bootstrap: <namespace>.servicebus.windows.net:9093
  topics:
    replicationFactor: 3
    numPartitions: 4    
security:
  kafka:
    protocol: SASL_SSL
    sasl:
      mechanism: "PLAIN"
      client:
        username: $ConnectionString
        secret: azure-kafka-secret
    ssl:
      client:
        secret:
        brokerValidationSecret:
        
opentelemetry:
  enable: false

scheduler:
  service:
    type: ClusterIP

serverConfig:
  mlserver:
    resources:
      cpu: 1
      memory: 2Gi

  triton:
    resources:
      cpu: 1
      memory: 2Gi

serviceGRPCPrefix: "http2-"

  1. To enable Kafka Encryption (TLS) you need to reference the secret that you created in the security.kafka.ssl.client.secret field of the Helm chart values. The resulting set of values to include in components-values.yaml is similar to:

security:
  kafka:
    ssl:
      secret:
        brokerValidationSecret: kafka-broker-tls

  1. Change to the directory that contains the components-values.yaml file and then install Seldon Core 2 operator in the namespace seldon-system.

 helm upgrade seldon-core-v2-components seldon-charts/seldon-core-v2-setup \
 --version 2.8.0 \
 -f components-values.yaml \
 --namespace seldon-system \
 --install

After you integrated Seldon Core 2 with Kafka, you need to Install an Ingress Controller that adds an abstraction layer for traffic routing by receiving traffic from outside the Kubernetes platform and load balancing it to Pods running within the Kubernetes cluster.

PreviousKafka IntegrationNextIngress Controller

Last updated

Was this helpful?