Managed Kafka

When you use SASL as the authentication mechanism for Kafka, the credentials consist of a username and password pair. The password is supplied through a secret.

To create a password for Seldon Core 2 in the namespace seldon, run the following command:

```
kubectl create secret generic kafka-sasl-secret --from-literal password=<kafka-password> -n seldon
```

Values in Seldon Core 2

In Seldon Core 2 you need to specify these values in components-values.yaml

• security.kafka.sasl.mechanism - SASL security mechanism, e.g. SCRAM-SHA-512

• security.kafka.sasl.client.username - Kafka username

• security.kafka.sasl.client.secret - Created secret with password

• security.kafka.ssl.client.brokerValidationSecret - Certificate Authority of Kafka Brokers

The resulting set of values to include in components-values.yaml is similar to:

security:
  kafka:
    protocol: SASL_SSL
    sasl:
      mechanism: SCRAM-SHA-512
      client:
        username: <kafka-username>      # TODO: Replace with your Kafka username
        secret: kafka-sasl-secret       # NOTE: Secret name from previous step
    ssl:
      client:
        secret:                                   # NOTE: Leave empty
        brokerValidationSecret: kafka-broker-tls  # NOTE: Optional

The security.kafka.ssl.client.brokerValidationSecret field is optional. Leave it empty if your brokers use well known Certificate Authority such as Let’s Encrypt.

When you use OAuth 2.0 as the authentication mechanism for Kafka, the credentials consist of a Client ID and Client Secret, which are used with your Identity Provider to obtain JWT tokens for authenticating with Kafka brokers.

1. Create a Kubernetes secret kafka-oauth.yaml file.

   Copy

   apiVersion: v1
   kind: Secret
   metadata:
     name: kafka-oauth
   type: Opaque
   stringData:
     method: OIDC
     client_id: <client id>
     client_secret: <client secret>
     token_endpoint_url: <token endpoint url>
     extensions: ""
     scope: ""

2. Store the secret in the seldon namespace to configure with Seldon Core 2.

   Copy

   kubectl apply -f kafka-oauth.yaml -n seldon

   This secret must be present in seldon-logs namespace and every namespace containing Seldon Core 2 runtime.

Client ID, client secret and token endpoint url should come from identity provider such as Keycloak or Azure AD.

Values in Seldon Core 2

In Seldon Core 2 you need to specify these values:

• security.kafka.sasl.mechanism - set to OAUTHBEARER

• security.kafka.sasl.client.secret - Created secret with client credentials

• security.kafka.ssl.client.brokerValidationSecret - Certificate Authority of Kafka brokers

The resulting set of values in components-values.yaml is similar to:

security:
  kafka:
    protocol: SASL_SSL
    sasl:
      mechanism: OAUTHBEARER
      client:
          secret: kafka-oauth                     # NOTE: Secret name from earlier step
    ssl:
      client:
        secret:                                   # NOTE: Leave empty
        brokerValidationSecret: kafka-broker-tls  # NOTE: Optional

The security.kafka.ssl.client.brokerValidationSecret field is optional. Leave it empty if your brokers use well known Certificate Authority such as Let’s Encrypt.

When you use mTLS as authentication mechanism Kafka uses a set of certificates to authenticate the client.

• A client certificate, referred to as tls.crt.

• A client key, referred to as tls.key.

• A root certificate, referred to as ca.crt.

These certificates are expected to be encoded as PEM certificates and are provided through a secret, which can be created in teh namespace seldon:

kubectl create secret generic kafka-client-tls -n seldon \
  --from-file ./tls.crt \
  --from-file ./tls.key \
  --from-file ./ca.crt

This secret must be present in seldon-logs namespace and every namespace containing Seldon Core 2 runtime.

Ensure that the field used within the secret follow the same naming convention: tls.crt, tls.key and ca.crt. Otherwise, Seldon Core 2 may not be able to find the correct set of certificates.

Reference these certificates within the corresponding Helm values for both Seldon Core 2.

Values for Seldon Core 2 In Seldon Core 2 you need to specify these values:

• security.kafka.ssl.client.secret - Secret name containing client certificates

• security.kafka.ssl.client.brokerValidationSecret - Certificate Authority of Kafka Brokers

The resulting set of values to include in components-values.yaml is similar to:

security:
  kafka:
    protocol: SSL
    ssl:
      client:
        secret: kafka-client-tls                  # NOTE: Secret name from earlier step
        brokerValidationSecret: kafka-broker-tls  # NOTE: Optional

The security.kafka.ssl.client.brokerValidationSecret field is optional. Leave it empty if your brokers use well known Certificate Authority such as Let’s Encrypt.

Prerequisites:

• You must use at least the Standard tier for your Event Hub namespace because the Basic tier does not support the Kafka protocol.

• Seldon Core 2 creates two Kafka topics for each model and pipeline, plus one global topic for errors. This results in a total number of topics calculated as: 2 x (number of models + number of pipelines) + 1. This topic count is likely to exceed the limit of the Standard tier in Azure Event Hub. For more information, see quota information.

Creating a namespace and obtaining the connection string

These are the steps that you need to perform in Azure Portal.

1. Create an Azure Event Hub namespace. You need to have an Azure Event Hub namespace. Follow the Azure quickstart documentation to create one. Note: You do not need to create individual Event Hubs (topics) as Seldon Core 2 automatically creates all necessary topics.

2. Connection string for Kafka Integration. To connect to the Azure Event Hub using the Kafka API, you need to obtain Kafka endpoint and Connection string. For more information, see Get an Event Hubs connection string

   Note: Ensure you get the Connection string at the namespace level, as it is needed to dynamically create new topics. The format of the Connection string should be:

   Copy

   Endpoint=sb://<namespace>.servicebus.windows.net/;SharedAccessKeyName=XXXXXX;SharedAccessKey=XXXXXX

Creating secrets for Seldon Core 2 To store the SASL password in the Kubernetes cluster that run Seldon Core 2, create a secret named azure-kafka-secret for Core 2 in the namespace seldon. In the following command make sure to replace <password> with a password of your choice and <namespace> with the namespace form Azure Event Hub.

kubectl create secret generic azure-kafka-secret --from-literal=<password>="Endpoint=sb://<namespace>.servicebus.windows.net/;SharedAccessKeyName=XXXXXX;SharedAccessKey=XXXXXX" -n seldon

Creating API Keys

These are the steps that you need to perform in Confluent Cloud.

1. Navigate to Clients > New client and choose a client, for example GO and generate new Kafka cluster API key. For more information, see Confluent documentation.

Confluent generates a configuration file with the details.

1. Save the values of Key, Secret, and bootstrap.servers from the configuration file.

Creating secrets for Seldon Core 2 These are the steps that you need to perform in the Kubernetes cluster that run Seldon Core 2 to store the SASL password.

1. Create a secret named confluent-kafka-sasl for Seldon Core 2 in the namespace seldon. In the following command make sure to replace <password> with with the value of Secret that you generated in Confluent cloud.

kubectl create secret generic confluent-kafka-sasl --from-literal password="<password>" -n seldon

Confluent Cloud managed Kafka supports OAuth 2.0 to authenticate your Kafka clients. See Confluent Cloud documentation for further details.

Configuring Identity Provider

In Confluent Cloud Console Navigate to Account & Access / Identity providers and complete these steps:

1. register your Identity Provider. See Confluent Cloud documentation for further details.

2. add new identity pool to your newly registered Identity Provider. See Confluent Cloud documentation for further details.

3. Obtain these details from Confluent Cloud:

   • Cluster ID: Cluster Overview → Cluster Settings → General → Identification

   • Identity Pool ID: Accounts & access → Identity providers → .

4. Obtain these details from your identity providers such as Keycloak or Azure AD.

   • Client ID

   • Client secret

   • Token Endpoint URL

If you are using Azure AD you may will need to set scope: api://<client id>/.default.

Creating Kubernetes secret

1. Create Kubernetes secrets to store the required client credentials. For example, create a kafka-secret.yaml file by replacing the values of <client id>, <client secret>, <token endpoint url>, <cluster id>,<identity pool id> with the values that you obtained from Confluent Cloud and your identity provider.

   Copy

   apiVersion: v1
   kind: Secret
   metadata:
     name: confluent-kafka-oauth
   type: Opaque
   stringData:
     method: OIDC
     client_id: <client id>
     client_secret: <client secret>
     token_endpoint_url: <token endpoint url>
     extensions: logicalCluster=<cluster id>,identityPoolId=<identity pool id>
     scope: ""

2. Provide the secret named confluent-kafka-oauth in the seldon namespace to configure with Seldon Core 2.

   Copy

   kubectl apply -f kafka-secret.yaml -n seldon

   This secret must be present in seldon-logs namespace and every namespace containing Seldon Core 2 runtime.

Update the initial configuration for Seldon Core 2 in the components-values.yaml file. Use your preferred text editor to update and save the file with the following content:

controller:
  clusterwide: true

dataflow:
  resources:
    cpu: 500m

envoy:
  service:
    type: ClusterIP

kafka:
  bootstrap: <namespace>.servicebus.windows.net:9093
  topics:
    replicationFactor: 3
    numPartitions: 4    
security:
  kafka:
    protocol: SASL_SSL
    sasl:
      mechanism: "PLAIN"
      client:
        username: $ConnectionString
        secret: azure-kafka-secret
    ssl:
      client:
        secret:
        brokerValidationSecret:
        
opentelemetry:
  enable: false

scheduler:
  service:
    type: ClusterIP

serverConfig:
  mlserver:
    resources:
      cpu: 1
      memory: 2Gi

  triton:
    resources:
      cpu: 1
      memory: 2Gi

serviceGRPCPrefix: "http2-"

Update the initial configuration for Seldon Core 2 Operator in the components-values.yaml file. Use your preferred text editor to update and save the file with the following content:

controller:
  clusterwide: true

dataflow:
  resources:
    cpu: 500m

envoy:
  service:
    type: ClusterIP

kafka:
  bootstrap: <confluent-endpoints>
  topics:
    replicationFactor: 3
    numPartitions: 4
  consumer:
    messageMaxBytes: 8388608
  producer:
    messageMaxBytes: 8388608

security:
  kafka:
    protocol: SASL_SSL
    sasl:
      mechanism: "PLAIN"
      client:
        username: <username>
        secret: confluent-kafka-sasl
    ssl:
      client:
        secret:
        brokerValidationSecret:

opentelemetry:
  enable: false

scheduler:
  service:
    type: ClusterIP

serverConfig:
  mlserver:
    resources:
      cpu: 1
      memory: 2Gi

  triton:
    resources:
      cpu: 1
      memory: 2Gi

serviceGRPCPrefix: "http2-"

Update the initial configuration for Seldon Core 2 Operator in the components-values.yaml file. Use your preferred text editor to update and save the file with the following content:

controller:
  clusterwide: true

dataflow:
  resources:
    cpu: 500m

envoy:
  service:
    type: ClusterIP

kafka:
  bootstrap: <confluent-endpoints>
  topics:
    replicationFactor: 3
    numPartitions: 4
  consumer:
    messageMaxBytes: 8388608
  producer:
    messageMaxBytes: 8388608

security:
  kafka:
    protocol: SASL_SSL
    sasl:
      mechanism: OAUTHBEARER
      client:
        secret: confluent-kafka-oauth
    ssl:
      client:
        secret:
        brokerValidationSecret:

opentelemetry:
  enable: false

scheduler:
  service:
    type: ClusterIP

serverConfig:
  mlserver:
    resources:
      cpu: 1
      memory: 2Gi

  triton:
    resources:
      cpu: 1
      memory: 2Gi

serviceGRPCPrefix: "http2-"