1 of 2

Configuration

Seldon can be configured via various config files.

Kafka Configuration

We allow configuration of the Kafka integration. In general this configuration looks like:

The top level keys are:

topicPrefix : the prefix to add to kafka topics created by Seldon
consumerGroupIdPrefix : the prefix to add to Kafka consumer group IDs created by Seldon
bootstrap.servers : the global bootstrap kafka servers to use
consumer : consumer settings
producer : producer settings
streams : KStreams settings

For topicPrefix you can use any acceptable kafka topic characters which are a-z, A-Z, 0-9, . (dot), _ (underscore), and - (dash). We use . (dot) internally as topic naming separator so we would suggest you don't end your topic prefix with a dot for clarity. For illustration, an example topic could be seldon.default.model.mymodel.inputs where seldon is the topic prefix.

The consumerGroupIdPrefix will ensure that all consumer groups created have a given prefix.

Kubernetes

For Kubernetes this is controlled via a ConfigMap called seldon-kafka whose default values are defined in the SeldonConfig custom resource. For more details see the componets.yaml file

When the SeldonRuntime is installed in a namespace a configMap will be created with these settings for Kafka configuration.

To customize the settings you can add and modify the Kafka configuration via Helm, for example below is a custom Helm values file that add compression for producers:

To use this with the SeldonRuntime Helm chart:

helm install seldon-v2-runtime k8s/helm-charts/seldon-core-v2-runtime \
    --namespace seldon-mesh \
    --values k8s/samples/values-runtime-kafka-compression.yaml

Topic and consumer isolation

If you use a shared Kafka cluster with other applications you may want to isolate the topic names and consumer group IDs from other users of the cluster to ensure there is no name clash. For this we provide two settings:

topicPrefix: set a prefix for all topics
consumerGroupIdPrefix: set a prefix for all consumer groups

An example to set this in the configuration when using the helm installation is showm below for creating the default SeldonConfig:

helm upgrade --install seldon-v2 k8s/helm-charts/seldon-core-v2-setup/ -n seldon-mesh \
    --set controller.clusterwide=true \
    --set kafka.topicPrefix=myorg \
    --set kafka.consumerGroupIdPrefix=myorg

You can find a worked example here.

You can create alternate SeldonConfigs with different values or override values for particular SeldonRuntime installs.

Tracing Configuration

We allow configuration of tracing. This file looks like:

The top level keys are:

enable : whether to enable tracing
otelExporterEndpoint : The host and port for the OTEL exporter
otelExporterProtocol : The protocol for the OTEL exporter. Currently used for jvm-based components only (such as dataflow-engine), because opentelemetry-java-instrumentation requires a http(s) URI for the endpoint but defaults to http/protobuf as a protocol. Because of this, gRPC connections (over http) can only be set up by setting this option to grpc
ratio : The ratio of requests to trace. Takes values between 0 and 1 inclusive.

Kubernetes

For Kubernetes this is controlled via a ConfigMap call seldon-tracing whose default value is shown below:

Note, this ConfigMap is created via our Helm charts and there is usually no need to modify it manually.

At present Java instrumentation (for the dataflow engine) is duplicated via separate keys.

Managed Kafka

Seldon recommends managed Kafka for production installation. On this page we will demonstrate how you can integrate and configure your managed Kafka with Seldon Core 2.

Securing managed Kafka services

You can secure your Seldon Core 2 integration with managed Kafka services by setting up encryption and authenticaion.

Kafka Encryption (TLS)

In production settings, always set up TLS encryption with Kafka. This ensures that neither the credentials nor the payloads are transported in plaintext.

Note: TLS encryption involves only single-sided TLS. This means that the contents are encrypted and sent to the server, but the client won’t send any form of certificate. Therefore, it does not take care of authenticating the client. Client authentication can be configured through mutual TLS (mTLS) or SASL mechanism, which are covered in the Kafka Authentication section .

When TLS is enabled, the client needs to know the root CA certificate used to create the server’s certificate. This is used to validate the certificate sent back by the Kafka server.

Create a certificate named ca.crt that is encoded as a PEM certificate. It is important that the certificate is saved as ca.crt. Otherwise, Seldon Core 2 may not be able to find the certificate. Within the cluster, you can provide the server’s root CA certificate through a secret. For example, a secret named kafka-broker-tls with a certificate.

kubectl create secret generic kafka-broker-tls -n seldon --from-file ./ca.crt

Kafka Authentication

In production environments, Kafka clusters often require authentication, especially when using managed Kafka solutions. Therefore, when installing Seldon Core 2 components, it is crucial to provide the correct credentials for a secure connection to Kafka.

The type of authentication used with Kafka varies depending on the setup but typically includes one of the following:

Simple Authentication and Security Layer (SASL): Requires a username and password.
Mutual TLS (mTLS): Involves using SSL certificates as credentials.
OAuth 2.0: Uses the client credential flow to acquire a JWT token.

These credentials are stored as Kubernetes secrets within the cluster. When setting up Seldon Core 2 you must create the appropriate secret in the correct format and update the components-values.yaml, and install-values files respectively.

When you use SASL as the authentication mechanism for Kafka, the credentials consist of a username and password pair. The password is supplied through a secret.

Note:

Ensure that the field used for the password within the secret is named password. Otherwise, Seldon Core 2 may not be able to find the correct password.
This password must be present in the seldon-logs namespace (or whichever namespace you wish to use for logging) and every namespace containing Seldon Core 2 runtime.

To create a password for Seldon Core 2 in the namespace seldon, run the following command:

```
kubectl create secret generic kafka-sasl-secret --from-literal password=<kafka-password> -n seldon
```

Values in Seldon Core 2

In Seldon Core 2 you need to specify these values in components-values.yaml

security.kafka.sasl.mechanism - SASL security mechanism, e.g. SCRAM-SHA-512
security.kafka.sasl.client.username - Kafka username
security.kafka.sasl.client.secret - Created secret with password
security.kafka.ssl.client.brokerValidationSecret - Certificate Authority of Kafka Brokers

The resulting set of values to include in components-values.yaml is similar to:

security:
  kafka:
    protocol: SASL_SSL
    sasl:
      mechanism: SCRAM-SHA-512
      client:
        username: <kafka-username>      # TODO: Replace with your Kafka username
        secret: kafka-sasl-secret       # NOTE: Secret name from previous step
    ssl:
      client:
        secret:                                   # NOTE: Leave empty
        brokerValidationSecret: kafka-broker-tls  # NOTE: Optional

The security.kafka.ssl.client.brokerValidationSecret field is optional. Leave it empty if your brokers use well known Certificate Authority such as Let’s Encrypt.

When you use OAuth 2.0 as the authentication mechanism for Kafka, the credentials consist of a Client ID and Client Secret, which are used with your Identity Provider to obtain JWT tokens for authenticating with Kafka brokers.

Create a Kubernetes secret kafka-oauth.yaml file.

apiVersion: v1
kind: Secret
metadata:
  name: kafka-oauth
type: Opaque
stringData:
  method: OIDC
  client_id: <client id>
  client_secret: <client secret>
  token_endpoint_url: <token endpoint url>
  extensions: ""
  scope: ""

Store the secret in the seldon namespace to configure with Seldon Core 2.
```
kubectl apply -f kafka-oauth.yaml -n seldon
```
This secret must be present in seldon-logs namespace and every namespace containing Seldon Core 2 runtime.

Client ID, client secret and token endpoint url should come from identity provider such as Keycloak or Azure AD.

Values in Seldon Core 2

In Seldon Core 2 you need to specify these values:

security.kafka.sasl.mechanism - set to OAUTHBEARER
security.kafka.sasl.client.secret - Created secret with client credentials
security.kafka.ssl.client.brokerValidationSecret - Certificate Authority of Kafka brokers

The resulting set of values in components-values.yaml is similar to:

security:
  kafka:
    protocol: SASL_SSL
    sasl:
      mechanism: OAUTHBEARER
      client:
          secret: kafka-oauth                     # NOTE: Secret name from earlier step
    ssl:
      client:
        secret:                                   # NOTE: Leave empty
        brokerValidationSecret: kafka-broker-tls  # NOTE: Optional

The security.kafka.ssl.client.brokerValidationSecret field is optional. Leave it empty if your brokers use well known Certificate Authority such as Let’s Encrypt.

When you use mTLS as authentication mechanism Kafka uses a set of certificates to authenticate the client.

A client certificate, referred to as tls.crt.
A client key, referred to as tls.key.
A root certificate, referred to as ca.crt.

These certificates are expected to be encoded as PEM certificates and are provided through a secret, which can be created in teh namespace seldon:

kubectl create secret generic kafka-client-tls -n seldon \
  --from-file ./tls.crt \
  --from-file ./tls.key \
  --from-file ./ca.crt

This secret must be present in seldon-logs namespace and every namespace containing Seldon Core 2 runtime.

Ensure that the field used within the secret follow the same naming convention: tls.crt, tls.key and ca.crt. Otherwise, Seldon Core 2 may not be able to find the correct set of certificates.

Reference these certificates within the corresponding Helm values for both Seldon Core 2.

Values for Seldon Core 2 In Seldon Core 2 you need to specify these values:

security.kafka.ssl.client.secret - Secret name containing client certificates
security.kafka.ssl.client.brokerValidationSecret - Certificate Authority of Kafka Brokers

The resulting set of values to include in components-values.yaml is similar to:

security:
  kafka:
    protocol: SSL
    ssl:
      client:
        secret: kafka-client-tls                  # NOTE: Secret name from earlier step
        brokerValidationSecret: kafka-broker-tls  # NOTE: Optional

The security.kafka.ssl.client.brokerValidationSecret field is optional. Leave it empty if your brokers use well known Certificate Authority such as Let’s Encrypt.

Example configurations for managed Kafka services

Here are some examples to create secrets for managed Kafka services such as Azure Event Hub, Confluent Cloud(SASL), Confluent Cloud(OAuth2.0).

Prerequisites:

You must use at least the Standard tier for your Event Hub namespace because the Basic tier does not support the Kafka protocol.
Seldon Core 2 creates two Kafka topics for each model and pipeline, plus one global topic for errors. This results in a total number of topics calculated as: 2 x (number of models + number of pipelines) + 1. This topic count is likely to exceed the limit of the Standard tier in Azure Event Hub. For more information, see quota information.

Creating a namespace and obtaining the connection string

These are the steps that you need to perform in Azure Portal.

Create an Azure Event Hub namespace. You need to have an Azure Event Hub namespace. Follow the Azure quickstart documentation to create one. Note: You do not need to create individual Event Hubs (topics) as Seldon Core 2 automatically creates all necessary topics.
Connection string for Kafka Integration. To connect to the Azure Event Hub using the Kafka API, you need to obtain Kafka endpoint and Connection string. For more information, see Get an Event Hubs connection string
Note: Ensure you get the Connection string at the namespace level, as it is needed to dynamically create new topics. The format of the Connection string should be:
```
Endpoint=sb://<namespace>.servicebus.windows.net/;SharedAccessKeyName=XXXXXX;SharedAccessKey=XXXXXX
```

Creating secrets for Seldon Core 2 To store the SASL password in the Kubernetes cluster that run Seldon Core 2, create a secret named azure-kafka-secret for Core 2 in the namespace seldon. In the following command make sure to replace <password> with a password of your choice and <namespace> with the namespace form Azure Event Hub.

kubectl create secret generic azure-kafka-secret --from-literal=<password>="Endpoint=sb://<namespace>.servicebus.windows.net/;SharedAccessKeyName=XXXXXX;SharedAccessKey=XXXXXX" -n seldon

Creating API Keys

These are the steps that you need to perform in Confluent Cloud.

Navigate to Clients > New client and choose a client, for example GO and generate new Kafka cluster API key. For more information, see Confluent documentation.

Confluent generates a configuration file with the details.

Save the values of Key, Secret, and bootstrap.servers from the configuration file.

Creating secrets for Seldon Core 2 These are the steps that you need to perform in the Kubernetes cluster that run Seldon Core 2 to store the SASL password.

Create a secret named confluent-kafka-sasl for Seldon Core 2 in the namespace seldon. In the following command make sure to replace <password> with with the value of Secret that you generated in Confluent cloud.

kubectl create secret generic confluent-kafka-sasl --from-literal password="<password>" -n seldon

Confluent Cloud managed Kafka supports OAuth 2.0 to authenticate your Kafka clients. See Confluent Cloud documentation for further details.

Configuring Identity Provider

In Confluent Cloud Console Navigate to Account & Access / Identity providers and complete these steps:

register your Identity Provider. See Confluent Cloud documentation for further details.
add new identity pool to your newly registered Identity Provider. See Confluent Cloud documentation for further details.
Obtain these details from Confluent Cloud:
- Cluster ID: Cluster Overview → Cluster Settings → General → Identification
- Identity Pool ID: Accounts & access → Identity providers → .
Obtain these details from your identity providers such as Keycloak or Azure AD.
- Client ID
- Client secret
- Token Endpoint URL

If you are using Azure AD you may will need to set scope: api://<client id>/.default.

Creating Kubernetes secret

Create Kubernetes secrets to store the required client credentials. For example, create a kafka-secret.yaml file by replacing the values of <client id>, <client secret>, <token endpoint url>, <cluster id>,<identity pool id> with the values that you obtained from Confluent Cloud and your identity provider.

apiVersion: v1
kind: Secret
metadata:
  name: confluent-kafka-oauth
type: Opaque
stringData:
  method: OIDC
  client_id: <client id>
  client_secret: <client secret>
  token_endpoint_url: <token endpoint url>
  extensions: logicalCluster=<cluster id>,identityPoolId=<identity pool id>
  scope: ""

Provide the secret named confluent-kafka-oauth in the seldon namespace to configure with Seldon Core 2.
```
kubectl apply -f kafka-secret.yaml -n seldon
```
This secret must be present in seldon-logs namespace and every namespace containing Seldon Core 2 runtime.

Configuring Seldon Core 2

To integrate Kafka with Seldon Core 2.

Update the initial configuration.

Note: In these configurations you may need:

to tweak the values for replicationFactor and numPartitions that best suits your cluster configuration.
set the value for username as $ConnectionString this is not a variable.
replace <namespace> with the namespace in Azure Event Hub.

Update the initial configuration for Seldon Core 2 in the components-values.yaml file. Use your preferred text editor to update and save the file with the following content:

controller:
  clusterwide: true

dataflow:
  resources:
    cpu: 500m

envoy:
  service:
    type: ClusterIP

kafka:
  bootstrap: <namespace>.servicebus.windows.net:9093
  topics:
    replicationFactor: 3
    numPartitions: 4    
security:
  kafka:
    protocol: SASL_SSL
    sasl:
      mechanism: "PLAIN"
      client:
        username: $ConnectionString
        secret: azure-kafka-secret
    ssl:
      client:
        secret:
        brokerValidationSecret:
        
opentelemetry:
  enable: false

scheduler:
  service:
    type: ClusterIP

serverConfig:
  mlserver:
    resources:
      cpu: 1
      memory: 2Gi

  triton:
    resources:
      cpu: 1
      memory: 2Gi

serviceGRPCPrefix: "http2-"

Note: In these configurations you may need:

to tweak the values for replicationFactor and numPartitions that best suits your cluster configuration.
replace <username> with thevalue of Key that you generated in Confluent Cloud.
replace <confluent-endpoints> with the value of bootstrap.server that you generated in Confluent Cloud.

Update the initial configuration for Seldon Core 2 Operator in the components-values.yaml file. Use your preferred text editor to update and save the file with the following content:

controller:
  clusterwide: true

dataflow:
  resources:
    cpu: 500m

envoy:
  service:
    type: ClusterIP

kafka:
  bootstrap: <confluent-endpoints>
  topics:
    replicationFactor: 3
    numPartitions: 4
  consumer:
    messageMaxBytes: 8388608
  producer:
    messageMaxBytes: 8388608

security:
  kafka:
    protocol: SASL_SSL
    sasl:
      mechanism: "PLAIN"
      client:
        username: <username>
        secret: confluent-kafka-sasl
    ssl:
      client:
        secret:
        brokerValidationSecret:

opentelemetry:
  enable: false

scheduler:
  service:
    type: ClusterIP

serverConfig:
  mlserver:
    resources:
      cpu: 1
      memory: 2Gi

  triton:
    resources:
      cpu: 1
      memory: 2Gi

serviceGRPCPrefix: "http2-"

Note: In these configurations you may need:

to tweak the values for replicationFactor and numPartitions that best suits your cluster configuration.
replace <confluent-endpoints> with the value of bootstrap.server that you generated in Confluent Cloud.

Update the initial configuration for Seldon Core 2 Operator in the components-values.yaml file. Use your preferred text editor to update and save the file with the following content:

controller:
  clusterwide: true

dataflow:
  resources:
    cpu: 500m

envoy:
  service:
    type: ClusterIP

kafka:
  bootstrap: <confluent-endpoints>
  topics:
    replicationFactor: 3
    numPartitions: 4
  consumer:
    messageMaxBytes: 8388608
  producer:
    messageMaxBytes: 8388608

security:
  kafka:
    protocol: SASL_SSL
    sasl:
      mechanism: OAUTHBEARER
      client:
        secret: confluent-kafka-oauth
    ssl:
      client:
        secret:
        brokerValidationSecret:

opentelemetry:
  enable: false

scheduler:
  service:
    type: ClusterIP

serverConfig:
  mlserver:
    resources:
      cpu: 1
      memory: 2Gi

  triton:
    resources:
      cpu: 1
      memory: 2Gi

serviceGRPCPrefix: "http2-"

To enable Kafka Encryption (TLS) you need to reference the secret that you created in the security.kafka.ssl.client.secret field of the Helm chart values. The resulting set of values to include in components-values.yaml is similar to:

security:
  kafka:
    ssl:
      secret:
        brokerValidationSecret: kafka-broker-tls

Change to the directory that contains the components-values.yaml file and then install Seldon Core 2 operator in the namespace seldon-system.

 helm upgrade seldon-core-v2-components seldon-charts/seldon-core-v2-setup \
 --version 2.8.5 \
 -f components-values.yaml \
 --namespace seldon-system \
 --install

Configuration

Seldon can be configured via various config files.

Kafka Configuration

We allow configuration of the Kafka integration. In general this configuration looks like:

The top level keys are:

topicPrefix : the prefix to add to kafka topics created by Seldon
consumerGroupIdPrefix : the prefix to add to Kafka consumer group IDs created by Seldon
bootstrap.servers : the global bootstrap kafka servers to use
consumer : consumer settings
producer : producer settings
streams : KStreams settings

The consumerGroupIdPrefix will ensure that all consumer groups created have a given prefix.

Kubernetes

For Kubernetes this is controlled via a ConfigMap called seldon-kafka whose default values are defined in the SeldonConfig custom resource. For more details see the componets.yaml file

When the SeldonRuntime is installed in a namespace a configMap will be created with these settings for Kafka configuration.

To customize the settings you can add and modify the Kafka configuration via Helm, for example below is a custom Helm values file that add compression for producers:

https://github.com/SeldonIO/seldon-core/blob/v2/k8s/samples/values-runtime-kafka-compression.yaml

To use this with the SeldonRuntime Helm chart:

helm install seldon-v2-runtime k8s/helm-charts/seldon-core-v2-runtime \
    --namespace seldon-mesh \
    --values k8s/samples/values-runtime-kafka-compression.yaml

Topic and consumer isolation

topicPrefix: set a prefix for all topics
consumerGroupIdPrefix: set a prefix for all consumer groups

An example to set this in the configuration when using the helm installation is showm below for creating the default SeldonConfig:

helm upgrade --install seldon-v2 k8s/helm-charts/seldon-core-v2-setup/ -n seldon-mesh \
    --set controller.clusterwide=true \
    --set kafka.topicPrefix=myorg \
    --set kafka.consumerGroupIdPrefix=myorg

You can find a worked example here.

You can create alternate SeldonConfigs with different values or override values for particular SeldonRuntime installs.

Tracing Configuration

We allow configuration of tracing. This file looks like:

https://github.com/SeldonIO/seldon-core/blob/v2/scheduler/config/tracing-internal.json

{
  "disable": false,
  "otelExporterEndpoint": "otel-collector:4317",
  "otelExporterProtocol": "grpc",
  "ratio": "1"
}

The top level keys are:

enable : whether to enable tracing
otelExporterEndpoint : The host and port for the OTEL exporter
otelExporterProtocol : The protocol for the OTEL exporter. Currently used for jvm-based components only (such as dataflow-engine), because opentelemetry-java-instrumentation requires a http(s) URI for the endpoint but defaults to http/protobuf as a protocol. Because of this, gRPC connections (over http) can only be set up by setting this option to grpc
ratio : The ratio of requests to trace. Takes values between 0 and 1 inclusive.

Kubernetes

For Kubernetes this is controlled via a ConfigMap call seldon-tracing whose default value is shown below:

https://github.com/SeldonIO/seldon-core/blob/v2/scheduler/k8s/config/tracing.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: tracing
data:
  tracing.json: |-
   {
     "enable": true,
     "otelExporterEndpoint": "seldon-collector:4317",
     "otelExporterProtocol": "grpc",
     "ratio": "1"
   }
  OTEL_JAVAAGENT_ENABLED: "true"
  OTEL_EXPORTER_OTLP_ENDPOINT: "http://seldon-collector:4317"
  OTEL_EXPORTER_OTLP_PROTOCOL: "grpc"

Note, this ConfigMap is created via our Helm charts and there is usually no need to modify it manually.

At present Java instrumentation (for the dataflow engine) is duplicated via separate keys.

Managed Kafka

Seldon recommends managed Kafka for production installation. On this page we will demonstrate how you can integrate and configure your managed Kafka with Seldon Core 2.

Securing managed Kafka services

You can secure your Seldon Core 2 integration with managed Kafka services by setting up encryption and authenticaion.

Kafka Encryption (TLS)

In production settings, always set up TLS encryption with Kafka. This ensures that neither the credentials nor the payloads are transported in plaintext.

When TLS is enabled, the client needs to know the root CA certificate used to create the server’s certificate. This is used to validate the certificate sent back by the Kafka server.

Create a certificate named ca.crt that is encoded as a PEM certificate. It is important that the certificate is saved as ca.crt. Otherwise, Seldon Core 2 may not be able to find the certificate. Within the cluster, you can provide the server’s root CA certificate through a secret. For example, a secret named kafka-broker-tls with a certificate.

kubectl create secret generic kafka-broker-tls -n seldon --from-file ./ca.crt

Kafka Authentication

The type of authentication used with Kafka varies depending on the setup but typically includes one of the following:

Simple Authentication and Security Layer (SASL): Requires a username and password.
Mutual TLS (mTLS): Involves using SSL certificates as credentials.
OAuth 2.0: Uses the client credential flow to acquire a JWT token.

When you use SASL as the authentication mechanism for Kafka, the credentials consist of a username and password pair. The password is supplied through a secret.

Note:

Ensure that the field used for the password within the secret is named password. Otherwise, Seldon Core 2 may not be able to find the correct password.
This password must be present in the seldon-logs namespace (or whichever namespace you wish to use for logging) and every namespace containing Seldon Core 2 runtime.

To create a password for Seldon Core 2 in the namespace seldon, run the following command:

```
kubectl create secret generic kafka-sasl-secret --from-literal password=<kafka-password> -n seldon
```

Values in Seldon Core 2

In Seldon Core 2 you need to specify these values in components-values.yaml

security.kafka.sasl.mechanism - SASL security mechanism, e.g. SCRAM-SHA-512
security.kafka.sasl.client.username - Kafka username
security.kafka.sasl.client.secret - Created secret with password
security.kafka.ssl.client.brokerValidationSecret - Certificate Authority of Kafka Brokers

The resulting set of values to include in components-values.yaml is similar to:

security:
  kafka:
    protocol: SASL_SSL
    sasl:
      mechanism: SCRAM-SHA-512
      client:
        username: <kafka-username>      # TODO: Replace with your Kafka username
        secret: kafka-sasl-secret       # NOTE: Secret name from previous step
    ssl:
      client:
        secret:                                   # NOTE: Leave empty
        brokerValidationSecret: kafka-broker-tls  # NOTE: Optional

The security.kafka.ssl.client.brokerValidationSecret field is optional. Leave it empty if your brokers use well known Certificate Authority such as Let’s Encrypt.

Create a Kubernetes secret kafka-oauth.yaml file.

apiVersion: v1
kind: Secret
metadata:
  name: kafka-oauth
type: Opaque
stringData:
  method: OIDC
  client_id: <client id>
  client_secret: <client secret>
  token_endpoint_url: <token endpoint url>
  extensions: ""
  scope: ""

Store the secret in the seldon namespace to configure with Seldon Core 2.
```
kubectl apply -f kafka-oauth.yaml -n seldon
```
This secret must be present in seldon-logs namespace and every namespace containing Seldon Core 2 runtime.

Client ID, client secret and token endpoint url should come from identity provider such as Keycloak or Azure AD.

Values in Seldon Core 2

In Seldon Core 2 you need to specify these values:

security.kafka.sasl.mechanism - set to OAUTHBEARER
security.kafka.sasl.client.secret - Created secret with client credentials
security.kafka.ssl.client.brokerValidationSecret - Certificate Authority of Kafka brokers

The resulting set of values in components-values.yaml is similar to:

security:
  kafka:
    protocol: SASL_SSL
    sasl:
      mechanism: OAUTHBEARER
      client:
          secret: kafka-oauth                     # NOTE: Secret name from earlier step
    ssl:
      client:
        secret:                                   # NOTE: Leave empty
        brokerValidationSecret: kafka-broker-tls  # NOTE: Optional

The security.kafka.ssl.client.brokerValidationSecret field is optional. Leave it empty if your brokers use well known Certificate Authority such as Let’s Encrypt.

When you use mTLS as authentication mechanism Kafka uses a set of certificates to authenticate the client.

A client certificate, referred to as tls.crt.
A client key, referred to as tls.key.
A root certificate, referred to as ca.crt.

These certificates are expected to be encoded as PEM certificates and are provided through a secret, which can be created in teh namespace seldon:

kubectl create secret generic kafka-client-tls -n seldon \
  --from-file ./tls.crt \
  --from-file ./tls.key \
  --from-file ./ca.crt

This secret must be present in seldon-logs namespace and every namespace containing Seldon Core 2 runtime.

Ensure that the field used within the secret follow the same naming convention: tls.crt, tls.key and ca.crt. Otherwise, Seldon Core 2 may not be able to find the correct set of certificates.

Reference these certificates within the corresponding Helm values for both Seldon Core 2.

Values for Seldon Core 2 In Seldon Core 2 you need to specify these values:

security.kafka.ssl.client.secret - Secret name containing client certificates
security.kafka.ssl.client.brokerValidationSecret - Certificate Authority of Kafka Brokers

The resulting set of values to include in components-values.yaml is similar to:

security:
  kafka:
    protocol: SSL
    ssl:
      client:
        secret: kafka-client-tls                  # NOTE: Secret name from earlier step
        brokerValidationSecret: kafka-broker-tls  # NOTE: Optional

The security.kafka.ssl.client.brokerValidationSecret field is optional. Leave it empty if your brokers use well known Certificate Authority such as Let’s Encrypt.

Example configurations for managed Kafka services

Here are some examples to create secrets for managed Kafka services such as Azure Event Hub, Confluent Cloud(SASL), Confluent Cloud(OAuth2.0).

Prerequisites:

You must use at least the Standard tier for your Event Hub namespace because the Basic tier does not support the Kafka protocol.
Seldon Core 2 creates two Kafka topics for each model and pipeline, plus one global topic for errors. This results in a total number of topics calculated as: 2 x (number of models + number of pipelines) + 1. This topic count is likely to exceed the limit of the Standard tier in Azure Event Hub. For more information, see quota information.

Creating a namespace and obtaining the connection string

These are the steps that you need to perform in Azure Portal.

Create an Azure Event Hub namespace. You need to have an Azure Event Hub namespace. Follow the Azure quickstart documentation to create one. Note: You do not need to create individual Event Hubs (topics) as Seldon Core 2 automatically creates all necessary topics.
Connection string for Kafka Integration. To connect to the Azure Event Hub using the Kafka API, you need to obtain Kafka endpoint and Connection string. For more information, see Get an Event Hubs connection string
Note: Ensure you get the Connection string at the namespace level, as it is needed to dynamically create new topics. The format of the Connection string should be:
```
Endpoint=sb://<namespace>.servicebus.windows.net/;SharedAccessKeyName=XXXXXX;SharedAccessKey=XXXXXX
```

kubectl create secret generic azure-kafka-secret --from-literal=<password>="Endpoint=sb://<namespace>.servicebus.windows.net/;SharedAccessKeyName=XXXXXX;SharedAccessKey=XXXXXX" -n seldon

Creating API Keys

These are the steps that you need to perform in Confluent Cloud.

Navigate to Clients > New client and choose a client, for example GO and generate new Kafka cluster API key. For more information, see Confluent documentation.

Confluent generates a configuration file with the details.

Save the values of Key, Secret, and bootstrap.servers from the configuration file.

Creating secrets for Seldon Core 2 These are the steps that you need to perform in the Kubernetes cluster that run Seldon Core 2 to store the SASL password.

Create a secret named confluent-kafka-sasl for Seldon Core 2 in the namespace seldon. In the following command make sure to replace <password> with with the value of Secret that you generated in Confluent cloud.

kubectl create secret generic confluent-kafka-sasl --from-literal password="<password>" -n seldon

Confluent Cloud managed Kafka supports OAuth 2.0 to authenticate your Kafka clients. See Confluent Cloud documentation for further details.

Configuring Identity Provider

In Confluent Cloud Console Navigate to Account & Access / Identity providers and complete these steps:

register your Identity Provider. See Confluent Cloud documentation for further details.
add new identity pool to your newly registered Identity Provider. See Confluent Cloud documentation for further details.
Obtain these details from Confluent Cloud:
- Cluster ID: Cluster Overview → Cluster Settings → General → Identification
- Identity Pool ID: Accounts & access → Identity providers → .
Obtain these details from your identity providers such as Keycloak or Azure AD.
- Client ID
- Client secret
- Token Endpoint URL

If you are using Azure AD you may will need to set scope: api://<client id>/.default.

Creating Kubernetes secret

apiVersion: v1
kind: Secret
metadata:
  name: confluent-kafka-oauth
type: Opaque
stringData:
  method: OIDC
  client_id: <client id>
  client_secret: <client secret>
  token_endpoint_url: <token endpoint url>
  extensions: logicalCluster=<cluster id>,identityPoolId=<identity pool id>
  scope: ""

Provide the secret named confluent-kafka-oauth in the seldon namespace to configure with Seldon Core 2.
```
kubectl apply -f kafka-secret.yaml -n seldon
```
This secret must be present in seldon-logs namespace and every namespace containing Seldon Core 2 runtime.

Configuring Seldon Core 2

To integrate Kafka with Seldon Core 2.

Update the initial configuration.

Note: In these configurations you may need:

to tweak the values for replicationFactor and numPartitions that best suits your cluster configuration.
set the value for username as $ConnectionString this is not a variable.
replace <namespace> with the namespace in Azure Event Hub.

Update the initial configuration for Seldon Core 2 in the components-values.yaml file. Use your preferred text editor to update and save the file with the following content:

controller:
  clusterwide: true

dataflow:
  resources:
    cpu: 500m

envoy:
  service:
    type: ClusterIP

kafka:
  bootstrap: <namespace>.servicebus.windows.net:9093
  topics:
    replicationFactor: 3
    numPartitions: 4    
security:
  kafka:
    protocol: SASL_SSL
    sasl:
      mechanism: "PLAIN"
      client:
        username: $ConnectionString
        secret: azure-kafka-secret
    ssl:
      client:
        secret:
        brokerValidationSecret:
        
opentelemetry:
  enable: false

scheduler:
  service:
    type: ClusterIP

serverConfig:
  mlserver:
    resources:
      cpu: 1
      memory: 2Gi

  triton:
    resources:
      cpu: 1
      memory: 2Gi

serviceGRPCPrefix: "http2-"

Note: In these configurations you may need:

to tweak the values for replicationFactor and numPartitions that best suits your cluster configuration.
replace <username> with thevalue of Key that you generated in Confluent Cloud.
replace <confluent-endpoints> with the value of bootstrap.server that you generated in Confluent Cloud.

Update the initial configuration for Seldon Core 2 Operator in the components-values.yaml file. Use your preferred text editor to update and save the file with the following content:

controller:
  clusterwide: true

dataflow:
  resources:
    cpu: 500m

envoy:
  service:
    type: ClusterIP

kafka:
  bootstrap: <confluent-endpoints>
  topics:
    replicationFactor: 3
    numPartitions: 4
  consumer:
    messageMaxBytes: 8388608
  producer:
    messageMaxBytes: 8388608

security:
  kafka:
    protocol: SASL_SSL
    sasl:
      mechanism: "PLAIN"
      client:
        username: <username>
        secret: confluent-kafka-sasl
    ssl:
      client:
        secret:
        brokerValidationSecret:

opentelemetry:
  enable: false

scheduler:
  service:
    type: ClusterIP

serverConfig:
  mlserver:
    resources:
      cpu: 1
      memory: 2Gi

  triton:
    resources:
      cpu: 1
      memory: 2Gi

serviceGRPCPrefix: "http2-"

Note: In these configurations you may need:

to tweak the values for replicationFactor and numPartitions that best suits your cluster configuration.
replace <confluent-endpoints> with the value of bootstrap.server that you generated in Confluent Cloud.

Update the initial configuration for Seldon Core 2 Operator in the components-values.yaml file. Use your preferred text editor to update and save the file with the following content:

controller:
  clusterwide: true

dataflow:
  resources:
    cpu: 500m

envoy:
  service:
    type: ClusterIP

kafka:
  bootstrap: <confluent-endpoints>
  topics:
    replicationFactor: 3
    numPartitions: 4
  consumer:
    messageMaxBytes: 8388608
  producer:
    messageMaxBytes: 8388608

security:
  kafka:
    protocol: SASL_SSL
    sasl:
      mechanism: OAUTHBEARER
      client:
        secret: confluent-kafka-oauth
    ssl:
      client:
        secret:
        brokerValidationSecret:

opentelemetry:
  enable: false

scheduler:
  service:
    type: ClusterIP

serverConfig:
  mlserver:
    resources:
      cpu: 1
      memory: 2Gi

  triton:
    resources:
      cpu: 1
      memory: 2Gi

serviceGRPCPrefix: "http2-"

To enable Kafka Encryption (TLS) you need to reference the secret that you created in the security.kafka.ssl.client.secret field of the Helm chart values. The resulting set of values to include in components-values.yaml is similar to:

security:
  kafka:
    ssl:
      secret:
        brokerValidationSecret: kafka-broker-tls

Change to the directory that contains the components-values.yaml file and then install Seldon Core 2 operator in the namespace seldon-system.

 helm upgrade seldon-core-v2-components seldon-charts/seldon-core-v2-setup \
 --version 2.8.5 \
 -f components-values.yaml \
 --namespace seldon-system \
 --install