> For the complete documentation index, see [llms.txt](https://docs.seldon.ai/mpm/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.seldon.ai/mpm/getting-started/installation.md).

# Installation

This section provides instructions on how to install the Model Performance Metrics module on a Kubernetes cluster. It is deployed using a Helm chart, which is stored in a private Google Artifact Registry, managed by Seldon. The Helm chart makes use of a Docker image, which is also stored in the same registry.

## Prerequisites

{% hint style="warning" %}
This guide assumes you have received Seldon Artifact Registry access credentials in a `JSON` format. If this is not the case, please [reach out to us](https://www.seldon.io/contact).
{% endhint %}

Before installing the module, ensure you have the following:

* Access to the private Google Artifact Registry where the Helm chart and the Docker image are stored.
* Kubernetes cluster configured and running.
* [Seldon Core 2](https://docs.seldon.ai/seldon-core-2) installed and running on the Kubernetes cluster.
* Kafka installed, configured with [Seldon Core 2](https://docs.seldon.ai/seldon-core-2) and running on the Kubernetes cluster or reachable from it.
* PostgreSQL installed and running on the Kubernetes cluster reachable from it.
* Istio installed and running on the Kubernetes cluster (Optional). Only needed if you wish to make the module externally accessible from the cluster. Requires an Istio Gateway to be set up as well.

### Components compatibility matrix

| Component     | Supported Versions               | Importance                                                                             |
| ------------- | -------------------------------- | -------------------------------------------------------------------------------------- |
| Kubernetes    | minor versions of 1.27 or later  | Required for deploying the Helm chart                                                  |
| Seldon Core 2 | minor versions of 2.8 or later   | Required for producing inference response events to Kafka                              |
| Kafka         | minor versions of 3.3.1 or later | Required for Kafka Consumer, consuming inference responses produced by Seldon Core 2   |
| PostgreSQL    | version 10 - version 15          | Required for storing metrics data, inference responses and more                        |
| Istio         | minor versions of 1.17 or later  | Optional for exposing the Model Performance Metrics module externally from the cluster |

#### Required CLIs

Ensure you have the following CLIs installed on your local machine:

* kubectl
* docker
* Helm
* gcloud

## Accessing the artifacts

{% hint style="info" %}
**Note**

In the following examples, we assume you have received the credentials in a `JSON` format and stored them in a file named `credentials.json`.
{% endhint %}

### Authenticate to the Google Artifact Registry

To be able to pull the Helm chart and the Docker image from the Google Artifact Registry, you need to authenticate with the Docker CLI first.

```bash
REGISTRY=europe-west2-docker.pkg.dev
cat credentials.json | docker login -u _json_key --password-stdin ${REGISTRY}
```

The result of this command should be a successful login to the Google Artifact Registry:

```bash
Login Succeeded
```

To pull the Docker image, you can use the following command:

```bash
docker pull europe-west2-docker.pkg.dev/seldon-registry/metrics-server/metrics-server:0.1.0
```

The result of this command should be a successful pull of the Docker image.

To be able to pull the Helm chart from the Google Artifact Registry, you can use the following command:

```bash
helm pull oci://europe-west2-docker.pkg.dev/seldon-registry/charts/metrics-server --version 0.1.0 --untar
```

This command will pull the Helm chart from the Google Artifact Registry and extract it in the current directory. You can then navigate to the directory and inspect the Helm chart.

{% hint style="info" %}
**Note**

If you prefer to pull the Helm chart in a tar format, you can remove the `--untar` flag from the command.
{% endhint %}

### Helm Chart Components

The Helm chart consists of the following components:

* `Chart.yaml`: Contains the metadata of the Helm chart.
* `templates/`: Contains the Kubernetes resources to be deployed on the cluster.
  * `deployment.yaml`: Contains the deployment configuration for the service.
  * `service.yaml`: Contains the service configuration for the service.
  * `virtualservice.yaml`: Contains the virtual service(Istio) configuration for the service (optional).
* `values.yaml`: Contains the default values for the Helm chart.

## Configuring the Helm Chart

### Helm Chart Values

After downloading the Helm chart, you can inspect the `values.yaml` file to see the default values for the Helm chart. There are values that must be set to install the Chart, values that must be set for the module to initialise successfully and values that are optional. The following table lists all the possible values, their description and their importance.

#### Chart

| Helm Value Key      | Description                                                                                                                                     | Importance                    |
| ------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------- |
| appName             | The name of the application. This value is used to label the resources created by the Helm chart.                                               | **Required to install chart** |
| namespace           | The namespace where the **Deployment**, the **Service** and, optionally, the **VirtualService** will be deployed.                               | **Required to install chart** |
| image               | The Docker image to be used by the service. This value should be the full path to the image in the Google Artifact Registry, excluding the tag. | **Required to install chart** |
| imageTag            | The Docker image version tag. We use [Semantic Versioning](https://semver.org/) for tagging the Docker image.                                   | **Required to install chart** |
| imagePullSecretName | The name of the Kubernetes secret that contains the credentials to pull the Docker image from the Google Artifact Registry.                     | **Required to install chart** |

#### General

| Helm Value Key | Description                                                                                                                                                                                          | Importance              |
| -------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------- |
| logLevel       | The log level for the module. The value is case-insensitive and can be either one of: `"disabled"`, `"trace"`, `"debug"`, `"info"`, `"warn"`, `"error"`, `"fatal"`, `"panic"`. Defaults to `"info"`. | Optional for the module |

#### Virtual Service

| Helm Value Key              | Description                                                                                                                                                               | Importance                                |
| --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------- |
| istioVirtualService.create  | Whether or not to create an Istio VirtualService. Defaults to `false`.                                                                                                    | Optional to create the **VirtualService** |
| istioVirtualService.gateway | The gateway name for the Istio VirtualService. Must be supplied if `istioVirtualService.create` is true. An example default value could be `istio-system/seldon-gateway`. | Optional to create the **VirtualService** |
| istioVirtualService.path    | The prefix path which follows the external load balancer IP and before the Model Performance Metrics module's endpoints. An example value could be `/metrics-server/`.    | Optional to create the **VirtualService** |

#### Kafka Metadata

| Helm Value Key                        | Description                                                                                                                                                             | Importance              |
| ------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------- |
| kafka.metadata.retryMax               | The total number of times to retry a metadata request when the cluster is in the middle of a leader election. Defaults to `3`.                                          | Optional for the module |
| kafka.metadata.retryBackoff           | How long to wait for leader election to occur before retrying. Defaults to `250ms`.                                                                                     | Optional for the module |
| kafka.metadata.refreshFrequency       | How frequently to refresh the cluster metadata in the background. Defaults to `10s`.                                                                                    | Optional for the module |
| kafka.metadata.full                   | Whether or not to maintain a full set of metadata for all topics, or just the minimal set that has been necessary so far. Defaults to `true`.                           | Optional for the module |
| kafka.metadata.allowAutoTopicCreation | Whether or not to allow auto-create topics in metadata refresh. For production environments, it is recommended to disable automatic topic creation. Defaults to `true`. | Optional for the module |

#### Kafka Auth

| Helm Value Key                     | Description                                                                                                            | Importance                        |
| ---------------------------------- | ---------------------------------------------------------------------------------------------------------------------- | --------------------------------- |
| kafka.auth.sasl.enabled            | Whether or not to enable `SASL_SSL`(with SCRAM) authentication. Defaults to `false`.                                   | Optional to authenticate securely |
| kafka.auth.sasl.mechanism          | The SASL mechanism to use. Accepted values are `"SCRAM-SHA-512"` and `"SCRAM-SHA-256"`. Defaults to `"SCRAM-SHA-512"`. | Optional to authenticate securely |
| kafka.auth.sasl.username           | The username to use for SASL authentication. Defaults to `"seldon"`.                                                   | Optional to authenticate securely |
| kafka.auth.sasl.passwordSecretName | The name of the secret containing the password for SASL authentication. Defaults to `"kafka-client-auth"`.             | Optional to authenticate securely |
| kafka.auth.sasl.passwordSecretKey  | The key in the secret containing the password for SASL authentication. Defaults to `"password"`.                       | Optional to authenticate securely |
| kafka.auth.sasl.caCertLocation     | The location of the CA certificate file. Defaults to `"/tmp/certs/kafka/broker/ca.crt"`.                               | Optional to authenticate securely |
| kafka.auth.sasl.caCertSecretName   | The name of the secret containing the CA certificate. Defaults to `"kafka-broker-auth"`.                               | Optional to authenticate securely |
| kafka.auth.sasl.caCertSecretKey    | The key in the secret containing the CA certificate. Defaults to `"ca.crt"`.                                           | Optional to authenticate securely |

#### Kafka Consumer

| Helm Value Key                            | Description                                                                                                                                                                                                                                | Importance                     |
| ----------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------ |
| kafka.consumer.bootstrapServer            | <p>The Kafka bootstrap server address, including a port. This value is used by the Kafka consumer to connect to the Kafka broker.<br>Defaults to <code>"seldon-kafka-bootstrap.kafka.svc.cluster.local:9092"</code>.</p>                   | **Required to run the module** |
| kafka.consumer.topicPrefix                | The prefix of the topics MPM will attempt to start consuming messages from. Defaults to `seldon`.                                                                                                                                          | Optional for the module        |
| kafka.consumer.pollInterval               | How often the Kafka consumer within MPM will check if there are new topics matching [a regex](/mpm/getting-started/installation/kafka-consumer.md#message-consumption-workflow) and start consuming messages from them. Defaults to `10s`. | Optional for the module        |
| kafka.consumer.offsets.autoCommitEnable   | Whether or not to auto-commit updated offsets back to the broker. Defaults to `true`.                                                                                                                                                      | Optional for the module        |
| kafka.consumer.offsets.autoCommitInterval | How frequently to commit updated offsets. Ineffective unless auto-commit is enabled. Defaults to `1s`.                                                                                                                                     | Optional for the module        |
| kafka.consumer.offsets.initial            | The initial offset to use if no offset was previously committed. Accepted values are `"oldest"` and `"newest"`. Defaults to `"newest"`.                                                                                                    | Optional for the module        |

#### PostgreSQL

| Helm Value Key        | Description                                                                                                                                                                                                                 | Importance                     |
| --------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------ |
| postgres.host         | The host of the PostgreSQL database, excluding the port number. Defaults to `"localhost"`.                                                                                                                                  | **Required to run the module** |
| postgres.port         | The port of the PostgreSQL database. Defaults to `5432`.                                                                                                                                                                    | **Required to run the module** |
| postgres.databaseName | The name of the database to use. Defaults to `"seldon_metrics"`.                                                                                                                                                            | **Required to run the module** |
| postgres.user         | The username to use for authentication. Defaults to `"seldon"`.                                                                                                                                                             | **Required to run the module** |
| postgres.secretName   | The name of the secret containing the password. Defaults to `"password"`.                                                                                                                                                   | **Required to run the module** |
| postgres.secretKey    | The key in the secret containing the password. Defaults to `"password"`.                                                                                                                                                    | **Required to run the module** |
| postgres.sslmode      | Controls the encryption and verification of PostgreSQL connections. Defaults to `"prefer"`. Available options can be found [here](https://www.postgresql.org/docs/15/libpq-ssl.html), except `verify-ca` and `verify-full`. | Optional for the module        |

An example `values.yaml` file with the minimal values required to install the Helm chart is shown below:

```yaml
appName: metrics-server
namespace: seldon-system
image: europe-west2-docker.pkg.dev/seldon-registry/metrics-server/metrics-server
imageTag: 0.1.0
imagePullSecretName: seldon-registry

kafka:
  consumer:
    bootstrapServer: "localhost:9092"

postgres:
  host: "localhost"
  port: 5432
  databaseName: "seldon_metrics"
  user: "seldon"
  secretName: "seldon-metrics-postgres"
  secretKey: "password"
```

{% hint style="info" %}
**Notes**

* When `kafka.consumer.bootstrapServer` is provided and `kafka.auth.sasl.enabled` is set to `false`, the module will attempt to establish a connection to the Kafka broker using the `PLAINTEXT` protocol.
* When `kafka.auth.sasl.enabled` is set to `true`, the module will attempt to establish a connection to the Kafka broker using the `SASL_SSL` protocol. In such cases, the port of the Kafka bootstrap server could be different. For example, 9093 instead of 9092.
* The Kafka consumer only supports a single replicate of the module's `Deployment` definition which is reflected in its `spec` object: `replica:1`.
  {% endhint %}

### Creating a Kubernetes Secret (Optional)

After you have successfully pulled the Docker image and the Helm chart, and you have configured the Helm values to suit your environment, optionally you may have to create a Kubernetes secret so that the Helm `Deployment` can pull the Docker image from the Google Artifact Registry.

To create a Kubernetes secret with the credentials, you can use the following command:

```bash
NAMESPACE=seldon-system
REGISTRY=europe-west2-docker.pkg.dev
CREDENTIALS=$(cat credentials.json)
SECRET_NAME=seldon-registry

kubectl create secret docker-registry ${SECRET_NAME} \
	--docker-server="${REGISTRY}" \
	--docker-username="_json_key" \
	--docker-password="${CREDENTIALS}" \
	--dry-run=client -o yaml | kubectl apply -n ${NAMESPACE} -f -
```

You can verify the secret was created successfully by running:

```bash
kubectl get secret ${SECRET_NAME} -n ${NAMESPACE}
```

## Installing the Helm Chart

After you have configured the Helm chart values and created the Kubernetes secret, you can install the Helm chart on the Kubernetes cluster. To do so, using the modified `values.yaml`, you can use the following command:

```bash
helm install metrics-server ./metrics-server
```

To uninstall the Helm chart, you can use the following command:

```bash
helm uninstall metrics-server
```

## Ingress Controller (Optional)

An ingress controller functions as a reverse proxy and load balancer, implementing a Kubernetes Ingress. It adds an abstraction layer for traffic routing by receiving traffic from outside the Kubernetes platform and load balancing it to Pods running within the Kubernetes cluster.

The Model Performance Metrics will work with any service mesh or ingress controller, offering flexibility in your deployment setup. In this example, we are going to use Istio to expose the module and make it accessible from outside the Kubernetes cluster.

### Install Istio Base Component

1. Add the Istio Helm charts repository and update it:

```bash
helm repo add istio https://istio-release.storage.googleapis.com/charts
helm repo update
```

2. Create the istio-system namespace where Istio components are installed:

```bash
kubectl create namespace istio-system
```

3. Install the base component:

```bash
helm install istio-base istio/base -n istio-system
```

4. Install Istiod, the Istio control plane:

```bash
helm install istiod istio/istiod -n istio-system --wait
```

### Install Istio Ingress Gateway deployment

1. Install the Istio Ingress Gateway deployment:

```bash
helm install istio-ingressgateway istio/gateway -n istio-system
```

2. Verify it is installed:

```bash
kubectl get svc istio-ingressgateway -n istio-system
```

This should return details of the Istio Ingress Gateway pod of type `LoadBalancer`, including the external IP address.

{% hint style="info" %}
**Notes**

If your Istio Ingress Gateway pod shows EXTERNAL-IP: , it means that Kubernetes is still waiting to assign an external IP address to the Istio Ingress Gateway `LoadBalancer` service.

When using a cloud provider:

* If the cloud provider of choice supports automatic `LoadBalancer` provisioning, Kubernetes should create one and assign an external IP after a few minutes.
* If the cloud provider of choice does not support automatic `LoadBalancer` provisioning, either you have to manually reserve and assign an external IP or manually create one. This will depend on the cloud provider's documentation.

When using a local Kubernetes cluster:

* In a local Kubernetes cluster(e.g. Minikube, Kind), there is **no external cloud provider** to provision the IP automatically.
* You can install MetalLB to provide a load balancer implementation for bare metal Kubernetes clusters.
  {% endhint %}

#### Install MetalLB in a local Kubernetes cluster (Optional)

```bash
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/main/config/manifests/metallb-native.yaml
```

This will install MetalLB in the `metallb-system` namespace.

* Determine the IPv4 subnet of the corresponding docker network(assuming the cluster is running in Docker with `kind`):

```bash
docker network inspect kind | jq -r '.[0].IPAM.Config[0].Subnet'
```

An example output could be `172.18.0.0/16`

* Create a MetalLB configuration manifest, named `metallb-config.yaml` and replace x.y with the subnet values, in this case `172.18`:

```yaml
---
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: kind-pool
  namespace: metallb-system
spec:
  addresses:
  - x.y.255.1-x.y.255.255

---

apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: advertisement
  namespace: metallb-system
```

After both resources have been applied, MetalLB should be able to assign an external IP address to the Istio Ingress Gateway `LoadBalancer` service.\
To verify this, you can run:

```bash
kubectl get svc istio-ingressgateway -n istio-system
```

### Install Istio Gateway

You will need to create a Kubernetes `Gateway` resource to expose the Model Performance Metrics module to the outside world. The `Gateway` resource will be used by the `VirtualService` resource to route traffic to the module.

```yaml
apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  name: seldon-gateway
  namespace: istio-system
spec:
  selector:
    istio: ingressgateway
  servers:
    - port:
        number: 80
        name: http
        protocol: HTTP
      hosts:
        - "*"
```

### Install the Virtual Service for the Model Performance Metrics module

* You can do so by setting the following values in the `values.yaml` file:

```yaml
istioVirtualService:
  create: true
  gateway: istio-system/seldon-gateway # referencing the name of the Gateway resource
  path: /metrics-server/ # the prefix path to the module's endpoints
```

You can then install the Helm chart with the modified `values.yaml` file:

```bash
helm install metrics-server ./metrics-server
```

* or by creating a `VirtualService` resource manually in a file called `virtual-service.yaml`:

You will have to update the corresponding values in the template using the aforementioned yaml, and then apply the `VirtualService` resource to the Kubernetes cluster:

```bash
kubectl apply -f virtual-service.yaml
```

### Verify the installation

You can verify the Model Performance Metrics module is accessible from outside the Kubernetes cluster.

Obtain the external IP address of the Istio Ingress Gateway service:

```bash
kubectl get svc istio-ingressgateway -n istio-system -o jsonpath='{.status.loadBalancer.ingress[0].ip}'
```

Then, you can access the Model Performance Metrics module by navigating to `http://<EXTERNAL_IP>/metrics-server/api/v1/pipeline/subscriptions` in your browser, which should return an empty JSON list with 200 Status Code:

```bash
curl -v http://<EXTERNAL_IP>/metrics-server/api/v1/pipeline/subscriptions
```