Metrics

Metrics are exposed for scrapping by Prometheus.

Example Installation

We recommend to install kube-prometheus that provides an all-in-one package with the Prometheus operator.

RBAC

You will need to modify the default RBAC installed by kube-prometheus as describedhere.

From the prometheus folder in the project run:

kubectl apply -f rbac/cr.yaml

Monitors

We use a PodMonitor for scrapping agent metrics. The envoy and server monitors are there for completeness but not presently needed.

kubectl apply -f monitors

Includes:

• Agent pod monitor. Monitors the metrics port of server inference pods.

• Server pod monitor. Monitors the server-metrics port of inference server pods.

• Envoy service monitor. Monitors the Envoy gateway proxies.

• Pipeline gateway pod monitor. Monitors the metrics port of pipeline gateway pods.

Pod monitors were chosen as ports for metrics are not exposed at service level as we do not have a top level service for server replicas but 1 headless service per replica. Future discussions could reference this.

Example Grafana Dashboard

Check metrics for more information.

Reference

Prometheus CRDs