Upgrading to 2.4.0

Overview

Upgrading to version 2.4.0 brings important updates and improvements to Seldon Core and the Seldon Enterprise Platform (SEP). These enhancements include better compatibility, a streamlined user experience, and updated dependencies for improved security and performance.

Pre-Upgrade Checklist

Before upgrading, ensure the following:

• Use Seldon Core 2.8.5 or later.

• Use Kubernetes version 1.27 or later to ensure compatibility with Seldon Core 2.

• Use Helm version 3.0 or later to ensure compatibility with Seldon Core 1 and 2.

• Existing deployments are backed up, and current configurations are documented.

Post-Upgrade Testing Recommendations

After upgrading, validate the following:

• Health and readiness of all Seldon Deployments.

• Benchmark inference latency and throughput.

• Verify that logging and monitoring systems are functioning correctly.

Seldon Enterprise Platform Upgrading Guide

SEP 2.4 Release Notes

Inference Protocol Changes

Following the SEP 2.4 release, we recommend using the Open Inference Protocol (OIP) with MLServer for both Seldon Core 1 deployments and Seldon Core 2 pipelines.

Migration Steps

To migrate existing deployments to OIP and MLServer:

• Reconfigure deployment configurations to reference OIP as the inference protocol.

• Check compatibility by reviewing the SEP 2.4 Release Notes.

• Validate deployment behavior post-upgrade using automated tests or sample requests.

• Review the Seldon MLServer documentation.

Changes to the SEP UI

Creating a new Seldon Core 1 Deployment via the SEP UI no longer includes the option to choose from multiple inference protocols. All new deployments now use OIP by default. While this change does not apply to using the backend API or the SEP SDK, we still recommend switching to using OIP and MLServer for all your ML deployments and pipelines.

Existing deployments using other inference protocols will continue to work as expected. Additionally, the Metrics Server is no longer available for new Seldon Core 1 deployments. A new dedicated module is available in early access to support model performance metrics using OIP. For more information, contact your customer success representative.

These changes aim is to provide a consistent and streamlined experience by adopting the industry-standard Open Inference Protocol.

Helm values: Seldon Enterprise Platform

Following the SEP 2.4 release, the default image used for Seldon Enterprise Platform has changed. Use the following Helm command to upgrade to the latest version of Seldon Enterprise Platform:

helm upgrade seldon-deploy seldon-charts/seldon-deploy \
    -f install-values.yaml \
    --namespace=seldon-system \
    --version 2.4.0 \
    --install

Or, you can manually update the image in the install-values.yaml file:

image:
  image: seldonio/seldon-deploy-server:2.4.0

For more information on how to install Seldon Enterprise Platform, see the Seldon Enterprise Platform Installation Documentation.

Helm values: Elasticsearch and Request Logging

Request Logging using OpenSearch

To continue using OpenSearch for request logging, use Metronome version 1.7.1 or earlier.

requestLogger:
  image: seldonio/metronome:1.7.1 # cannot be later than 1.7.1

Request Logging using Elasticsearch

For Elasticsearch request logging, use the new Metronome version 1.8.4 with Elasticsearch version 7.x for production use. This Metronome version resolves several Critical and High CVEs. This is now the recommended approach for request logging in SEP.

requestLogger:
  image: seldonio/metronome:1.8.4

For more information, see the Elasticsearch documentation.