Overview

The Model Performance Metrics (MPM) module enables monitoring and evaluation of machine learning models in production. It processes predictions and corresponding feedback to calculate various metrics, ensuring the ongoing validation of model suitability. MPM integrates seamlessly with Seldon Core 2 and includes support for defining Pipelines through custom resource definitions (CRDs).

Usage of the MPM module

Metrics should be interpreted with an awareness of their context and limitations, including:

• Feedback might not reflect the true ground truth distribution, as false predictions may be over-reported compared to true ones.

• Small feedback volumes can lead to statistically insignificant metrics.

• A change to the model artifact that underlies a particular URI will not be known to the MPM module. Therefore, a change of this kind will produce misleading metrics.