Edit

Model Subscriptions

Overview

A subscription is a registration of a model output schema, accompanied by additional metadata for the model. It consists of a combination of the model name, pipeline name, and namespace, and the model output features. A subscription is required to parse the inference responses for a model when supplying Ground Truth and calculating Metrics for that model.

Notes

  • A subscription is not required to store inference responses for models.

  • The behavior for consuming logs from Kafka topics is configured using an environment variable set during installation. Please refer to the Installation documentation for more details.

Importance of Subscriptions

To effectively work with the output features of a model, a subscription is required. The subscription facilitates:

  • Interpreting Inference Responses: Gain a clear understanding of the response format returned by the model.

  • Extracting Predicted Values: Easily pull relevant output features from the inference response for further analysis.

  • Calculating Metrics: Use the predicted values to compute important performance metrics, helping you evaluate the model’s effectiveness.

Output Feature Types

The service supports three main types of output features when registering a subscription for a model, each serving distinct purposes based on the nature of the prediction task:

  1. Numeric

    • Definition: A numeric output feature represents continuous values, such as Length or Price.

    • Typical Use Cases: These features are commonly used in regression tasks, where the goal is to predict a numerical outcome based on input data.

  2. Categorical

    • Definition: A categorical output feature describes discrete categories, such as Education level or Marital Status.

    • Typical Use Cases: Categorical features are often utilized in classification tasks, where the objective is to assign input data to specific categories or groups.

  3. Categorical Probability

    • Definition: A categorical probability output feature provides discrete categories (e.g., Car, Truck) along with associated probabilities for each category.

    • Typical Use Cases: This feature type is frequently used in classification tasks to convey the likelihood of an input belonging to each category, allowing for more nuanced predictions.

Unique Model Identification

To ensure precise interactions with a specific model, each model is uniquely identified by a combination of three components:

  • Model Name - The name of the model, contained within a Pipeline, as defined in the metadata.name field of the Seldon Core 2 Model CRD.

  • Pipeline Name - The name of the Pipeline containing the model, as defined in the metadata.name field of the Seldon Core 2 Pipeline CRD.

  • namespace - The Kubernetes namespace where the Pipeline resides, as defined in the metadata.namespace field of the Seldon Core 2 Pipeline CRD.

A 409 error code is returned if a subscription with the same model name, pipeline name, and namespace already exists. For more details, refer to the Subscriptions API.

Common Subscription Operations

  1. Subscribe to a Model

    • Purpose: Create a subscription for a model, enabling the service to interpret inference responses, when supplying Ground Truth and calculating Metrics for said model.

    • Steps:

      1. Specify Model: Provide the model name, pipeline name, and namespace.

      2. Define Output Features: Specify the output features you expect to receive in the inference response.

      3. Create Subscription: Submit the request to create the subscription.

  2. List Subscriptions

    • Purpose: View all active model subscriptions (supplying a pipeline name and/or namespace is optional).

    • Steps:

      1. Retrieve Subscriptions: Fetch a list of all subscriptions.

      2. Review Details: Examine the output features and other subscription information.

  3. Unsubscribe from a Model

    • Purpose: Indicate your intent to stop supplying Ground Truth and calculating Metrics.

    • Steps:

      1. Retrieve Subscription ID: Fetch the unique subscription ID associated with the model.

      2. Delete Subscription: Submit the request to delete the subscription.

Examples

Notes

In the following Python examples:

  • the value of the CLUSTER_IP variable should be replaced with the IP address of the metrics server.

  • the name of each feature in the featuresMetadata list does not need to match the output tensor name in the model's inference responses.

Subscribe to a Model with a Categorical Output Feature

For this example, it is assumed that there exists an ML model based on the Iris dataset. The model is trained to predict the species of an iris flower based on its sepal and petal dimensions. This model can be deployed using Seldon Core 2 and MLServer with SKLearn capabilities. The model artefacts can be found at this link: Iris Model.

An example of an inference request for this model is shown below:

{
  "inputs": [
    {
      "name": "predict",
      "data": [0.38606369295833043, 0.006894049558299753, 0.6104082981607108, 0.3958954239450676],
      "datatype": "FP64",
      "shape": [1, 4]
    }
  ]
}

This model takes in 4 input features, describing the sepal length, sepal width, petal length, and petal width in centimeters of an iris flower.

An example of an inference response for this model is shown below:

{
  "outputs": [
    {
      "name": "predict",
      "datatype": "INT32",
      "shape": [1, 1],
      "data": [1]
    }
  ]
}

This model returns a single output feature named predict, which is a categorical feature representing the predicted species of the iris flower. The feature is encoded as an integer value, where 0 corresponds to the Setosa species, 1 corresponds to the Versicolor species, and 2 corresponds to the Virginica species. In this case, the model predicts that the iris flower belongs to the Versicolor species, which corresponds to the category index value 1.

The following example demonstrates how to construct a POST request to subscribe to a Model named iris-model, contained within a Pipeline named iris-pipeline in the seldon namespace. The model output contains a single categorical feature named Iris Species, which corresponds to the species of the iris flower.

url = f'http://{CLUSTER_IP}/metrics-server/api/v1/pipeline/model/subscribe'
data = {
    "modelName": "iris-model",
    "pipelineName": "iris-pipeline",
    "featuresMetadata": [
        {
            "categorical": {
                "categories": [
                    "Setosa",
                    "Versicolor",
                    "Virginica"
                ],
                "name": "Iris Species"
            }
        }
    ],
    "namespace": "seldon",
}

response = requests.post(url, json=data)

Subscribe to a Model with a Category Probability Output Feature

For this example, it is assumed that there exists an ML model based on the CIFAR-10 dataset. The model is trained to predict the likelihood of an input image belonging to each of the ten CIFAR-10 classes. This model can be deployed on an inference server having tensorflow capabilities by using the artefacts located at this link: CIFAR-10 Model.

An example of an inference request, containing a frog image, for this model can be seen in this file:

85KB
cifar10-frog-oip.json

An example of an inference response for this model is shown below:

{
  "outputs": [
    {
      "data": [
        1.4500083e-8,
        1.2525716e-9,
        1.6298381e-7,
        0.115293,
        1.7431331e-7,
        0.000006185637,
        0.8847003,
        6.073888e-9,
        7.4378995e-8,
        4.7317203e-9
      ],
      "datatype": "FP32",
      "name": "fc10",
      "shape": [1, 10]
    }
  ]
}

This model returns a single output feature named fc10, which is a categorical probability feature representing the likelihood of the input image belonging to each of the ten CIFAR-10 classes. The feature is encoded as an array of floating-point values and in this case, the model predicts that the input image is most likely a frog, with a probability of approximately 0.8847.

The following example demonstrates how to construct a POST request to subscribe to a Model named cifar10-model, contained within a Pipeline named cifar10-pipeline in the seldon namespace. The model output contains a single categorical probability feature named CIFAR-10 Class.

url = f'http://{CLUSTER_IP}/metrics-server/api/v1/pipeline/model/subscribe'
data = {
    "modelName": "cifar10-model",
    "pipelineName": "cifar10-pipeline",
    "featuresMetadata": [
        {
            "categoricalProbaTensor": {
                "categories": [
                    "airplane",
                    "automobile",
                    "bird",
                    "cat",
                    "deer",
                    "dog",
                    "frog",
                    "horse",
                    "ship",
                    "truck"
                ],
                "name": "CIFAR-10 Class"
            }
        }
    ],
    "namespace": "seldon",
}

Subscribe to a Model with a Numeric Output Feature

For this example, it is assumed that there exists an ML model based on the California Housing dataset. The model is trained to predict the median house value in California districts in the hundreds of thousands of dollars based on various features. This model can be deployed using Seldon Core 2 and MLServer with SKLearn capabilities. The model artefacts can be found at this link: Iris Model.

An example of an inference request for this model is shown below:

{
  "inputs": [
    {
      "name": "predict",
      "data": [4.32, 41, 4.54, 2.23, 4324, 2.02, 37.88, -122.23],
      "datatype": "FP64",
      "shape": [1, 8]
    }
  ]
}

This model takes in eight input features, describing various attributes of a California district, such as the median income in a block group, median house age in a block group, average number of rooms and bedrooms in a household, block group population, average number of household members, block group latitude, and block group longitude, respectively.

An example of an inference response for this model is shown below:

{
  "outputs": [
    {
      "data": [3.594715773483869],
      "datatype": "FP64",
      "name": "predict",
      "shape": [1, 1]
    }
  ]
}

This model returns a single output feature named predict, which is a numeric feature representing the predicted median house value in hundreds of thousands of dollars. In this case, the model predicts that the median house value in the California district is approximately $359,471.58.

The following example demonstrates how to construct a POST request to subscribe to a Model named cali-housing-model, contained within a Pipeline named cali-housing-pipeline in the seldon namespace. The model output contains a single numeric feature named Median House Value.

url = f'http://{CLUSTER_IP}/metrics-server/api/v1/pipeline/model/subscribe'
data = {
    "modelName": "cali-housing-model",
    "pipelineName": "cali-housing-pipeline",
    "featuresMetadata": [
        {
            "numeric": {
                "name": "Median House Value"
            }
        }
    ],
    "namespace": "seldon",
}

response = requests.post(url, json=data)

List Subscriptions

The following example demonstrates how to construct a GET request to list all active subscriptions for a specificPipeline named iris-pipeline in the seldon namespace.

url = f"http://{CLUSTER_IP}/metrics-server/api/v1/pipeline/subscriptions"
params = {
    "namespace": "seldon",
    "pipelineName": "iris-pipeline",
}

response = requests.get(url, params=params)

The parameters namespace and pipelineName are optional and can be omitted to retrieve all active subscriptions.

Unsubscribe from a Model

The following example demonstrates how to construct a DELETE request to unsubscribe from a Model named iris-model, contained within a Pipeline named iris-pipeline in the seldon namespace. The id of the subscription you wish to delete can be obtained by listing the subscriptions as shown in the previous example, or by recording it upon subscribing to the model.

url = f"http://{CLUSTER_IP}/metrics-server/api/v1/pipeline/model/unsubscribe"
params = {
    "subscriptionId": 123,
}

response = requests.delete(url, params=params)

API Documentation

To facilitate integration with the service, Seldon provides comprehensive API documentation using Swagger. This documentation allows you to inspect the available endpoints, request parameters, and response formats in detail.

You can access the API Swagger documentation by navigating to the following page: API Documentation.

PreviousConsuming Kafka EventsNextGround Truth

Last updated

Was this helpful?