Secrets Management

Seldon Enterprise Platform can create and manage secrets for models or artifacts stored in private buckets, or custom container images stored in a private registry.

If OPA is enabled, you will need write permission to the relevant namespace in order to create or delete secrets in it.

Management Methods

Seldon Enterprise Platform supports the management of secrets through both its UI and its API.

UI

You can manage secrets by navigating to the user icon in the top-right of the Seldon Enterprise Platform UI and clicking on Secrets.

Secrets menu

You will be greeted by a page similar to the following.

Secrets Management page

API

You will be greeted by a page similar to the following.

Secrets management page

You can manage secrets via the API by using the provided endpoints under the SecretsService section in the API docs. Those can be accessed either:

• by navigating to the user icon in the top-right of the Seldon Enterprise Platform UI and clicking on the API docs or

• by navigating to the corresponding docs page.

In addition to this, the endpoints are also made available through the Seldon Enterprise Platform Python SDK.

API docs menu

Bucket Secrets

Bucket stores, also called blob or object stores, provide storage for arbitrary data files. These might be binary files for serialized models, JSON or other textual formats for model settings, or even inference requests for batch jobs or reference data. Examples of bucket stores include Amazon S3, Azure Blob Storage, and Google Cloud Storage (GCS).

Storage buckets can be publicly accessible, such as Seldon's seldon-models bucket, or they can be private and protected with some form of secret or access credentials.

Bucket secrets are used in Seldon for accessing various types of data, such as model artifacts and inference data from private bucket stores. It is a storage initializer that actually uses the secret to perform the download. Secrets need to be provided in a format that the storage initializer understands. The following sections discuss these uses of bucket secrets in more detail.

Seldon Core 1

The default storage initializer for Core 1 is Rclone, although custom initializers can be used instead.

You can easily configure Rclone-compatible bucket secrets for both S3 and GCS from the UI, or you can also create generic Rclone bucket secrets, which unlocks a wide variety of providers, such as Microsoft Azure Blob Storage, Dropbox, and many more.

Seldon Core 1 uses environment variables to pass configuration to the storage initializer for a model. As such, the keys in a Core 1 bucket secret must be valid environment variables for the initializer.

When creating a bucket secret, you will need to specify a Remote name which will be used to create a secret called <remote>-bucket-envvars. This remote name is your choice and determines how you will refer to your bucket provider in storage URIs, i.e. <remote>://<path to bucket object>.

GCS

A GCS bucket secret only requires a Remote name and Account credentials.

Please see the Google Cloud Auth documentation for how to obtain these credentials for your bucket.

For example, to create a GCS bucket secret with a remote called test using a Service Account's private key, in JSON, you need to fill the form in such a way.

GCS secret form

This, in turn, will create a bucket secret called test-bucket-envvars. The secret will be made available for selection in Core 1-related wizards and the remote name can be used for specifying private resources, i.e. test://<path to bucket objects>.

S3

Along with a Remote name, an S3 bucket secret requires an Access key ID and a Secret access key.

Please see the AWS documentation for obtaining these.

S3 secret form

Generic Rclone

You need to select Core 1 as Seldon core and the Generic Rclone as bucket type.

You can then choose a Remote name and Remote type, and provide configuration parameters for that type as key-value pairs.

Each configuration key will become an environment variable.

These keys must be of the form:

RCLONE_CONFIG_<remote name>_<config parameter>

The values <remote name> and <configuration parameter> must be given in uppercase.

Further details on this format can be found in the Rclone documentation.

You can find the appropriate Rclone configuration parameters by selecting your storage provider and checking the available options.

For example, S3 has standard options and advanced options.

Note that you should use the config variant of each option as the <config parameter>.

Rclone secret form

An example for a Dropbox initializer is shown below for a remote called dropbox:

RCLONE_CONFIG_DROPBOX_CLIENT_ID: <client_id_value>
RCLONE_CONFIG_DROPBOX_CLIENT_SECRET: <client_secret_value>

Seldon Core 2

Like Seldon Core 1, both Rclone and custom storage initializers can be used.

When creating a bucket secret, you will need to specify a Remote name which will be used to create a secret called <remote>-bucket-params. This remote name is your choice and determines how you will refer to your bucket provider in storage URIs, i.e. <remote>://<path to bucket object>.

GCS

A GCS bucket secret only requires a Remote name and Account credentials.

Please see the Google Cloud Auth documentation for how to obtain these credentials for your bucket.

For example, to create a GCS bucket secret with a remote called test using a Service Account's private key, in JSON, you need to fill the form in such a way.

GCS secret form

This, in turn, will create a bucket secret called test-bucket-params. The secret will be made available for selection in Core 2-related wizards and the remote name can be used for specifying private resources, i.e. test://<path to bucket objects>.

S3

Along with a Remote name, an S3 bucket secret requires an Access key ID and a Secret access key.

Please see the AWS documentation for obtaining these.

S3 secret form

Generic Rclone

You need to select Core 2 as Seldon core and the Generic Rclone as bucket type.

After choosing a Remote name, you need to specify the Remote type. This input value is Rclone specific i.e. you need to find out that store's remote type, according to Rclone. For example, the remote type for GCS is google cloud storage, whereas for S3 is s3. Those can be found here and here respectively for GCS and S3.

Finally, you need to provide configuration parameters for that type as key-value pairs. You can find the appropriate Rclone configuration parameters by selecting your storage provider and checking the available options. Note that you should use the config variant of each Rclone parameter.

Rclone secret form

Inference Data

Inference data can be retrieved from and written to storage buckets via batch jobs and reference data uploads.

Batch jobs and reference data jobs in Seldon Enterprise Platform use the same approach as Seldon Core 1 for bucket secrets. Namely, they use environment variables to pass configuration from bucket secrets to a storage initializer. The format is exactly the same as for Core 1 and the same secrets can be used.

The default storage initializer for batch and reference data jobs is Rclone.

Registry Secrets

Private container registries can be used when deploying a custom runtime model.

When creating a registry secret, you need to specify the Secret name, which is your choice, and the Account credentials.

Create registry secret form

The format of container registry credentials can be seen by running the following example. For further information on creating and interacting with registry secrets, please refer to the Kubernetes documentation.

docker login
cat ~/.docker/config.json

The above example displays your personal credentials, which should not be used in a production system.

Using Secrets

Once the secrets have been created, you can use them in the Seldon Enterprise Platform UI, e.g. in the Deployment Wizard, and through the API.

For bucket secrets, you need to use the Remote name you chose as the URI prefix. For example, if you named your remote minio then your URI should start with minio://.

Core 1

For Core 1, the Remote name of your bucket secret should match the prefix of your model URI.

Using Core 1 bucket secrets in the deployment wizard

You can use registry secrets for custom model runtimes.

Using Core 1 registry secrets in the deployment wizard

Core 2

For Core 2, the Remote name of your bucket secret should match the prefix of your model URI.

Using Core 2 secrets in the deployment wizard

Batch Jobs

For batch jobs, whether in Core 1 or Core 2, the Remote name of your bucket secret should match the input and output data locations.

Reference Data Jobs

For reference data jobs, whether in Core 1 or Core 2, the Remote name of your bucket secret should match the storage location of your reference data.

Using bucket secrets in the reference data wizard

In the Seldon Enterprise Platform UI, you will only be shown secrets with the appropriate format for that context. For example, when creating a Core 1 deployment you will only see env-var bucket secrets.

For backwards compatibility, any bucket secrets that are not labelled with their format are assumed to be in the env-var format.