Examples

To see MLServer in action you can check out the examples below. These are end-to-end notebooks, showing how to serve models with MLServer.

Inference Runtimes

If you are interested in how MLServer interacts with particular model frameworks, you can check the following examples. These focus on showcasing the different inference runtimes that ship with MLServer out of the box. Note that, for advanced use cases, you can also write your own custom inference runtime (see the example below on custom models).

• Serving Scikit-Learn models

• Serving XGBoost models

• Serving LightGBM models

• Serving CatBoost models

• Serving MLflow models

• Serving custom models

• Serving Alibi Detect models

• Serving HuggingFace models

MLServer Features

To see some of the advanced features included in MLServer (e.g. multi-model serving), check out the examples below.

• Multi-Model Serving with multiple frameworks

• Loading / unloading models from a model repository

• Content-Type Decoding

• Custom Conda environment

• Serving custom models requiring JSON inputs or outputs

• Serving models through Kafka

• Streaming inference

Tutorials

Tutorials are designed to be beginner-friendly and walk through accomplishing a series of tasks using MLServer (and other tools).

• Deploying a Custom Tensorflow Model with MLServer and Seldon Core