# Inference Runtimes

Inference runtimes allow you to define how your model should be used within MLServer. You can think of them as the **backend glue** between MLServer and your machine learning framework of choice.

![](/files/q2D7OdlLJwLXRTYi1JAi)

Out of the box, MLServer comes with a set of pre-packaged runtimes which let you interact with a subset of common ML frameworks. This allows you to start serving models saved in these frameworks straight away. To avoid bringing in dependencies for frameworks that you don't need to use, these runtimes are implemented as independent (and optional) Python packages. This mechanism also allows you to **rollout your** [**own custom runtimes**](/mlserver/runtimes/custom.md) **very easily**.

To pick which runtime you want to use for your model, you just need to make sure that the right package is installed, and then point to the correct runtime class in your `model-settings.json` file.

## Included Inference Runtimes

| Framework    | Package Name            | Implementation Class                       | Example                                                                                                  | Documentation                                               |
| ------------ | ----------------------- | ------------------------------------------ | -------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------- |
| Scikit-Learn | `mlserver-sklearn`      | `mlserver_sklearn.SKLearnModel`            | [Scikit-Learn example](/mlserver/examples/sklearn.md)                                                    | [MLServer SKLearn](/mlserver/runtimes/sklearn.md)           |
| XGBoost      | `mlserver-xgboost`      | `mlserver_xgboost.XGBoostModel`            | [XGBoost example](/mlserver/examples/xgboost.md)                                                         | [MLServer XGBoost](/mlserver/runtimes/xgboost.md)           |
| Spark MLlib  | `mlserver-mllib`        | `mlserver_mllib.MLlibModel`                | [MLlib example](https://github.com/SeldonIO/MLServer/blob/master/docs-gb/examples/mllib/README.md)       | [MLServer MLlib](/mlserver/runtimes/mllib.md)               |
| LightGBM     | `mlserver-lightgbm`     | `mlserver_lightgbm.LightGBMModel`          | [LightGBM example](/mlserver/examples/lightgbm.md)                                                       | [MLServer LightGBM](/mlserver/runtimes/lightgbm.md)         |
| CatBoost     | `mlserver-catboost`     | `mlserver_catboost.CatboostModel`          | [CatBoost example](https://github.com/SeldonIO/MLServer/blob/master/docs-gb/examples/catboost/README.md) | [MLServer CatBoost](/mlserver/runtimes/catboost.md)         |
| MLflow       | `mlserver-mlflow`       | `mlserver_mlflow.MLflowRuntime`            | [MLflow example](/mlserver/examples/mlflow.md)                                                           | [MLServer MLflow](/mlserver/runtimes/mlflow.md)             |
| Alibi-Detect | `mlserver-alibi-detect` | `mlserver_alibi_detect.AlibiDetectRuntime` | [Alibi-detect example](/mlserver/examples/alibi-detect.md)                                               | [MLServer Alibi-Detect](/mlserver/runtimes/alibi-detect.md) |


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.seldon.ai/mlserver/runtimes.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.