Serving Scikit-Learn models
Last updated
Was this helpful?
Last updated
Was this helpful?
Out of the box, mlserver
supports the deployment and serving of scikit-learn
models. By default, it will assume that these models have been .
In this example, we will cover how we can train and serialise a simple model, to then serve it using mlserver
.
The first step will be to train a simple scikit-learn
model. For that, we will use the which trains an SVM model.
Our model will be persisted as a file named mnist-svm.joblib
Now that we have trained and saved our model, the next step will be to serve it using mlserver
. For that, we will need to create 2 configuration files:
settings.json
: holds the configuration of our server (e.g. ports, log level, etc.).
model-settings.json
: holds the configuration of our model (e.g. input type, runtime to use, etc.).
settings.json
model-settings.json
Now that we have our config in-place, we can start the server by running mlserver start .
. This needs to either be ran from the same directory where our config files are or pointing to the folder where they are.
Since this command will start the server and block the terminal, waiting for requests, this will need to be ran in the background on a separate terminal.
We now have our model being served by mlserver
. To make sure that everything is working as expected, let's send a request from our test set.
For that, we can use the Python types that mlserver
provides out of box, or we can build our request manually.
As we can see above, the model predicted the input as the number 8
, which matches what's on the test set.
To save our trained model, we will serialise it using joblib
. While this is not a perfect approach, it's currently the recommended method to persist models to disk in the .