> For the complete documentation index, see [llms.txt](https://docs.seldon.ai/seldon-core-2/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.seldon.ai/seldon-core-2/user-guide/examples/custom-servers.md). # Custom Servers {% hint style="info" %} **Note**: The Seldon CLI allows you to view information about underlying Seldon resources and make changes to them through the scheduler in non-Kubernetes environments. However, it cannot modify underlying manifests within a Kubernetes cluster. Therefore, using the Seldon CLI for control plane operations in a Kubernetes environment is not recommended. For more details, see [Seldon CLI](/seldon-core-2/resources/apis/cli.md). {% endhint %} ```python import os os.environ["NAMESPACE"] = "seldon-mesh" ``` ```python MESH_IP=!kubectl get svc seldon-mesh -n ${NAMESPACE} -o jsonpath='{.status.loadBalancer.ingress[0].ip}' MESH_IP=MESH_IP[0] import os os.environ['MESH_IP'] = MESH_IP MESH_IP ``` ``` '172.18.255.2' ``` ## Custom Server with Capabilities The `capabilities` field replaces the capabilities from the ServerConfig. ```bash cat ./servers/custom-mlserver-capabilities.yaml ``` ```yaml apiVersion: mlops.seldon.io/v1alpha1 kind: Server metadata: name: mlserver-134 spec: serverConfig: mlserver capabilities: - mlserver-1.3.4 podSpec: containers: - image: seldonio/mlserver:1.3.4 name: mlserver ``` ```bash kubectl create -f ./servers/custom-mlserver-capabilities.yaml -n ${NAMESPACE} ``` ``` server.mlops.seldon.io/mlserver-134 created ``` ```bash kubectl wait --for condition=ready --timeout=300s server --all -n ${NAMESPACE} ``` ``` server.mlops.seldon.io/mlserver condition met server.mlops.seldon.io/mlserver-134 condition met server.mlops.seldon.io/triton condition met ``` ```bash cat ./models/iris-custom-requirements.yaml ``` ```yaml apiVersion: mlops.seldon.io/v1alpha1 kind: Model metadata: name: iris spec: storageUri: "gs://seldon-models/mlserver/iris" requirements: - mlserver-1.3.4 ``` ```bash kubectl create -f ./models/iris-custom-requirements.yaml -n ${NAMESPACE} ``` ``` model.mlops.seldon.io/iris created ``` ```bash kubectl wait --for condition=ready --timeout=300s model --all -n ${NAMESPACE} ``` ``` model.mlops.seldon.io/iris condition met ``` ```bash seldon model infer iris --inference-host ${MESH_IP}:80 \ '{"inputs": [{"name": "predict", "shape": [1, 4], "datatype": "FP32", "data": [[1, 2, 3, 4]]}]}' ``` ```json { "model_name": "iris_1", "model_version": "1", "id": "057ae95c-e6bc-4f57-babf-0817ff171729", "parameters": {}, "outputs": [ { "name": "predict", "shape": [ 1, 1 ], "datatype": "INT64", "parameters": { "content_type": "np" }, "data": [ 2 ] } ] } ``` ```bash kubectl delete -f ./models/iris-custom-server.yaml -n ${NAMESPACE} ``` ``` model.mlops.seldon.io "iris" deleted ``` ```bash kubectl delete -f ./servers/custom-mlserver.yaml -n ${NAMESPACE} ``` ``` server.mlops.seldon.io "mlserver-134" deleted ``` ## Custom Server with Extra Capabilities The `extraCapabilities` field extends the existing list from the ServerConfig. ```bash cat ./servers/custom-mlserver.yaml ``` ```yaml apiVersion: mlops.seldon.io/v1alpha1 kind: Server metadata: name: mlserver-134 spec: serverConfig: mlserver extraCapabilities: - mlserver-1.3.4 podSpec: containers: - image: seldonio/mlserver:1.3.4 name: mlserver ``` ```bash kubectl create -f ./servers/custom-mlserver.yaml -n ${NAMESPACE} ``` ``` server.mlops.seldon.io/mlserver-134 created ``` ```bash kubectl wait --for condition=ready --timeout=300s server --all -n ${NAMESPACE} ``` ``` server.mlops.seldon.io/mlserver condition met server.mlops.seldon.io/mlserver-134 condition met server.mlops.seldon.io/triton condition met ``` ```bash cat ./models/iris-custom-server.yaml ``` ```yaml apiVersion: mlops.seldon.io/v1alpha1 kind: Model metadata: name: iris spec: storageUri: "gs://seldon-models/mlserver/iris" server: mlserver-134 ``` ```bash kubectl create -f ./models/iris-custom-server.yaml -n ${NAMESPACE} ``` ``` model.mlops.seldon.io/iris created ``` ```bash kubectl wait --for condition=ready --timeout=300s model --all -n ${NAMESPACE} ``` ``` model.mlops.seldon.io/iris condition met ``` ```bash seldon model infer iris --inference-host ${MESH_IP}:80 \ '{"inputs": [{"name": "predict", "shape": [1, 4], "datatype": "FP32", "data": [[1, 2, 3, 4]]}]}' ``` ```json { "model_name": "iris_1", "model_version": "1", "id": "a3e17c6c-ee3f-4a51-b890-6fb16385a757", "parameters": {}, "outputs": [ { "name": "predict", "shape": [ 1, 1 ], "datatype": "INT64", "parameters": { "content_type": "np" }, "data": [ 2 ] } ] } ``` ```bash kubectl delete -f ./models/iris-custom-server.yaml -n ${NAMESPACE} ``` ``` model.mlops.seldon.io "iris" deleted ``` ```bash kubectl delete -f ./servers/custom-mlserver.yaml -n ${NAMESPACE} ``` ``` server.mlops.seldon.io "mlserver-134" deleted ``` ```python ``` --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.seldon.ai/seldon-core-2/user-guide/examples/custom-servers.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.