# Triton Inference Server

If you have a model that can be run on [NVIDIA Triton Inference Server](https://github.com/triton-inference-server/server) you can use Seldon's Prepacked Triton Server.

Triton has multiple supported backends including support for TensorRT, Tensorflow, PyTorch and ONNX models. For further details see the [Triton supported backends documentation](https://github.com/triton-inference-server/backend#where-can-i-find-all-the-backends-that-are-available-for-triton).

## Example

```yaml
apiVersion: machinelearning.seldon.io/v1alpha2
kind: SeldonDeployment
metadata:
  name: triton
spec:
  protocol: v2
  predictors:
  - graph:
      implementation: TRITON_SERVER
      modelUri: gs://seldon-models/trtis/simple-model
      name: simple
    name: simple
    replicas: 1
```

See more deployment examples in [triton examples](https://github.com/SeldonIO/seldon-core/blob/master/docs-gb/examples/triton_examples.html) and [protocol examples](https://github.com/SeldonIO/seldon-core/blob/master/docs-gb/examples/protocol_examples.html).

See also:

* [Tensorflow MNIST - e2e example with MinIO](https://github.com/SeldonIO/seldon-core/blob/master/docs-gb/examples/triton_mnist_e2e.html)
* [GPT2 Model - pretrained with Azure](https://github.com/SeldonIO/seldon-core/blob/master/docs-gb/examples/triton_gpt2_example_azure.html)
* [GPT2 Model - pretrained with MinIO](https://github.com/SeldonIO/seldon-core/blob/master/docs-gb/examples/triton_gpt2_example.html)


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.seldon.ai/seldon-core-1/configuration/servers/triton.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.