> For the complete documentation index, see [llms.txt](https://docs.seldon.ai/seldon-enterprise-platform/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.seldon.ai/seldon-enterprise-platform/demos/seldon-core-v2/custom-huggingface-model.md).

# Text Generation with Custom HuggingFace Model

This demo helps you learn about:

* Launching a pre trained custom text generation HuggingFace model in a Seldon Pipeline
* Sending a text input request to get a generated text prediction

The custom HuggingFace text generation model is based on the [TinyStories-1M](https://huggingface.co/roneneldan/TinyStories-1M) model in the HuggingFace hub.

## Create a Seldon ML Pipeline

1. In the **Overview** page click **Create new deployment**.
2. Enter the deployment details as follows:
   * Name: `hf-custom-tiny-stories`
   * Namespace: `seldon`
   * Type: `Seldon ML Pipeline`
3. Configure the default predictor as follows:

   * Runtime: `HuggingFace`
   * Model Project: `default`
   * Model URI: `gs://seldon-models/scv2/samples/mlserver_1.6.0/huggingface-text-gen-custom-tiny-stories`
   * Storage Secret: (leave blank/none)

   ![Default predictor spec](https://1921172648-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FvLlcdnP8YnaIFsv8EiMA%2Fuploads%2Fgit-blob-2f96955e5eea5b4fe9b95fc60f360c98f7e5b940%2Fwizard.png?alt=media)
4. Click **Next** for the remaining steps and click **Launch**.

When the deployment is launched successfully, the status of the deployment becomes `Available`.

## Make Predictions

1. Click the `hf-custom-tiny-stories` deployment that you created.
2. In the deployment **Dashboard** page , click **Predict** in the left pane.
3. In the **Predict REST API** dialog, click **Enter JSON** and paste the following text:

   ```JSON
   {
     "inputs": [{
       "name": "args",
       "shape": [1],
       "datatype": "BYTES",
       "data": ["The brown fox jumped"]
     }]
   }
   ```
4. Click **Predict**.

   ![A screenshot showing the Predict page with the textarea pre-populated](https://1921172648-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FvLlcdnP8YnaIFsv8EiMA%2Fuploads%2Fgit-blob-35b95aa9416530f84400841a6f86199644d31d30%2Fpredict.png?alt=media)

## Next steps

Try other [demos](https://github.com/SeldonIO/testing-gitbook/blob/v2.4/demos/README.md) or try a larger-scale model. You can find one in `gs://seldon-models/scv2/samples/mlserver_1.6.0/huggingface-text-gen-custom-gpt2`. However, you may need to request more memory.