# API

## LLMSettings

LLM base settings

### **provider\_id: `Literal["openai", "gemini"]`**

**Description:** Third-party provider id.

### **config: `Union[OpenAISettings, GeminiSettings]`**

**Description:** Third-party provider settings.

### **prompt\_utils: `Optional[PromptUtilsSettings]`**

**Description:** Prompt utils settings. **Default:** `None`

## OpenAISettings

OpenAI settings

### **api\_type: `Optional[Literal["open_ai", "azure", "azure_ad"]]`**

**Description:** OpenAI API type. Available options are: `"open_ai"` | `"azure"` | `"azure_ad"`. If `None` would default to `"open_ai"`.\
**Default:** `None`

### **azure\_endpoint: `Optional[str]`**

**Description:** Azure endpoint, including the resource. See also `base_url`.\
**Default:** `None`

### **azure\_deployment: `Optional[str]`**

**Description:** Azure deployment. A model deployment, if given sets the base client URL to include `/deployments/{azure_deployment}`. Note: this means you won't be able to use non-deployment endpoints. Not supported with Assistants APIs. See also `base_url`.\
**Default:** `None`

### **api\_version: `Optional[str]`**

**Description:** API version. Only needed for Azure API.\
**Default:** `None`

### **azure\_ad\_token: `Optional[str]`**

**Description:** Azure Active Directory token.\
**Default:** `None`

### **model\_id: `Optional[str]`**

**Description:** OpenAI model id. In azure this should be the deployment\_name.\
**Default:** `None`

### model\_type: `Literal["completions", "chat.completions", "embeddings", "images.generations"]`

**Description:** OpenAI model type. Available options are: `"chat.completions"` | `"completions"` | `"embeddings"` | `"images.generations"`."""\
**Default:** `"chat.completions"`

### api\_key: `Optional[str]`

**Description:** OpenAI API key.\
**Default:** `None`

### organization: `Optional[str]`

**Description:** OpenAI organization.\
**Default:** `None`

### project: `Optional[str]`

**Description:** OpenAI project.\
**Default:** `None`

### base\_url: `Optional[str]`

**Description:** OpenAI base url. Note: When using Azure, `base_url` is set to either `"{azure_endpoint}/openai/deployments/{azure_deployment}"` or `"{azure_endpoint}/openai"`. Thus, `base_url` and `azure_endpoint` are mutually exclusive.\
**Default:** `None`

### llm\_parameters: `Optional[dict]`

**Description:** OpenAI LLM parameters. This is a dictionary of parameters to be passed to the OpenAI LLM API.\
**Default:** `None`

## GeminiSettings

Gemini API settings

### model\_id: `Optional[str]`

**Description:** Gemini model id.\
**Default:** `None`

### model\_type: \`Literal\["completions", "chat.completions"]

**Description:** Gemini model type. Available options are: `"chat.completions"` | `"completions"`\
**Default:** `"chat.completions"`

### api\_key: `Optional[str]`

**Description:** Gemini API key.\
**Default:** `None`

### llm\_parameters: `Optional[GeminiGenerateContentParameters]`

**Description:** Gemini LLM parameters. This is a dictionary of parameters to be passed to the Gemini LLM API.\
**Default:** `None`


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.seldon.ai/llm-module/resources/reference/api.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.