> For the complete documentation index, see [llms.txt](https://docs.seldon.ai/seldon-enterprise-platform/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.seldon.ai/seldon-enterprise-platform/production-environment/ingress-controller/istio.md).

# Istio

Learn about installing Istio ingress controller in a Kubernetes cluster running Seldon Enterprise Platform.

Istio implements the Kubernetes ingress resource to expose a service and make it accessible from outside the cluster. You can install Istio in either a self-hosted Kubernetes cluster or a managed Kubernetes service provided by a cloud provider that is running the Seldon Enterprise Platform.

## Prerequisites

* Install[ Seldon Enterprise Platform](/seldon-enterprise-platform/production-environment/seldon-enterprise-platform.md#installing-seldon-enterprise-platform).
* Ensure that you install a version of Istio that is compatible with your Kubernetes cluster version. For detailed information on supported versions, refer to the [Istio Compatibility Matrix](https://istio.io/latest/docs/releases/supported-releases/#support-status-of-istio-releases).

## Installing Istio ingress controller

Installing Istio ingress controller in a Kubernetes cluster running Seldon Enterprise Platform involves these tasks:

1. [Install Istio](#install-istio)
2. [Install Istio Ingress Gateway](#install-istio-ingress-gateway)
3. [Expose Seldon mesh service](#expose-seldon-mesh-service)
4. [Install Seldon Enterprise Platform with Istio ingress controller](#install-seldon-enterprise-platform-with-istio-ingress-controller)

### Install Istio

1. Download the Istio installation package for the version you want to use. In the following command replace `<version>` with the version of Istio that you downloaded:

   ```
   curl -L https://istio.io/downloadIstio | sh -
   cd istio-<version>
   export PATH=$PWD/bin:$PATH
   ```
2. Install the Istio Custom Resource Definitions (CRDs) and Istio components in your cluster using the `istioctl` command line tool:

   ```
   istioctl install --set profile=default -y
   ```
3. Create a namespace where you want to enable Istio automatic sidecar injection. For example in the namespace `istio-system`:

   ```
   kubectl label namespace istio-system istio-injection=enabled
   ```

### Install Istio Ingress Gateway

1. Verify that Istio Ingress Gateway is installed:

   ```
   kubectl get svc istio-ingressgateway -n istio-system
   ```

   This should return details of the Istio Ingress Gateway, including the external IP address.
2. Create a YAML file to specify Gateway resource in the `istio-system` namespace to expose your application. For example, create the `istio-seldon-gateway.yaml` file. Use your preferred text editor to create and save the file with the following content:

   ```
    apiVersion: networking.istio.io/v1alpha3
    kind: Gateway
    metadata:
      name: my-gateway
      namespace: istio-system
    spec:
      selector:
        istio: ingressgateway # Use Istio's default ingress gateway
      servers:
      - port:
          number: 80
          name: http
          protocol: HTTP
        hosts:
        - "*"
   ```
3. Change to the directory that contains `istio-seldon-gateway.yaml` file and apply the configuration:

   ```
   kubectl apply -f istio-seldon-gateway.yaml
   ```

   When the configuration is applied, you should see this:

   ```
   gateway.networking.istio.io/seldon-gateway created
   ```
4. Find the IP address of the Seldon Enterprise Platform instance running with Istio:

   ```
   ISTIO_INGRESS=$(kubectl get svc -n istio-system istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
   ISTIO_INGRESS+=$(kubectl get svc -n istio-system istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')

   echo "Seldon Enterprise Platform: http://$ISTIO_INGRESS/seldon-deploy/"

   ```

   <div data-gb-custom-block data-tag="hint" data-style="info" class="hint hint-info"><p>Make a note of the IP address that is displayed in the output.</p></div>

### Expose Seldon mesh service

It is important to expose `seldon-service` service to enable communication between deployed machine learning models and external clients or services. The Seldon Core 2 inference API is exposed through the `seldon-mesh` service in the `seldon` namespace. If you install Core 2 in multiple namespaces, you need to expose the `seldon-mesh` service in each of namespace.

1. Verify if the `seldon-mesh` service is running for example, in the namespace `seldon`.

   ```bash
   kubectl get svc -n seldon
   ```

   When the services are running you should see something similar to this:

   ```bash
   mlserver-0               ClusterIP      None             <none>          9000/TCP,9500/TCP,9005/TCP                                                                  43m
   seldon-mesh              LoadBalancer   34.118.225.130   34.90.213.15    80:32228/TCP,9003:31265/TCP                                                                 45m
   seldon-pipelinegateway   ClusterIP      None             <none>          9010/TCP,9011/TCP                                                                           45m
   seldon-scheduler         LoadBalancer   34.118.225.138   35.204.34.162   9002:32099/TCP,9004:32100/TCP,9044:30342/TCP,9005:30473/TCP,9055:32732/TCP,9008:32716/TCP   45m
   triton-0                 ClusterIP      None             <none>          9000/TCP,9500/TCP,9005/TCP 
   ```
2. Create a YAML file to create a VirtualService Seldon Core 2 `seldon-mesh`. For example, create the `seldon-mesh-vs.yaml` file. Use your preferred text editor to create and save the file with the following content:

   ```yaml
   apiVersion: networking.istio.io/v1alpha3
   kind: VirtualService
   metadata:
     name: seldon-mesh
     namespace: seldon
   spec:
     gateways:
       - istio-system/seldon-gateway
     hosts:
       - "*"
     http:
       - name: "data-plane-seldon"
         match:
           - authority:
               exact: "seldon.inference.seldon"
         route:
           - destination:
               host: "seldon-mesh.seldon.svc.cluster.local"
               port:
                number: 80
     - name: "control-plane-seldon"
       match:
         - authority:
             exact: "seldon.admin.seldon"
       route:
         - destination:
             host: "seldon-scheduler.seldon.svc.cluster.local"
             port:
               number: 9004
   ```
3. Create a virtual service to expose the `seldon-mesh` service.

   ```
   kubectl apply -f seldon-mesh-vs.yaml
   ```

   When the virtual service is created, you should see this:

   ```
   virtualservice.networking.istio.io/seldon-mesh created
   ```

### Install Seldon Enterprise Platform with Istio ingress controller

1. Update the configurations in the `install-values.yaml` file you created during the Seldon Enterprise installation. Replace `<ip_address>` with the IP address noted during the [Istio Ingress Gateway installation](#install-istio-ingress-gateway) in the following values and save the file:

   ```yaml
   ingressGateway:
     seldonIngressService: "istio-ingressgateway"
     ingressNamespace: "istio-system"

   virtualService:
     create: true
     gateways:
       - istio-system/seldon-gateway

   seldon:
     curlForm: |
       curl -k https://<ip_address>/seldon/{{ .Namespace }}/{{ .ModelName }}/api/v0.1/predictions \<br/>
       &nbsp;&nbsp;-H "{{ .TokenHeader }}: {{ .Token }}" \<br/>
       &nbsp;&nbsp;-H "Content-Type: application/json" \<br/>
       &nbsp;&nbsp;-d '{{ .Payload }}'
     tensorFlowCurlForm: |
       curl -k https://<ip_address>/seldon/{{ .Namespace }}/{{ .ModelName }}/v1/models/:predict \<br/>
       &nbsp;&nbsp;-H "{{ .TokenHeader }}: {{ .Token }}" \<br/>
       &nbsp;&nbsp;-H "Content-Type: application/json" \<br/>
       &nbsp;&nbsp;-d '{{ .Payload }}'

   seldonCoreV2:
     curlForm: |
       curl -k https://<ip_address>/v2/models/{{ .ModelName }}/infer \<br/>
       &nbsp;&nbsp;-H "Host: {{ .Namespace }}.inference.seldon" \<br/>
       &nbsp;&nbsp;-H "Content-Type: application/json" \<br/>
       &nbsp;&nbsp;-H "Seldon-Model: {{ .ModelName }}.pipeline" \<br/>
       &nbsp;&nbsp;-d '{{ .Payload }}'
     enabled: true
     requestForm: '{{ .SeldonProtocol }}://seldon-mesh.{{ .Namespace }}.svc.cluster.local/v2/pipelines/{{
       .ModelName }}/infer'

   ```
2. Change to the directory that contains the `install-values.yaml` file and then upgrade the Seldon Enterprise Platform installation in the namespace `seldon-system`.

   ```
   helm upgrade seldon-enterprise seldon-charts/seldon-deploy --namespace seldon-system  -f install-values.yaml --version 2.4.0 --install
   ```
3. Check the status of the installation seldon-enterprise-seldon-deploy.

   ```
   kubectl rollout status deployment/seldon-enterprise-seldon-deploy -n seldon-system
   ```

   When the installation is complete you should see this:

   ```
   deployment "seldon-enterprise-seldon-deploy" successfully rolled out
   ```
4. Access Seldon Enterprise Platform.

{% tabs %}
{% tab title="Port forwarding" %}

1. Get the Pod that is running Seldon Enterprise Platform in the cluster and save it as `$POD_NAME.`

   ```
   export POD_NAME=$(kubectl get pods --namespace seldon-system -l "app.kubernetes.io/name=seldon-deploy,app.kubernetes.io/instance=seldon-enterprise" -o jsonpath="{.items[0].metadata.name}")
   ```
2. You can use port-forwarding to access your application locally.

   ```
   kubectl port-forward $POD_NAME 8000:8000 --namespace seldon-system
   ```
3. Open your browser and navigate to `http://127.0.0.1:8000/seldon-deploy/` to access Seldon Enterprise Platform.
   {% endtab %}

{% tab title="Static IP address" %}
Replace `<ip_address>` with the IP address noted during the [Istio Ingress Gateway installation](#install-istio-ingress-gateway).\
\
1\. Open your browser and navigate to `http://<ip_address>/seldon-deploy/` to access Seldon Enterprise Platform.
{% endtab %}
{% endtabs %}

#### Optional: Enable HTTPS/TLS

To secure your Ingress with HTTPS, you can configure TLS settings in the `Gateway` resource using a certificate and key. This involves additional steps like creating Kubernetes secrets for your certificates.

#### Additional Resources

* [Istio Documentation](https://istio.io/latest/docs/tasks/traffic-management/ingress/secure-ingress/)
* [GKE Ingress Guide](https://cloud.google.com/kubernetes-engine/docs/concepts/ingress)
* [AWS Documentation](https://docs.aws.amazon.com/eks/latest/userguide/what-is-eks.html)