> For the complete documentation index, see [llms.txt](https://docs.seldon.ai/seldon-enterprise-platform/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.seldon.ai/seldon-enterprise-platform/production-environment/ingress-controller/istio.md). # Istio Istio implements the Kubernetes ingress resource to expose a service and make it accessible from outside the cluster. You can install Istio in either a self-hosted Kubernetes cluster or a managed Kubernetes service provided by a cloud provider that is running the Seldon Enterprise Platform. ## Prerequisites * Install[ Seldon Enterprise Platform](/seldon-enterprise-platform/production-environment/seldon-enterprise-platform.md#installing-seldon-enterprise-platform). * Ensure that you install a version of Istio that is compatible with your Kubernetes cluster version. For detailed information on supported versions, refer to the [Istio Compatibility Matrix](https://istio.io/latest/docs/releases/supported-releases/#support-status-of-istio-releases). ## Installing Istio ingress controller Installing Istio ingress controller in a Kubernetes cluster running Seldon Enterprise Platform involves these tasks: 1. [Install Istio](#install-istio) 2. [Install Istio Ingress Gateway](#install-istio-ingress-gateway) 3. [Expose Seldon mesh service](#expose-seldon-mesh-service) 4. [Install Seldon Enterprise Platform with Istio ingress controller](#install-seldon-enterprise-platform-with-istio-ingress-controller) ### Install Istio 1. Download the Istio installation package for the version you want to use. In the following command replace `` with the version of Istio that you downloaded: ``` curl -L https://istio.io/downloadIstio | sh - cd istio- export PATH=$PWD/bin:$PATH ``` 2. Install the Istio Custom Resource Definitions (CRDs) and Istio components in your cluster using the `istioctl` command line tool: ``` istioctl install --set profile=default -y ``` 3. Create a namespace where you want to enable Istio automatic sidecar injection. For example in the namespace `istio-system`: ``` kubectl label namespace istio-system istio-injection=enabled ``` ### Install Istio Ingress Gateway 1. Verify that Istio Ingress Gateway is installed: ``` kubectl get svc istio-ingressgateway -n istio-system ``` This should return details of the Istio Ingress Gateway, including the external IP address. 2. Create a YAML file to specify Gateway resource in the `istio-system` namespace to expose your application. For example, create the `istio-seldon-gateway.yaml` file. Use your preferred text editor to create and save the file with the following content: ``` apiVersion: networking.istio.io/v1alpha3 kind: Gateway metadata: name: my-gateway namespace: istio-system spec: selector: istio: ingressgateway # Use Istio's default ingress gateway servers: - port: number: 80 name: http protocol: HTTP hosts: - "*" ``` 3. Change to the directory that contains `istio-seldon-gateway.yaml` file and apply the configuration: ``` kubectl apply -f istio-seldon-gateway.yaml ``` When the configuration is applied, you should see this: ``` gateway.networking.istio.io/seldon-gateway created ``` 4. Find the IP address of the Seldon Enterprise Platform instance running with Istio: ``` ISTIO_INGRESS=$(kubectl get svc -n istio-system istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}') ISTIO_INGRESS+=$(kubectl get svc -n istio-system istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].hostname}') echo "Seldon Enterprise Platform: http://$ISTIO_INGRESS/seldon-deploy/" ```

Make a note of the IP address that is displayed in the output.

### Expose Seldon mesh service It is important to expose `seldon-service` service to enable communication between deployed machine learning models and external clients or services. The Seldon Core 2 inference API is exposed through the `seldon-mesh` service in the `seldon` namespace. If you install Core 2 in multiple namespaces, you need to expose the `seldon-mesh` service in each of namespace. 1. Verify if the `seldon-mesh` service is running for example, in the namespace `seldon`. ```bash kubectl get svc -n seldon ``` When the services are running you should see something similar to this: ```bash mlserver-0 ClusterIP None 9000/TCP,9500/TCP,9005/TCP 43m seldon-mesh LoadBalancer 34.118.225.130 34.90.213.15 80:32228/TCP,9003:31265/TCP 45m seldon-pipelinegateway ClusterIP None 9010/TCP,9011/TCP 45m seldon-scheduler LoadBalancer 34.118.225.138 35.204.34.162 9002:32099/TCP,9004:32100/TCP,9044:30342/TCP,9005:30473/TCP,9055:32732/TCP,9008:32716/TCP 45m triton-0 ClusterIP None 9000/TCP,9500/TCP,9005/TCP ``` 2. Create a YAML file to create a VirtualService Seldon Core 2 `seldon-mesh`. For example, create the `seldon-mesh-vs.yaml` file. Use your preferred text editor to create and save the file with the following content: ```yaml apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: seldon-mesh namespace: seldon spec: gateways: - istio-system/seldon-gateway hosts: - "*" http: - name: "data-plane-seldon" match: - authority: exact: "seldon.inference.seldon" route: - destination: host: "seldon-mesh.seldon.svc.cluster.local" port: number: 80 - name: "control-plane-seldon" match: - authority: exact: "seldon.admin.seldon" route: - destination: host: "seldon-scheduler.seldon.svc.cluster.local" port: number: 9004 ``` 3. Create a virtual service to expose the `seldon-mesh` service. ``` kubectl apply -f seldon-mesh-vs.yaml ``` When the virtual service is created, you should see this: ``` virtualservice.networking.istio.io/seldon-mesh created ``` ### Install Seldon Enterprise Platform with Istio ingress controller 1. Update the configurations in the `install-values.yaml` file you created during the Seldon Enterprise installation. Replace `` with the IP address noted during the [Istio Ingress Gateway installation](#install-istio-ingress-gateway) in the following values and save the file: ```yaml ingressGateway: seldonIngressService: "istio-ingressgateway" ingressNamespace: "istio-system" virtualService: create: true gateways: - istio-system/seldon-gateway seldon: curlForm: | curl -k https:///seldon/{{ .Namespace }}/{{ .ModelName }}/api/v0.1/predictions \
  -H "{{ .TokenHeader }}: {{ .Token }}" \
  -H "Content-Type: application/json" \
  -d '{{ .Payload }}' tensorFlowCurlForm: | curl -k https:///seldon/{{ .Namespace }}/{{ .ModelName }}/v1/models/:predict \
  -H "{{ .TokenHeader }}: {{ .Token }}" \
  -H "Content-Type: application/json" \
  -d '{{ .Payload }}' seldonCoreV2: curlForm: | curl -k https:///v2/models/{{ .ModelName }}/infer \
  -H "Host: {{ .Namespace }}.inference.seldon" \
  -H "Content-Type: application/json" \
  -H "Seldon-Model: {{ .ModelName }}.pipeline" \
  -d '{{ .Payload }}' enabled: true requestForm: '{{ .SeldonProtocol }}://seldon-mesh.{{ .Namespace }}.svc.cluster.local/v2/pipelines/{{ .ModelName }}/infer' ``` 2. Change to the directory that contains the `install-values.yaml` file and then upgrade the Seldon Enterprise Platform installation in the namespace `seldon-system`. ``` helm upgrade seldon-enterprise seldon-charts/seldon-deploy --namespace seldon-system -f install-values.yaml --version 2.4.0 --install ``` 3. Check the status of the installation seldon-enterprise-seldon-deploy. ``` kubectl rollout status deployment/seldon-enterprise-seldon-deploy -n seldon-system ``` When the installation is complete you should see this: ``` deployment "seldon-enterprise-seldon-deploy" successfully rolled out ``` 4. Access Seldon Enterprise Platform. {% tabs %} {% tab title="Port forwarding" %} 1. Get the Pod that is running Seldon Enterprise Platform in the cluster and save it as `$POD_NAME.` ``` export POD_NAME=$(kubectl get pods --namespace seldon-system -l "app.kubernetes.io/name=seldon-deploy,app.kubernetes.io/instance=seldon-enterprise" -o jsonpath="{.items[0].metadata.name}") ``` 2. You can use port-forwarding to access your application locally. ``` kubectl port-forward $POD_NAME 8000:8000 --namespace seldon-system ``` 3. Open your browser and navigate to `http://127.0.0.1:8000/seldon-deploy/` to access Seldon Enterprise Platform. {% endtab %} {% tab title="Static IP address" %} Replace `` with the IP address noted during the [Istio Ingress Gateway installation](#install-istio-ingress-gateway).\ \ 1\. Open your browser and navigate to `http:///seldon-deploy/` to access Seldon Enterprise Platform. {% endtab %} {% endtabs %} #### Optional: Enable HTTPS/TLS To secure your Ingress with HTTPS, you can configure TLS settings in the `Gateway` resource using a certificate and key. This involves additional steps like creating Kubernetes secrets for your certificates. #### Additional Resources * [Istio Documentation](https://istio.io/latest/docs/tasks/traffic-management/ingress/secure-ingress/) * [GKE Ingress Guide](https://cloud.google.com/kubernetes-engine/docs/concepts/ingress) * [AWS Documentation](https://docs.aws.amazon.com/eks/latest/userguide/what-is-eks.html) --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.seldon.ai/seldon-enterprise-platform/production-environment/ingress-controller/istio.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.