Edit

Ambassador

Learn how to integrate Seldon Core 2 with Ambassador for service mesh and ingress management, including REST and gRPC endpoint configuration.

Ambassador provides service mesh and ingress products. Our examples here are based on the Emissary ingress.

We will run through some examples as shown in the notebook service-meshes/ambassador/ambassador.ipynb in our repo.

Single Model

  • Seldon Iris classifier model

  • Default Ambassador Host and Listener

  • Ambassador Mappings for REST and gRPC endpoints

# service-meshes/ambassador/static/single-model.yaml
apiVersion: getambassador.io/v3alpha1
kind: Host
metadata:
  name: wildcard
  namespace: seldon-mesh
spec:
  hostname: '*'
  requestPolicy:
    insecure:
      action: Route
---
apiVersion: getambassador.io/v3alpha1
kind: Listener
metadata:
  name: emissary-ingress-listener-8080
  namespace: seldon-mesh
spec:
  hostBinding:
    namespace:
      from: ALL
  port: 8080
  protocol: HTTP
  securityModel: INSECURE
---
apiVersion: getambassador.io/v3alpha1
kind: Mapping
metadata:
  name: iris-grpc
  namespace: seldon-mesh
spec:
  add_request_headers:
    seldon-model:
      value: iris
  grpc: true
  hostname: '*'
  prefix: /inference.GRPCInferenceService
  rewrite: ""
  service: seldon-mesh:80
---
apiVersion: getambassador.io/v3alpha1
kind: Mapping
metadata:
  name: iris-http
  namespace: seldon-mesh
spec:
  add_request_headers:
    seldon-model:
      value: iris
  hostname: '*'
  prefix: /v2/
  rewrite: ""
  service: seldon-mesh:80
---
apiVersion: mlops.seldon.io/v1alpha1
kind: Model
metadata:
  name: iris
  namespace: seldon-mesh
spec:
  requirements:
  - sklearn
  storageUri: gs://seldon-models/mlserver/iris

Traffic Split

Note: Traffic splitting does not presently work due to this issue. We recommend you use a Seldon Experiment instead.

Seldon provides an Experiment resource for service mesh agnostic traffic splitting but if you wish to control this via Ambassador and example is shown below to split traffic between two models.

Ambassador Notebook Example

Assumes

  • You have installed emissary as per their docs

Tested with

emissary-ingress-7.3.2 insatlled via helm

Ambassador Single Model Example

Traffic Split Two Models

Currently not working due to this issue

PreviousService MeshesNextIstio

Last updated

Was this helpful?