Huggingface speech to sentiment with explanations pipeline
In this example we create a Pipeline to chain two huggingface models to allow speech to sentiment functionalityand add an explainer to understand the result.
This example also illustrates how explainers can target pipelines to allow complex explanations flows.
This example requires ffmpeg package to be installed locally. run make install-requirements
for the Python dependencies.
Create a method to load speech from recorder; transform into mp3 and send at base64 data. On return of the result extract and show the text and sentiment.
Load Huggingface Models
We will load two Huggingface models for speech to text and text to sentiment.
Create Explain Pipeline
To allow Alibi-Explain to more easily explain the sentiment we will need:
input and output transfrorms that take the Dict values input and output by the Huggingface sentiment model and turn them into values that Alibi-Explain can easily understand with the core values we want to explain and the outputs from the sentiment model.
A separate Pipeline to allow us to join the sentiment model with the output transform
These transform models are MLServer custom runtimes as shown below:
Speech to Sentiment Pipeline with Explanation
We can now create the final pipeline that will take speech and generate sentiment alongwith an explanation of why that sentiment was predicted.
Test
We will wait for the explanation which is run asynchronously to the functional output from the Pipeline above.
Cleanup
Last updated
Was this helpful?