Connect the browser_audio to a custom frontend

Rasa Pro CALM
1

Hi everyone!

I’ve successfully built a voice bot using Rasa, and it works great when I test it with the rasa inspect --voice command.

Now, I’m looking to integrate this voice functionality into my own web frontend.

For text-based interactions, I’m currently using the rasa-chatbot-widget, which works perfectly.

Could anyone point me in the right direction or offer advice on how to get the voice version working on a separate page in my frontend? What are the common methods or recommended libraries for handling audio input from the user’s microphone, sending it to Rasa, and playing back the voice responses?

for now this is what my credentials looks like on the browser_audio part.

browser_audio: server_url: 0.0.0.0 asr: name: deepgram tts: name: cartesia

Thanks in advance!"

2

Hey, you can try using audiostream.ts with your custom frontend.

createAudioConnection is the function that you would be using to start streaming.

<Button
  onClick={async () =>
    await createAudioConnection(baseUrl)
  }
>Go</Button>

Note that baseUrl is parsed by the function getWebSocketUrl so you need to ensure that it points to wherever Rasa is deployed. Hope this helps!