Voice assistance

hey all,

Does Rasa support voice to text conversion or does it support integration with such API? I am new to this, I did a small chatbot project using rasa but now wanted to get rid of chat and use only voice for the conversation. Is it possible?

yes it is possible to apply voice in rasa with google I think that help

Hi @gauravkodmalwar. Yes, Hajar shared a great link :slight_smile: Rasa doesn’t come with in-built speech-to-text and text-to-speech components, but you can integrate it with existing ones just like Google Assistant :slight_smile:

1 Like

The fact that rasa is so easy to deploy locally is a huge selling point, so it’d be very cool to see an integration with something like Mycroft or another locally-hostable voice assistant.

I got the basics of a Mycroft skill that sends utterances to an arbitrary rasa REST endpoint, then speaks the text of the resulting messages.

I hooked it up to Sara the demo-bot, but I found there were a number of buttons in Sara's dialogues. I’m wondering if there’s guidance around how to most smoothly handle buttons with voice.

Should one:

  • Avoid buttons if the bot might be used through voice
  • Expect the voice assistant “skill” to function as an adapter that can match an utterance to one of the button options, then return the payload to the endpoint?
  • Expect the voice assistant “skill” read the buttons’ title values, but train your Rasa agent + nlu to handle whatever the user might respond, even if it might not map to a button value?

This Mycroft skill should let you talk to your rasa bot without the need for Google’s STT/TTS.

  • clone the Mycroft repo and run the setup [have had trouble with their docker image recently]
  • install the skill
  • start rasa on 5005
  • run mycroft-start.sh all
  • say “chat with rasa”

And you’ll be talking to your rasa bot via voice --> STT --> rasa http api --> TTS! The STT will by default by sent to Mycroft’s STT service, but you can host your own personal Mycroft backend to avoid that and have everything running locally



Some alternatives for voice integration


Very cool!