Voice assistance

[gauravkodmalwar]

hey all,

Does Rasa support voice to text conversion or does it support integration with such API? I am new to this, I did a small chatbot project using rasa but now wanted to get rid of chat and use only voice for the conversation. Is it possible?

[hajoura (hajar)]

yes it is possible to apply voice in rasa with google I think that help

Going beyond ‘Hey Google’: building a Rasa-powered Google Assistant

[Juste]

Hi @gauravkodmalwar. Yes, Hajar shared a great link Rasa doesn’t come with in-built speech-to-text and text-to-speech components, but you can integrate it with existing ones just like Google Assistant

[jamesmf]

The fact that rasa is so easy to deploy locally is a huge selling point, so it’d be very cool to see an integration with something like Mycroft or another locally-hostable voice assistant.

[jamesmf]

I got the basics of a Mycroft skill that sends utterances to an arbitrary rasa REST endpoint, then speaks the text of the resulting messages.

I hooked it up to Sara the demo-bot, but I found there were a number of buttons in Sara's dialogues. I’m wondering if there’s guidance around how to most smoothly handle buttons with voice.

Should one:

• Avoid buttons if the bot might be used through voice
• Expect the voice assistant “skill” to function as an adapter that can match an utterance to one of the button options, then return the payload to the endpoint?
• Expect the voice assistant “skill” read the buttons’ title values, but train your Rasa agent + nlu to handle whatever the user might respond, even if it might not map to a button value?

[jamesmf]

This Mycroft skill should let you talk to your rasa bot without the need for Google’s STT/TTS.

• clone the Mycroft repo and run the setup [have had trouble with their docker image recently]
• install the skill
• start rasa on 5005
• run mycroft-start.sh all
• say “chat with rasa”

And you’ll be talking to your rasa bot via voice --> STT --> rasa http api --> TTS! The STT will by default by sent to Mycroft’s STT service, but you can host your own personal Mycroft backend to avoid that and have everything running locally

[syntithenai]

@jamesmf

Some alternatives for voice integration

• GitHub - RasaHQ/rasa-voice-interface: A simple web interface for building voice assistants with Rasa - web browser direct to Deepspeech/RASA without intermediate service.
• my pet project GitHub - syntithenai/hermod: voice services stack from audio hardware through hotword, ASR, NLU, AI routing and TTS bound by messaging protocol over MQTT using a central MQTT message bus between a suite of microservices for ASR/hotword/NLU/TTS/RASA routing. Suitable for web services and standalone mycroft/alexa like devices.

[jamesmf]

Very cool!