Hello everyone, I’ve been working with rasa for a month now, I have implmented my data and trained the models and I’m satisfied with what I got. The next step in my project is to integrate rasa with Speech-To-Text and Text-to-Speech systems in order to make a fully working voice bot. Later on, I will use use the voice bot as an interviewer bot by integrating it on google meet. What I have tried so far is to follow this guide . I had a lot of versioning issues setting up the environment and then I kept having errors and I figured that it’s not gonna work especially after I checked the GitHub issues section of rasa-voice-interface (lots of issues posted, none of them are resolved). Right now I’m really lost on how am I gonna make this voice bot actually work and I would really appreciate some guidance if anyone here has worked on something similar. Text-to-Speech tested & implemented: pytts3 Speech-to-Text (ASR) tested & implemented: Whisper Here’s a showcase of something really similar for what I would like to achieve: https://rasa.community/showcase/josh-converse/

Not really, I will link the bot with google meet later on. As I mentioned, what I’m struggling with at this step is linking the Text-to-speech model (pytts3) and the Speech-to-Text tool (whisper) to rasa.

how many languages does this TTS support? The current text to speech online AI that I’m using supports over 40 global languages so I needed a fair idea. Also, how fast and accurate are the results? (a rough time frame as compared to other similar software if mentioned will be okay, since I need it to process large amount of data)

I’m not sure exactly how many languages it supports but for the French language, which is the one I need, it was the best that’s why I picked it. I remember it having a lot of languages including English, Spanish, French. Results are very fast and very accurate compared to other tools I tried, I conducted a whole benchmarking with around 10 tools but as I said it was only for French models. It also offers different models for every language going from tiny ( decent accuracy, very fast) to large(really good accuracy, x5-10 slower than tiny) [image] Introducing Whisper We’ve trained and are open-sourcing a neural net called Whisper that approaches human level robustness and accuracy on English speech recognition.

Making a Voice Bot with Rasa, Text-to-Speech & ASR

Rasa Open Source

stephens (Greg Stephens) June 3, 2023, 6:23pm 2

Do you want your bot to be available via the phone? If so, I would try the Twilio Voice connector.

Topic		Replies	Views
Voice integration in Rasa Feedback on Rasa Open Source	1	381	August 8, 2021
Voice assistance Rasa Open Source	7	3328	May 31, 2020
Speech to text and text to speech integration Feedback on Rasa Open Source	1	1219	August 2, 2021
Voice recognition in Rasa Feedback on Rasa Open Source	2	434	July 13, 2021
How can I implement text to speech and speech to text recognition in my rasa chatbot Rasa Open Source	1	3837	May 6, 2020

Making a Voice Bot with Rasa, Text-to-Speech & ASR

Related topics