# Making a Voice Bot with Rasa, Text-to-Speech & ASR

**URL:** <https://forum.rasa.com/t/making-a-voice-bot-with-rasa-text-to-speech-asr/58505>\
**Category:** Rasa Open Source\
**Created:** [June 3, 2023, 3:26pm UTC](https://forum.rasa.com/t/making-a-voice-bot-with-rasa-text-to-speech-asr/58505 "2023-06-03T15:26:55Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![KaisCommitted](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/kaiscommitted/32/22802_2.png) [@KaisCommitted](https://forum.rasa.com/u/KaisCommitted)\
**Post date:** [June 3, 2023, 3:26pm UTC](https://forum.rasa.com/t/making-a-voice-bot-with-rasa-text-to-speech-asr/58505/1 "2023-06-03T15:26:55Z")

</div>

Hello everyone, I’ve been working with rasa for a month now, I have implmented my data and trained the models and I’m satisfied with what I got. The next step in my project is to integrate rasa with Speech-To-Text and Text-to-Speech systems in order to make a fully working **voice bot**. Later on, I will use use the voice bot as an interviewer bot by integrating it on google meet.

What I have tried so far is to follow this [guide](https://rasa.com/blog/how-to-build-a-voice-assistant-with-open-source-rasa-and-mozilla-tools/). I had a lot of versioning issues setting up the environment and then I kept having errors and I figured that it’s not gonna work especially after I checked the GitHub issues section of rasa-voice-interface (lots of issues posted, none of them are resolved).

Right now I’m really lost on how am I gonna make this voice bot actually work and I would really appreciate some guidance if anyone here has worked on something similar.

Text-to-Speech tested & implemented: pytts3

Speech-to-Text (ASR) tested & implemented: Whisper

Here’s a showcase of something really similar for what I would like to achieve: [https://rasa.community/showcase/josh-converse/](https://rasa.community/showcase/josh-converse/)

---

<div class="post-metadata">

**Author:** ![stephens](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/stephens/32/1632_2.png) [@stephens](https://forum.rasa.com/u/stephens)\
**Post date:** [June 3, 2023, 6:23pm UTC](https://forum.rasa.com/t/making-a-voice-bot-with-rasa-text-to-speech-asr/58505/2 "2023-06-03T18:23:14Z")

</div>

Do you want your bot to be available via the phone? If so, I would try the [Twilio Voice connector](https://rasa.com/docs/rasa/connectors/twilio-voice/).

---

<div class="post-metadata">

**Author:** ![KaisCommitted](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/kaiscommitted/32/22802_2.png) [@KaisCommitted](https://forum.rasa.com/u/KaisCommitted)\
**Post date:** [June 4, 2023, 11:53am UTC](https://forum.rasa.com/t/making-a-voice-bot-with-rasa-text-to-speech-asr/58505/3 "2023-06-04T11:53:26Z")

</div>

Not really, I will link the bot with google meet later on.

As I mentioned, what I’m struggling with at this step is linking the Text-to-speech model (pytts3) and the Speech-to-Text tool (whisper) to rasa.

---

<div class="post-metadata">

**Author:** ![stephens](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/stephens/32/1632_2.png) [@stephens](https://forum.rasa.com/u/stephens)\
**Post date:** [June 4, 2023, 1:39pm UTC](https://forum.rasa.com/t/making-a-voice-bot-with-rasa-text-to-speech-asr/58505/4 "2023-06-04T13:39:50Z")

</div>

Take a look at this [post](https://forum.rasa.com/t/text-and-voice-chatbot-in-same-vue/29886/3)

---

<div class="post-metadata">

**Author:** ![annianni](https://avatars.discourse-cdn.com/v4/letter/a/87869e/32.png) [@annianni](https://forum.rasa.com/u/annianni)\
**Post date:** [September 13, 2023, 3:16pm UTC](https://forum.rasa.com/t/making-a-voice-bot-with-rasa-text-to-speech-asr/58505/5 "2023-09-13T15:16:11Z")

</div>

how many languages does this TTS support? The current [text to speech online](https://wavel.ai/solutions/realistic-text-to-speech-online/) AI that I’m using supports over 40 global languages so I needed a fair idea. Also, how fast and accurate are the results? (a rough time frame as compared to other similar software if mentioned will be okay, since I need it to process large amount of data)

---

<div class="post-metadata">

**Author:** ![KaisCommitted](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/kaiscommitted/32/22802_2.png) [@KaisCommitted](https://forum.rasa.com/u/KaisCommitted)\
**Post date:** [September 13, 2023, 3:31pm UTC](https://forum.rasa.com/t/making-a-voice-bot-with-rasa-text-to-speech-asr/58505/6 "2023-09-13T15:31:17Z")

</div>

I’m not sure exactly how many languages it supports but for the French language, which is the one I need, it was the best that’s why I picked it. I remember it having a lot of languages including English, Spanish, French. Results are very fast and very accurate compared to other tools I tried, I conducted a whole benchmarking with around 10 tools but as I said it was **only for French** models. It also offers different models for every language going from tiny ( decent accuracy, very fast) to large(really good accuracy, x5-10 slower than tiny)

> **[Introducing Whisper](https://openai.com/research/whisper)**
>
> We’ve trained and are open-sourcing a neural net called Whisper that approaches human level robustness and accuracy on English speech recognition.
