Voice IVR on Rasa Chatbot and Asterisk VoIP Platform

Rasa Open Source
1

Hello Rasa Community,

We have successfully developed and deployed a voice menu with speech recognition in our office, aimed at streamlining and enhancing caller interactions.

How It Works: Our solution is built upon three key technologies:

Rasa – for powerful dialogue management and intent recognition. Deepgram – for highly accurate real-time speech recognition. Asterisk (VoIP) – as the backbone for call routing and telephony management.

Rasa as the Brain of Call Routing: Rasa acts as the brain of the operation. It not only understands what the caller is asking for but also determines the best route based on intent classification and entity extraction.

  1. Deepgram captures and transcribes the caller’s speech in real-time.
  2. Rasa processes this transcription, identifies the intent and entities, and generates a structured response.
  3. Asterisk receives the response and executes the correct routing path via a custom Dialplan application that we developed specifically for seamless integration.

Asterisk & Custom Dialplan Application: We chose Asterisk VoIP as our telephony platform because of its open-source flexibility and powerful call handling capabilities.

We built a custom application for Asterisk’s Dialplan, which allows for smooth integration with Rasa and Deepgram.

This architecture significantly simplifies the deployment and configuration of the voice menu.

It enables real-time routing to the right department with minimal latency:

  • Technical Support
  • Commercial Department
  • Sales Department

This streamlined approach allows us to avoid complex IVR trees, enabling callers to simply state their purpose in free form and be routed instantly.

Language Support: Currently, the voice menu is fully operational in Russian. We are exploring the possibility of expanding to English and other languages if there is enough demand or interesting collaboration opportunities.

Looking for Collaboration: We are reaching out to the Rasa community to:

Get feedback on best practices for multilingual voice menus. Find potential partners who are interested in testing the system in English-speaking environments.

  • Share experience with those building similar voice-driven Rasa applications.
  • Explore integration ideas with other open-source platforms.

Would you be interested in collaborating with us to enhance this solution for English or other languages? We are open to ideas, partnerships, and testing opportunities.

Feel free to drop a comment or reach out directly. Let’s build smarter voice interactions together.

2 Likes
2

Hey @yarikrad , that sounds excellent. I’m from Rasa’s Engineering Team and we have been working on the Voice team to build first class voice support with CALM and more built-in integrations. Would you like to talk about this solution and the integrations you have built?

3

Hey @izi. We built a custom Asterisk application that connects the live audio stream to our own proxy service (written in Go).

This proxy handles several key responsibilities:

  1. Detecting end-of-utterance** in the caller’s speech
  2. Streaming audio to an external STT service to convert speech to text
  3. Sending the transcribed text to Rasafor intent recognition and dialogue handling
  4. Converting Rasa’s response back to speech via an external TTS engine
  5. Streaming the generated audio back to Asterisk for real-time playback to the caller

This approach allowed us to keep the voice logic modular and flexible, and to fine-tune timing and latency across the system. Let me know how you’d prefer to connect — happy to chat here, over a call, or contribute feedback to the voice initiative! What aspects of the implementation would you be most interested in.

1 Like
4

Let’s do this over a call. Can you please send me an email at s.paliwal@rasa.com and we’ll take it from there. I am pretty sure my colleagues will be interested and we can also explore partnerships. You can also directly stream the audio to Rasa using the new voice channels. I will also be glad to showcase the current support for voice primitives, experiments with multilingual voicebots.