Accessing voice as well as webchat

Rasa Open Source Feedback on Rasa Open Source
1

Hi, I am trying to add voice interface using RASA . I am following this blog Build an AI voice assistant with Rasa Open Source and Mozilla tools | Rasa

I am using botfront to build the flow. Now to deploy on website, the problem happens as socket.io channel is used by webchat. I get following error

sanic.router.RouteExists: Route already registered: /socket.io/ [GET,POST,OPTIONS]

My query is, Is there a way to support both voice and chat when we embed to website. ? i.e a Mic option along with text option in the chatbot . ?

Thanks and Regards,

Vinutha

2

@Vinutha_Nayak Heya! Your bot is developed based on chat or voice? If both as you able to run on localhost successfully?

3

I am trying to add voice capability along with text option . But as both use “socket.io” channel I am getting error. Hence I want to confirm if that is how it is supposed to be. ie. Either it has to be a voicebot (as mentioned in the link I mentioned ) or the regular chatbot

4

You can add a type signal in the socket msg.

like: b"voice:xxxx"

b"text:xxxxxx"

5

Oh ok.Could you please elaborate a little more on how do we do that?

6

Also I am trying to embed into website , so I would like to know how your solution will work for :

  1. At website end for user to input voice input .
  2. At backend how we will process the input, which channel I would need to listen

Currently I was trying to use connectors .

7

You can changed the message that front send:

Change the message structure.

If the message include a pair like {type: text}

You can used it by(Refer to the https://blog.rasa.com/how-to-build-a-voice-assistant-with-open-source-rasa-and-mozilla-tools/ ):

        @sio.on('user_uttered', namespace=self.namespace)
        async def handle_message(sid, data):

            output_channel = SocketIOOutput(sio, sid, self.bot_message_evt, data['message'])
            if data['message'] == "/get_started":
                message = data['message']
            elif data['type'] == "text":
                 message = data['message']
            else:
                ##receive audio
                received_file = 'output_'+sid+'.wav'

                urllib.request.urlretrieve(data['message'], received_file)
                path = os.path.dirname(__file__)

                fs, audio = wav.read("output_{0}.wav".format(sid))
8

Thanks for your quick response.

I am using botfront, so that is using the socket.io interface. So following the same steps as mentioned in the Rasa blog for implementing gives the issue route already registered message .

Could you please provide thoughts on how we could avoid that ??