Playing audio clips as RASA response

Rasa Open Source
1

Hello Everyone,

I want to play an audio file as a response to rasa chatbot used in an audio chatbot connected to a phone bot. Is it possible to autoplay an audio or video file. I know how to play a video link in rasa chatbot using embed component but it is not autoplaying.

2

Hmm, that sounds like something that would vary quite a bit by front end. The SocketIOutput method in the custom connector from this blog post on using Mozilla DeepSpeech & text to speech might be a good starting point.

3

Hello @rctatman,

Thank You for your response. i Have already seen that and implemented it. For AI voice clips i have thousands of recording to play the voice note. My issue is mentioned here .

On my windows system i used this to make utterance worked for autoplay :

  utter_recording:
  - custom:
    attachment:
      type: video
      payload:
        src: fullpath/filename.mp4
      autoplay: true

It worked perfectly fine. But when i tried doing the same on GCP like this :

  utter_recording:
  - custom:
    attachment:
      type: video
      payload:
        src: /etc/rasa/files/video.mp4
      autoplay: true

I get this error in webchat widget :

image
image478Ă—516 12.8 KB

Any luck finding solution either autoplay videos via link or local files on GCP will be very helpful.

4

Hm, gotcha. Since it’s working locally it looks like it might be a problem with the file path, or possibly a permissions thing? I might ping someone at GCP and see if they can’t provide some more insight.

5

That will be great.

6

Hello @rctatman. Any updates regarding my Question?

7

Ah, sorry, I realize what I said was a bit ambiguous: I’d recommend that you ask someone from GCP about this. If you tag a StackOverflow question they’re usually pretty responsive.

1 Like
8

Hi @rctatman ,please I’m looking for how can rasa understand a voice-message sent with Facebook or telegram?

9

Hello @Fares ,

Are you looking for Voice response from the bot or the user send voice response to RASA bot.

Incase you are looking for the later one i.e sending vouce message from facebook or telegram to the RASA bot, you need to take it as a payload and send it to some Speech to text(STT) api (for ex google or ibm) and then feed the response of STT api to rasa Bot nlu to get a response from RASA.

10

Thanks for your response Yes, I’m looking for the user send voice response to RASA bot. In fact how can 'I send the voice message from facebook to STT API? and then get the Text from these API to send it to RASA BOT? i’m really so confused :pensive:

11

I think you need to write a custom action to create text to be passed to rasa and then use the converted text.

Here this link will help you on how to send speech file to google to get converted into text.

13

Custom json: { “attachment”: { “type”: “audio”, “payload”: { “src”: “audio/Hi.mp3” }, “autoplay”: true } } that what I am getting