Voice-message

Rasa Open Source
1

Hi, Can I create a custom action that can convert each voice-message recieved from Telegram to text? and the send it to the bot

2

Custom Actions are used AFTER receiving the user message.

For doing things BEFORE receiving the user message, look at Custom Connectors and Custom Components.

3

@ChrisRahme thanks, can you please tell me how to test the type of the message in a custom component?

If message= texte
pass #so nothing to do
elif message=audio 
message = r.recognize_google(audio)
4

It depends on the format of the message you’re receiving. This is controlled by your interface, not by Rasa.

5

@ChrisRahme I didn’t understand. So my idea is to create a custom component that’s check the incoming message from the user if it is in format text it will be forwarded to the next component in the pipeline else if it is audio it will be converted to text then will be frowarded to the next component

6

Yes I know.

But the message is sent from your chatting interface. So you could send it from there as a JSON. For example:

{ "type": "text", "data": message }
{ "type": "audio", "data": message }

And you check for the type in the custom connector.

Sine Rasa requires to receive the message in the { sender, message } format, you’ll have to send it like so:

{
  "sender": sender_id,
  "message": {
    "type": type
    "data": data
  }
}
7

@ChrisRahme, If I understand I have to create a .json file that’s get the message from the chatting interface then forworded it to Rasa as message with type (text or audio) then I have to create custom connector to check the message type Finally I have to create custom component to convert audio type to text. Is this correct?

8

No, don’t create a JSON file. You send the message as JSON from the interface (meaning you have to change the code of the interface).

Then you check the type AND convert to text in the connector.

9

If I understand your use case, you wouldn’t want to do this with a custom component. I assume Telegram is sending the voice recording as an mp3 or other audio format and I’ll assume you want to convert that to text and send the message on to Rasa?

To do this, I would look at the modifying the Telegram code about here. Check for the message containing the audio file, convert to text and assign to the text variable.

10

@stephens thanks actually I’m trying to modify this file but until i didn’t success to put it on I still trying :pensive:

11

If you aren’t very familiar with git and modifying code there’s a contributor post that may be helpful as well as the main project README.md.

1 Like