Mutlilingual bot (capturing user input and intercepting bot's output for translation)

HI I build a bot in English and now I want it to be able to dialogue in French as well.

my approach is to capture user input, detect the language and reply accordingly.

so I would like to know: 1- how can I capture user input so as to detect the language? 2- how can I intercept the bot’s output so as to translate the reply before sending it to user if need be ?

You’ll need to build a Custom Pipeline Component (read more) that detects the language of the input.

You have 2 possibilities:

1. Understand and respond in only French and English:

Introduce intents with examples in both English and French.

- intent: greet
  examples: | 
  - hello
  - bonjour
etc...

Your Component will only detect the language only and save it.

All your responses will need to be Custom Actions; in which you access the language and dispatch the correct message accordingly:

if language == "fr":
    dispatcher.utter_message("Salut! Comment puis-je vous aider?")
    #/* or dispatcher.utter_message(response = "utter_greet_fr") */
else:
     dispatcher.utter_message("Hey! How can I help you?")
    #/* or dispatcher.utter_message(response = "utter_greet_en") */

UPDATE: You can now use Conditional Response Variations instead:

responses:
  utter_greet:
    - condition:
        - type: slot
          name: language
          value: fr
      text: "Salut! Comment puis-je vous aider?"
    - text: "Hey! How can I help you?"

2. Understand and respond in any language:

Introduce intents with examples in English only.

- intent: greet
  examples: | 
  - hello
  - hey there
etc...

Your Component will detect the language, save it, and then translate the user input into English before passing it to the rest of the Pipeline, which will therefore process the translated English text and match it to an intent.

All your responses will need to be Custom Actions; in which you access the language, and translate then dispatch the message accordingly:

english_text = translateEnglish("Hey! How can I help you?", language)
dispatcher.utter_message(english_text)
1 Like

Thanks a lot for your detailed reply.

I’ll go for the 2nd approach. And surely reach back out if I encounter some difficulties

thanks again

1 Like

Did you succeed?

If yes, please mark the answer as solution. If not, tell me what’s the problem and I’ll try to help :slight_smile:

Yes I did succeed

but I have a little issue. my bot has a form and when prompting for slots forms look for the utter_ask_slot response that is defined. So I don’t know how to handle these as custom actions. I tried in the custom actions to return “utter_ask_slot” as names for the actions but it doesn’t seem to work. I get the following error for each slot (below the error for the slot first_name)

rasa.core.actions.action - Couldn't create message for response 'utter_ask_first_name'.

And for your reply I tried to mark as solution but I couldn’t find the button, don’t know why, that’s why I just liked :heart: it

1 Like

Please create a new thread for your issue so that people can see it better… Unless this is the solution: :slight_smile:

  • Don’t name your action utter_ask_first_name but action_ask_first_name
1 Like

ok let me try that out

1 Like

Thanks It worked.

but after the first slot (first_name) is filled the bot can’t accurately detect a language using that answer so in this case it brought me back to English.

Is there a way I can access the Tracker in my custom component so that I just use it to globally define a language slot after the user’s first message to the bot? Or any other way I can go around this ?

Can you please share your code here(the code for the 2nd solution), I’m working on a project like yours and I didn’t know how to handle this problem

Sorry, I haven’t noticed your last message! Hope you have solved it by now.

And as Amanda requested, it would be awesome if you could share your solution since there is so little information available for both multilingual bots as well as custom components :slight_smile:

3 Likes

Hi

actually a custom component is not defined the same way as a custom action. there are a set a methods that are required in the custom component’s class you are defining of which the Process() method which does all the magic. Here’s my code language.py (5.3 KB) .

I used google translation api for translation but looking at my process() method you can adapt to whatever translation api you’re using.

if you try to run my code it won’t work because you need my google translation api key, so just look at the process() method and adapt it.

Here’s a good ressource that helped me understand custom components

2 Likes

I will try it, thanks for your help :blush:

2 Likes

You’re welcome. I hope it’ll help

feel free to ask questions if anything seems blurry

2 Likes

@amaelbogne Can you help me ? I want to make a bot in Arabic as well as in English , I have my English data is ready and bot is trained in English but now I want to just use that English data and translate that into Arabic and give response in Arabic

1 Like

you can use language.py posted by @amaelbogne , in another file create a function to do the translation (I used easyNMT), call this function in language.py so it can traslate user’s input to english after that in actions.py use the entity “language” already defined in language.py to detect user’s message language and call the translation function to do the job, don’t forget to change your pipeline

2 Likes

Concerning your second approach (translation), I think this is a technical solution but not one that users will find attractive. In my project, we have to support English, Dutch, and Italian. Plain translation will almost certainly end in bad conversations because the translations do not consider context.

Also, I assume the way people talk in their language is not the same as how people talk in English (natively). So instead if solving the technical issue only, the real problem is that training data among languages are not a 1-to-1 translation. Would love to hear other opinions on this.

2 Likes

Hi Rishabhh17, Sorry for not getting back to you. Did @Amanda 's indication work for you? If No please could you share with us how you solved the problem, and if no let me know so I can help

Hi @ScienceGuy I don’t necessarily think using plain translation ends in bad conversations. translation APIs are doing a great job nowadays (even without context, which is a good remark though). Best way to know is to ship the assistant and get real user feedback.

Would love to know your approach on handling the multilingual aspect of your assistant

I agree, and this is why I didn’t go with this approach for my bot :slight_smile:

But as @amaelbogne, translators are getting better and I don’t think it would be that much of a huge deal.

On the downside, good translation APIs are often limited and/or paid, like Google’s (I think).

Which approach did you use please ?