Translation Layer

Rasa Pro CALM
1

Currently Rasa PRO with CALM does not support translation out of the box. I think it would be a good idea to make it possible to add a translation layer to translate user messages to English, save the original language in metadata and translate bot’s message back to the original message using NLG maybe (prompt it so that it will rephrase + translate).

2

The contextual rephraser can do translation with the LLM if you customise the prompt: Contextual Response Rephraser

You can store the user’s language as a slot or you can tell the LLM to detect the user’s language and respond in the same language by adding to the rephraser prompt something like below:

Respond in the same language as USER’s Current input. Language translation is acceptable if needed.

1 Like
4

But how to integrate translation of user’s message to English with the current architecture? Creating a channel for this disables a possibility to test a bot from shell/inspect. Creating a GraphComponent which will intercept the message and change its text is not changing it in tracker events, therefore “current_conversation” variable will still hold original values.

5

Why is it important to have the user’s messages to be all translated to English in the tracker? LLMs are able to understand a lot of languages so it should perform fine with a mix of languages in the current conversation.

6

Mainly, because you use embeddings for flow retrieval. And yes there are multilanguage embeddings but they perform worse for languages other than English. And for some not so popular languages they perform terrible.

Secondly, cheaper models (gpt-3.5) perform far worse when dealing with languages other than English. So translating user’s input to English with a model like gpt-4 is optimal option to keep the quality of bot.

7

Using github codespaces to try Rasa Pro CALM, I’ve seted the fallowing. Albeit it works to translate utter_free_chitchat_response, it does not work to utter_ask_recipient. What should I be missing?

endpoints.yml

nlg: type: rephrase rephrase_all: true

domain.yml

    rephrase: True
    rephrase_prompt: |
      The following is a conversation with an AI assistant built with Rasa.
      The assistant can help the user transfer money.
      The assistant is helpful, creative, clever, and very friendly.
      The user is making small talk, and the assistant should respond, keeping things light.
      Always output the response in brazilian portuguese language.
      Context / previous conversation with the user:
      {{history}}
      {{current_input}}
      Suggested AI Response, brazilian portuguese:
8

I’ve got it, added prompt to my endpoint.yml

prompt: prompts/response-rephraser-template.jinja2