Adding LLM to rasa-open source, to rephrase responses

As you can see from the json example on the docs page, the NLG endpoint you create would receive the name you’ve chosen for the utterance, utter_what_can_do in the example. Your NLG would have some full text for the utter_what_can_do key and send the full text to the LLM to ask it to rephrase. You could pull the text from your domain.yml if you want. There is an example NLG in the Rasa repo here. I have also written a Django based NLG server with multi-language in mind here.

An advantage of the NLG approach is that the model is not involved. There are no changes. You only have to maintain a list of responses in the domain so these only change when you add/remove a response. You can change the response sent to the user in the NLG. In your case the LLM is generating the response.