Is it possible to add a LLM model to rephrase bot responses with rasa open source?
- I think, Updating
domain.ymlresponses can help you in this. - Could be possible with a custom action or maybe add a custom component that can change the final response from rasa.
I have written a blog on one scenario where Rasa faced NLUfallback , I used chatgpt to answer int hose cases: How to use ChatGPT with Rasa - Python Warriors
What do you think of adding a NLU component Custom Graph Components to rephrase the bot’s responses based on the context of the conversation, do you think it is a good idea?
Yes, But you cannot add it at the top, Most probably you’ll have to add it at the bottom as this will be the final response and no other algorithm should run on it.
By the way if you only want to enhance the response, I think you can enhance the response from domain file first and then train the new model. If you are going to return response from a custom action then you can directly interact with LLM their and send back the response.
Most of my responses are in my domain.yml file, can you please explain what you mean by “enhance the response from domain file”, because I already have some good pre-written responses, I just wanted the bot to be able to give more humanized and personalized responses similar to this LLMs for Natural Language Generation. I can’t do that using the domain.yml file or can I?
You can. See Ideally Rasa bot return the response that comes from the domain.yml response section right. So if those responses are personalized, like you can add the user name using slot in the response or or more fields as well. This will make the response personalized.
for ref: https://rasa.com/docs/rasa/responses#using-variables-in-responses
Yeah I get that, but by personalized I mean more of giving answers based on the context so:
If I want to implement something like this what options do you think I have?
I would look at the Rasa NLG instead of a custom action or custom component.
2 Likes
Can you explain a bit more how that would work? any examples? I added the code from NLG Servers to my project, and added https://github.com/RasaHQ/rasa/blob/main/examples/nlg_server/nlg_server.py, do I have to now host a model on 5006? and do I have to fine tune the model to extract the messages and intent from json format to then generate the output? This is what I get right now when I do rasa shell --debug after running my actions and running the nlg_server.py.
As you can see from the json example on the docs page, the NLG endpoint you create would receive the name you’ve chosen for the utterance, utter_what_can_do in the example. Your NLG would have some full text for the utter_what_can_do key and send the full text to the LLM to ask it to rephrase. You could pull the text from your domain.yml if you want. There is an example NLG in the Rasa repo here. I have also written a Django based NLG server with multi-language in mind here.
An advantage of the NLG approach is that the model is not involved. There are no changes. You only have to maintain a list of responses in the domain so these only change when you add/remove a response. You can change the response sent to the user in the NLG. In your case the LLM is generating the response.
Hello, I want to make a chatbot that will take up a question from a user and extract 2 or more slots which will be prameters for my api and call that api and fetch the answer whatever that api is returning. Is this possible? Can LLM or rasa pro help in this? Is there any way I can get some guidance on this?
Yes, with Rasa Pro, you can see an example in the rasa-calm-demo here. This is a flow that calls an API to add a contact to a db.