EnterpriseSearchPolicy and LLM Prompt

Rasa CALM
1

Hi,

Hope everyone is doing well. I have a question in regards to EnterpirseSearch policy. I’m creating an FAQ element of the chatbot and it seems when I ask questions like “Can I get a discount” or “Can I cancel an order?”, the LLM doesn’t categorize it to be a knowledgebase question and instead as an action. Ultimately, it goes to “pattern_cannot_handle”. If rephrase the question as “Do you give discounts” or “do you allow the cancellation of orders” then it gives the appropriate response.

As I’m new to Rasa and CALM, I can’t really figure out how to overcome this issue. The only thing I can think of is perhaps create a custom action on pattern_cannot_handle and redirect it to use the pattern_search flow? But then what about the case where it actually can’t handle something or doesn’t find anything in the knowledgebase, wouldn’t it be a loop?

Any advice on how to approach this?

Another question in regards to the LLM, is the only way to avoid having answers that start “Based on the documents provided…”, to customise the prompt?

Apologies for the essay, but any help would be much appreciated. Kind regards

2

Hey @Masroor, you can create a new flow that describes the additional situations you would like EnterpriseSearch Policy to be triggered. This flow can invoke the action action_trigger_search to trigger EnterpriseSearch Policy. It would look like this,

flows:
  respond_to_faq:
    name: Respond to FAQ
    description: Respond to questions about Discounts and Order Cancellations
    steps:
      - action: action_trigger_search

You can modify the description of this flow to be more generic enough to fit all the FAQ that you have.

The only thing I can think of is perhaps create a custom action on pattern_cannot_handle and redirect it to use the pattern_search flow

You don’t need a custom action for that, just use action_trigger_search to invoke ES Policy from any pattern or flows.

is the only way to avoid having answers that start “Based on the documents provided…”, to customise the prompt?

Yes

1 Like
3

Thanks for the help, much appreciated! That seems to work