Fallback policy not triggering for retrival intents or sub intents

Rasa Open Source
1

Hi

I am unable to trigger the fallback policy for rasa retrieval intents or sub-intents of the main faq class.

I’ve 220 faq questions defined as sub intents of the main intent faq. When some random questions such as ‘How is the weather or how is life going or shall I invest in cryptos?’ are asked, the system defaults to faq intent with 0.99 probability.

Afterward, it picks a random faq answer (sub intent/retrieval intent) (with the highest confidence of ~0.1) and throws back its response. I was able to ascertain this by running rasa in debug and nlu mode.

nlu and domain example:

  • intent: faq/FAQ_ID_08 examples: |

    • Asset status when workorder is raised for lost tool

    utter_faq/FAQ_ID_08:

    • text: 'The workorder status will be UNDER TOOLS SEARCH until the tool is found. ’

Domain:

utter_please_rephrase:

  • text: I’m sorry, I didn’t quite understand that. Could you rephrase?
  • text: I didn’t quite get it. Can you put it in different words?
  • text: Apologies, I did not understand. Can you ask in again in different words?

Rules:

  • rule: Ask the user to rephrase whenever they send a message with low NLU confidence steps:
    • intent: nlu_fallback
    • action: utter_please_rephrase

Config:

  • name: FallbackClassifier threshold: 0.3 ambiguity_threshold: 0.1

Rasa version: 2.4.3

2

Hi @Snorlaxcode , apologies for the late reply. This is definitely not possible with the fallback classifier threshold mechanism and is a valid feature request. Could you please open a new feature request on our github repo?

3

I am interested in this feature. Do you have an issue number that I can follow?

4

Sorry I missed Daksh’s response. I scrapped the concept of sub-intents in my original project and made ~240 separate intents to resolve this issue. I’ve raised feature request on rasa git just now. Issue #9723.

5

Thanks for that!