Two stage fallback

Rasa Open Source
1

Hello All. I have been using two stage fallback with custom ask_affirm function. How do I handle a situation where, the user doesn’t select any of the options provided by the affirm action but goes on typing what he/she wants. As of now this triggers the rephrase utterance and it goes on infinite loop. What I would ideally want is, if the user is not selecting from the button provided and types something and if that text having some high confidence with any of the trained intents, that story should be triggered. Which is not happening currently. Can any of you help me here.

Thank you in advance.

2

@sML can you share us the config.yml file and section of domain.yml file in which you written message?

PS: If you are 100% sure with your code, can I ask you to delete the older trained model and train and run again, and let me know if issue is still persist ?

3

Thanks for prompt reply. Sure. Here is the config -

   - name: HFTransformersNLP
     model_name: bert
     model_weights: rasa/LaBSE
   - name: LanguageModelTokenizer
   - name: LanguageModelFeaturizer
   - name: RegexFeaturizer
   - name: LexicalSyntacticFeaturizer
   - name: CountVectorsFeaturizer
   - name: CountVectorsFeaturizer
     analyzer: char_wb
     min_ngram: 1
     max_ngram: 4
   - name: DIETClassifier
     epochs: 100
     constrain_similarities: true
   - name: EntitySynonymMapper
   - name: ResponseSelector
     epochs: 100
     constrain_similarities: true
   - name: FallbackClassifier
     threshold: 0.65
policies:
   - name: MemoizationPolicy
     max_history: 2
   - name: RulePolicy
   - name: FormPolicy
   - name: TEDPolicy
     max_history: 5
     epochs: 100
     constrain_similarities: true

I have customized the functions action_default_ask_affirmation and action_default_fallback. To make the response little more readable. domain file contains utterances for utter_ask_rephrase - which are simple sentence to rephrase the question. I retrained the model as you said. What i observe is, the rephrase message is going on a loop only for out_of_scope questions. Is there anything i can do there ? If the text is going to be classified as out_of_scope, i want the action_out_of_scope to be called, which provides proper answer.

Thank you in advance.

4

@sML

please update this in your config.yml

- name: RulePolicy
    nlu_threshold: 0.4
    core_threshold: 0.4
    fallback_action_name: "action_default_fallback"

In domain.yml


utter_default:
    - text: I'm sorry, I didn't quite get you? Please try again.

Please do ref this link: Fallback and Human Handoff I know you did, but still have close look.

This approach can solve your issue. please try and let me know.

5

@nik202. Thank you again. I tired as you said, but no luck. Appending some debug logs for review.

rasa.core.policies.rule_policy - Current tracker state:
[state 1] user intent: nlu_fallback | previous action name: action_listen
[state 2] user intent: nlu_fallback | previous action name: action_two_stage_fallback | active loop: {'name': 'action_two_stage_fallback'}
[state 3] user text: game of thrones | previous action name: action_listen | active loop: {'name': 'action_two_stage_fallback'}
**DEBUG** rasa.core.policies.rule_policy - There is no applicable rule.
 **DEBUG** rasa.core.policies.rule_policy - Predicted loop 'action_two_stage_fallback'.
 **DEBUG** rasa.core.policies.form_policy - There is an active form 'action_two_stage_fallback'
**DEBUG** rasa.core.policies.ted_policy - TED predicted 'action_out_of_scope' based on user intent.
 **DEBUG** rasa.core.policies.ensemble - Made prediction using user intent.
 **DEBUG** rasa.core.policies.ensemble - Added `DefinePrevUserUtteredFeaturization(False)` event.
**DEBUG** rasa.core.policies.ensemble - Predicted next action using policy_1_RulePolicy.
**DEBUG** rasa.core.processor - Predicted next action 'action_two_stage_fallback' with confidence 1.00.

what helped me was; i renamed my actual out_of_scope intent and now it seems to be working!

6

@sML The one I suggested is generic fallback, all just you need to mention what I suggested that’s it and it will default message for every random input which user input on the other hand, if you really want to use Two stage fallback, then please follow the doc link or please follow this video tutorial : Two Stage Fallback Policy | Rasa Chatbot - YouTube (Note: Please update the generic code as rasa is updated :slight_smile: )

@sML Please follow what I suggested first, if this solving your issue then please you can use two stage fallback?

Deal?

7

@nik202 Hey! Thanks again. Yes, generic - normal fallback is perfect. No issues on that. But I have a requirement for 2-stage fallback with some customisation. That also works fine if the user selects from the buttons provided which asks the next best intent. Only problem here is, if the user doesn’t select any of the button and go on typing something random;issue pops up. But yeah, I will go through the video that you provided as well.