Hi everyone, I’m having trouble implementing my fallback utter message. As of now, I already defined utter_default template on the domain.yaml and registered it as an action in that same file. Also, I created two stories: ## story_fallback * out_of_scope - action_default_fallback ## fallback - utter_fallback The first one is working: when the intent is out_of_scope (I have some examples of this on my nlu_data.md) it activates the action_default_fallback and spits out my utter_fallback template. However, since I’m training my model using: python -m rasa_core.train -d domain.yml -s data/stories.md -o models/current/dialogue --nlu_threshold 0.1 --core_threshold 0.1 --fallback_action action_default_fallback I was expecting the model to fallback to utter_fallback when the NLU confidence was bellow 0.1. But on reality this doesn’t happen: 2018-10-04 15:21:03 DEBUG urllib3.connectionpool - Starting new HTTP connection (1): localhost 2018-10-04 15:21:03 DEBUG rasa_core.tracker_store - Recreating tracker for id 'default' 2018-10-04 15:21:03 DEBUG rasa_core.processor - Received user message '' with intent '{'name': None, 'confidence': 0.0}' and entities '[]' 2018-10-04 15:21:03 DEBUG rasa_core.processor - Logged UserUtterance - tracker now has 19 events 2018-10-04 15:21:03 DEBUG rasa_core.processor - Current slot values: address: None delivery_state: None name: paulo 2018-10-04 15:21:03 DEBUG rasa_core.policies.memoization - Current tracker state [{'slot_name_0': 1.0, 'intent_name': 1.0, 'prev_action_listen': 1.0}, {'slot_name_0': 1.0, 'intent_name': 1.0, 'prev_utter_greet': 1.0}, {'prev_action_listen': 1.0, 'slot_name_0': 1.0}] 2018-10-04 15:21:03 DEBUG rasa_core.policies.memoization - There is no memorised next action 2018-10-04 15:21:03 DEBUG rasa_core.policies.ensemble - Predicted next action using policy_2_KerasPolicy 2018-10-04 15:21:03 DEBUG rasa_core.processor - Predicted next action 'utter_thanks' with prob 0.66. As you can see, it predicts another action (utter_thanks) when I type an empty string (that has an intent confidence of 0.0). What am I doing wrong? I followed all the directions on the tutorial and I still can’t default when the confidence is low.

if some one help according to rasa-core==0.9.4 rasa-nlu==0.12.3

Anyone has any idea of what is happening?

@vinaynagpal9 we only added the fallback policy in version 0.10 as far as i remember, please upgrade to the latest version 0.11 @prcastro that looks strange. which version of core are you using?

I’m using rasa_core version 0.11.1

@prcastro got this from the code type: (…) -> bool """It should predict fallback action only if a. predicted NLU confidence is lower than ``nlu_threshold`` && b. last action is NOT fallback action """ Was your last action a fallback action by any chance?

No, this happens even at the very start of the conversation

can you paste the contents of your metadata.json from your trained model here please? Also try upgrading to 0.11.11, that’s the latest version

Do you mean my NLU model? Here it is: { "language": "pt", "pipeline": [ { "model": "pt", "case_sensitive": false, "name": "nlp_spacy", "class": "rasa_nlu.utils.spacy_utils.SpacyNLP" }, { "name": "tokenizer_spacy", "class": "rasa_nlu.tokenizers.spacy_tokenizer.SpacyTokenizer" }, { "name": "intent_featurizer_spacy", "class": "rasa_nlu.featurizers.spacy_featurizer.SpacyFeaturizer" }, { "name": "intent_entity_featurizer_regex", "regex_file": "regex_featurizer.json", "class": "rasa_nlu.featurizers.regex_featurizer.RegexFeaturizer" }, { "BILOU_flag": true, "features": [ [ "low", "title", "upper" ], [ "bias", "low", "prefix5", "prefix2", "suffix5", "suffix3", "suffix2", "upper", "title", "digit", "pattern" ], [ "low", "title", "upper" ] ], "max_iterations": 50, "L1_c": 0.1, "L2_c": 0.1, "name": "ner_crf", "classifier_file": "crf_model.pkl", "class": "rasa_nlu.extractors.crf_entity_extractor.CRFEntityExtractor" }, { "name": "ner_synonyms", "synonyms_file": "entity_synonyms.json", "class": "rasa_nlu.extractors.entity_synonyms.EntitySynonymMapper" }, { "C": [ 1, 2, 5, 10, 20, 100 ], "kernels": [ "linear" ], "max_cross_validation_folds": 5, "scoring_function": "f1_weighted", "name": "intent_classifier_sklearn", "classifier_file": "intent_classifier_sklearn.pkl", "class": "rasa_nlu.classifiers.sklearn_intent_classifier.SklearnIntentClassifier" } ], "training_data": "training_data.json", "trained_at": "20181009-114048", "rasa_nlu_version": "0.13.5" } I’m retraining with the new version you suggested now. Let’s see.

Apparently it was a problem with the version, now it is working. Thanks for the tip.

Implementing simple fallback

Rasa Open Source

vinaynagpal9 (Vinay Nagpal) October 8, 2018, 7:36am 2

i have also same problem

Topic		Replies	Views
Fallback implementation in new version Rasa Open Source	7	1634	January 8, 2019
Fallback error Rasa Open Source	8	1174	February 20, 2020
Fallback thresholds is wrong Rasa Open Source	2	519	May 21, 2019
Rasa_core threshold working for fallback policy Rasa Open Source	0	259	February 19, 2021
Fallback policy for action and NLU Rasa Open Source	1	442	October 25, 2022

Implementing simple fallback

Related topics