Check out our blog post on the TwoStageFallbackPolicy which came with Rasa Core 0.13. It provides an intuitive and user-friendly conversation flow in case of NLU messages with low classification confidence.
Have you tried it already? Share your experiences with us in the thread below 
3 Likes
Hey @Tobias_Wochinger, we are going to replace the new fallback policy in our project, and looking at the code it seems that the default configuration is only saying something like âDo you mean [name_intent]â, but in your blog, you are proposing a choice between (I think) the two highest confident intent.
Could you share the code for this custom action ?
I would like to personalise it with something like propose two intent if the difference in confidence between then is lower than 10, or only propose the highest confident one.
But itâs a really nice feature, everyone was looking forward to have it in our bot 
1 Like
@huberrom Great to your that you happy with the new policy
This is our current implementation which we are using inour bot on the documentation: rasa-demo/actions.py at 096568e64df4c25902f91fcf7b409fa2fcd05b16 ¡ RasaHQ/rasa-demo ¡ GitHub .
We are currently also working on an improved version of it, which is currently a pr include entities in button titles by wochinge ¡ Pull Request #277 ¡ RasaHQ/rasa-demo ¡ GitHub (might be merged till you read this 
).
The basic idea of the implementation is that we read the button titles from a csv file. We saw that it is important to add potentially recognised entities to the payload of the button and to the button message title. We first put the entities in brackets after the intent description, but now we want to actually use different button titles depending on the intent and the extracted entities.
Let me know if you have any more questions
1 Like
Hey i am having an issue when i tried to use the csv file to read the meaning of intent just like in the demo botâŚit shows up like did you mean user_ask_bot_name? It shows intent name instead of the meaning in csv file why is this?
Can you share your csv file to be sure it has the right format ?
It seems to work for me
Hey @Tobias_Wochinger, finally have the time to try the policy, look really great so far but I have a small problem :
I override both action_default_ask_rephrase and action_default_ask_affirmation because I want to change the text from english to french, so it shouldnât change the general behaviour, but I have the following problem :
(The second picture if following the first one, after asking to rephrase, Iâm just saying âVitaleâ again, and the end of the second picture is the bot asking to rephrase again)
Iâm mind, I thought that after the second âSomething elseâ, the bot would answer âSorry I donât provide this skillâ, but itâs only running âaction_default_ask_rephraseâ
Any idea on this ?
1 Like
@huberrom hey thanks for the reply, i used the same format and code in the demo bot and its not executing the right intent when i press the button i have raised the issue on the github page #3133. intent_description_mapping.csv (4.1 KB)
If anyone could help. Thanks in advance
Yes sounds good to me.
Donât you have a warning saying that it canât find the file ?
Maybe itâll help, I override both action_default_ask_rephrase and action_default_ask_affirmation with the code I found here : rasa-demo/actions.py at 096568e64df4c25902f91fcf7b409fa2fcd05b16 ¡ RasaHQ/rasa-demo ¡ GitHub and here rasa_core/action.py at 431d4c89c5025bec2be51c065add2ab272089bdf ¡ RasaHQ/rasa_core ¡ GitHub .
Add some logs to see if the csv is correctly read. Donât forget to add both of them in your domain.yml so that they are called instead of the default one
No warnings or anything!..the error i got is in issue #3133 I didnât add it in the domain file ! Thank you for the help, i ll get back after checking! âŚ
Oook sorry for the question before, but after reading the policy, I noticed this :
Args:
nlu_threshold: minimum threshold for NLU confidence.
If intent prediction confidence is lower than this,
predict fallback action with confidence 1.0.
core_threshold: if NLU confidence threshold is met,
predict fallback action with confidence
`core_threshold`. If this is the highest confidence in
the ensemble, the fallback action will be executed.
fallback_core_action_name: This action is executed if the Core
threshold is not met.
fallback_nlu_action_name: This action is executed if the user
denies the recognised intent for the second time.
deny_suggestion_intent_name: The name of the intent which is used
to detect that the user denies the suggested intents.
So I changed my policy to this :
policies:
- name: KerasPolicy
epochs: 200
max_history: 10
- name: TwoStageFallbackPolicy
nlu_threshold: 0.7
core_threshold: 0.3
fallback_nlu_action_name: "action_hors_perimetre"
fallback_core_action_name: "action_default_ask_affirmation"
deny_suggestion_intent_name: "hors_perimetre"
- name: AugmentedMemoizationPolicy
max_history: 10
And now itâs working. So fallback_nlu_action_name is the action called when the user denies the second time, and the fallback_core_action_name is the action called to propose the recognized intents. Wasnât clear to me with the doc, but working like a charm now
2 Likes
i just overrided the class ActionDefaultAskAffirmation so i can customize the default message but that didnt work i followed the example in rasa demo can any one help me
Did you put ActionDefaultAskAffirmation in your domain.yml ?
When i addded action_default_ask_affirmation to the domain.yml i start getting this error
Failed to run custom action âaction_default_ask_affirmationâ. Action server responded with a non 200 status code of 500. Make sure your action server properly runs actions and returns a 200 once the action is executed. Error: 500 Server Error: INTERNAL SERVER ERROR for url: http://localhost:5055/webhook
Well, that means that either your action server is not started (or correctly set up) or you didnât define the action_default_ask_affirmation
Well the server is working fine and here is the code that i puted in my action.py i think its fine too
> class ActionDefaultAskAffirmation(Action):
> def name(self):
> return "action_default_ask_affirmation"
> def __init__(self):
> import pandas as pd
> self.intent_mappings = {}
> # read the mapping from a csv and store it in a dictionary
> with open('data\intent_description_mapping.csv', newline='', encoding='utf-8') as file:
> csv_reader = csv.reader(file)
> for row in csv_reader:
>
> self.intent_mappings[row[0]] = row[1]
> def run(self, dispatcher, tracker, domain):
> # get the most likely intent
> last_intent_name = tracker.latest_message['intent']['name']
>
> # get the prompt for the intent
> intent_prompt = self.intent_mappings[last_intent_name]
>
> # Create the affirmation message and add two buttons to it.
> # Use '/<intent_name>' as payload to directly trigger '<intent_name>'
> # when the button is clicked.
> message_title = ("Sorry, I'm not sure I've understood "
> "you correctly đ¤ is your Question...")
> print('heyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy')
> message = "Hahahahah '{}'?".format(intent_prompt)
> buttons = [{'title': 'yes',
> 'payload': '/{}'.format(last_intent_name)},
> {'title': 'No',
> 'payload': '/out_of_scope'}]
> dispatcher.utter_button_message(message, buttons=buttons)
>
> return []I should point too that i have other actions and they are running without any problem
Do you have the logs of the action server when the action is called ?
2019-04-01 10:36:30 DEBUG rasa_core.processor - Received user message 'how can we ge tthis all' with intent '{'name': 'ask_whatspossible', 'confidence': 0.2748849016662266}' and entities '[]'
2019-04-01 10:36:30 DEBUG rasa_core.processor - Logged UserUtterance - tracker now has 32 events
2019-04-01 10:36:30 DEBUG rasa_core.processor - Current slot values:
age: None
category: None
month: None
nationality: None
requested_slot: nationality
ssc: None
ssc_level: None
year: None
2019-04-01 10:36:30 DEBUG rasa_core.policies.memoization - Current tracker state [{'intent_inform_about_internship_nationality': 1.0, 'prev_utter_start_scholarship': 1.0, 'active_form_scholarship_form': 1.0},
{'prev_scholarship_form': 1.0, 'intent_inform_about_internship_nationality': 1.0, 'active_form_scholarship_form': 1.0}, {'prev_action_listen': 1.0, 'intent_inform_about_internship_nationality': 1.0, 'active_form_scholarship_form': 1.0}, {'intent_inform_about_internship_nationality': 1.0, 'active_form_scholarship_form': 1.0, 'prev_utter_noworries': 1.0}, {'prev_action_listen': 1.0, 'intent_ask_whatspossible': 1.0, 'active_form_scholarship_form': 1.0}]
2019-04-01 10:36:30 DEBUG rasa_core.policies.memoization - There is no memorised next action
2019-04-01 10:36:30 DEBUG rasa_core.policies.form_policy - There is an active form 'scholarship_form'
2019-04-01 10:36:30 DEBUG rasa_core.policies.two_stage_fallback - User '321652a109204396829239e948e08a4d' has to affirm intent 'ask_whatspossible'.
2019-04-01 10:36:30 DEBUG rasa_core.policies.ensemble - Predicted next action using policy_2_FormPolicy
2019-04-01 10:36:30 DEBUG rasa_core.processor - Predicted next action 'scholarship_form' with prob 1.00.
2019-04-01 10:36:30 DEBUG rasa_core.actions.action - Calling action endpoint to run action 'scholarship_form'.
2019-04-01 10:36:31 DEBUG rasa_core.actions.action - Failed to validate slot nationality with action scholarship_form
2019-04-01 10:36:31 DEBUG rasa_core.policies.memoization - Current tracker state [{'intent_inform_about_internship_nationality': 1.0, 'prev_utter_start_scholarship': 1.0, 'active_form_scholarship_form': 1.0},
{'prev_scholarship_form': 1.0, 'intent_inform_about_internship_nationality': 1.0, 'active_form_scholarship_form': 1.0}, {'prev_action_listen': 1.0, 'intent_inform_about_internship_nationality': 1.0, 'active_form_scholarship_form': 1.0}, {'intent_inform_about_internship_nationality': 1.0, 'active_form_scholarship_form': 1.0, 'prev_utter_noworries': 1.0}, {'prev_action_listen': 1.0, 'intent_ask_whatspossible': 1.0, 'active_form_scholarship_form': 1.0}]
2019-04-01 10:36:31 DEBUG rasa_core.policies.memoization - There is no memorised next action
2019-04-01 10:36:31 DEBUG rasa_core.policies.form_policy - There is an active form 'scholarship_form'
2019-04-01 10:36:31 DEBUG rasa_core.policies.two_stage_fallback - User '321652a109204396829239e948e08a4d' has to affirm intent 'ask_whatspossible'.
2019-04-01 10:36:31 DEBUG rasa_core.policies.ensemble - Predicted next action using policy_3_TwoStageFallbackPolicy
2019-04-01 10:36:31 DEBUG rasa_core.processor - Predicted next action 'action_default_ask_affirmation' with prob 1.00.
2019-04-01 10:36:31 DEBUG rasa_core.actions.action - Calling action endpoint to run action 'action_default_ask_affirmation'.
2019-04-01 10:36:32 ERROR rasa_core.actions.action - Failed to run custom action 'action_default_ask_affirmation'. Action server responded with a non 200 status code of 500. Make sure your action server properly runs actions and returns a 200 once the action is executed. Error: 500 Server Error: INTERNAL SERVER ERROR for url: http://localhost:5055/webhook
2019-04-01 10:36:32 ERROR rasa_core.processor - Encountered an exception while running action 'action_default_ask_affirmation'. Bot will continue, but the actions events are lost. Make sure to fix the exception in your custom code.
2019-04-01 10:36:33 DEBUG rasa_core.processor - Failed to execute custom action.
Traceback (most recent call last):
File "C:\Users\mohamed-hassan.kadri\AppData\Local\Programs\Python\Python36\lib\site-packages\rasa_core\actions\action.py", line 342, in run
response.raise_for_status()
File "C:\Users\mohamed-hassan.kadri\AppData\Local\Programs\Python\Python36\lib\site-packages\requests\models.py", line 940, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 500 Server Error: INTERNAL SERVER ERROR for url: http://localhost:5055/webhook
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\mohamed-hassan.kadri\AppData\Local\Programs\Python\Python36\lib\site-packages\rasa_core\processor.py", line 346, in _run_action
events = action.run(dispatcher, tracker, self.domain)
File "C:\Users\mohamed-hassan.kadri\AppData\Local\Programs\Python\Python36\lib\site-packages\rasa_core\actions\action.py", line 362, in run
raise Exception("Failed to execute custom action.")
Exception: Failed to execute custom action.
2019-04-01 10:36:33 DEBUG rasa_core.processor - Action 'action_default_ask_affirmation' ended with events '[]'
2019-04-01 10:36:33 DEBUG rasa_core.policies.memoization - Current tracker state [{'prev_scholarship_form': 1.0, 'intent_inform_about_internship_nationality': 1.0, 'active_form_scholarship_form': 1.0}, {'prev_action_listen': 1.0, 'intent_inform_about_internship_nationality': 1.0, 'active_form_scholarship_form': 1.0}, {'intent_inform_about_internship_nationality': 1.0, 'active_form_scholarship_form': 1.0, 'prev_utter_noworries': 1.0}, {'prev_action_listen': 1.0, 'intent_ask_whatspossible': 1.0, 'active_form_scholarship_form': 1.0}, {'intent_ask_whatspossible': 1.0, 'active_form_scholarship_form': 1.0, 'prev_action_default_ask_affirmation': 1.0}]
2019-04-01 10:36:33 DEBUG rasa_core.policies.memoization - There is no memorised next action
2019-04-01 10:36:33 DEBUG rasa_core.policies.form_policy - There is an active form 'scholarship_form'
2019-04-01 10:36:33 DEBUG rasa_core.policies.ensemble - Predicted next action using policy_3_TwoStageFallbackPolicy
2019-04-01 10:36:33 DEBUG rasa_core.processor - Predicted next action 'action_listen' with prob 1.00.
2019-04-01 10:36:33 DEBUG rasa_core.processor - Action 'action_listen' ended with events '[]Any help guys