Rasa is not able to pick up the correct intent even though the NLU confidence is higher than the threshold value in the config. It is deciding based on the Core threshold and the Keras Policy and picking up the wrong next action.
Config :
policies:
- name: MemoizationPolicy
max_history: 5
priority: 2
- name: KerasPolicy
epochs: 100
max_history: 5
priority: 1
batch_size: 100
validation_split: 0.2
- name: MappingPolicy
priority: 3
- name: TwoStageFallbackPolicy
nlu_threshold: 0.3
core_threshold: 0.3
ambiguity_threshold: 0.2
fallback_core_action_name: "action_default_fallback"
fallback_nlu_action_name: "action_default_fallback"
deny_suggestion_intent_name: "out_of_scope"
Below is the intent ranking from the user message:
"event":"user",
"timestamp":1571911446.4738436,
"text":"Question 1",
"parse_data":{
"intent":{
"name":"ask_faq_question_1",
"confidence":0.43492206931114197
},
"entities":[
],
"intent_ranking":[
{
"name":"ask_faq_question_1",
"confidence":0.43492206931114197
},
{
"name":"ask_faq_question_2",
"confidence":0.0868556797504425
}
Below is the bot actions logs :
{
"event":"action",
"timestamp":1571911446.4778328,
"name":"utter_help",
"policy":"policy_1_KerasPolicy",
"confidence":0.5282850861549377
}
Below is the debug logs :
2019-10-24 08:47:17 INFO engineio.server - ab1ef0275ea240d1b1c77c32e8f86b21: Received packet MESSAGE data 2["user_uttered",{"message":"question 1","customData":{},"session_id":"4ffa2d2cad764b668eb11a3c8c6ce4cb"}]
2019-10-24 08:47:17 INFO socketio.server - received event "user_uttered" from ab1ef0275ea240d1b1c77c32e8f86b21 [/]
2019-10-24 08:47:17 DEBUG rasa.core.tracker_store - Recreating tracker from sender id '4ffa2d2cad764b668eb11a3c8c6ce4cb'
2019-10-24 08:47:17 DEBUG rasa.core.processor - Received user message 'question 1' with intent '{'name': 'ask_faq_question_1', 'confidence': 0.3850984275341034}' and entities '[]'
2019-10-24 08:47:17 DEBUG rasa.core.processor - Logged UserUtterance - tracker now has 372 events
2019-10-24 08:47:17 DEBUG rasa.core.policies.memoization - Current tracker state [{'slot_anythingelse_value_0': 1.0, 'prev_utter_help': 1.0, 'intent_ask_question_1': 1.0}, {'slot_anythingelse_value_0': 1.0, 'intent_ask_faq_question_1': 1.0, 'prev_action_listen': 1.0}, {'slot_anythingelse_value_0': 1.0, 'prev_action_faqs': 1.0, 'intent_ask_faq_question 1': 1.0}, {'slot_anythingelse_value_0': 1.0, 'prev_utter_anythingelse': 1.0, 'intent_ask_faq_question 1': 1.0}, {'sl
ot_anythingelse_value_0': 1.0, 'intent_ask_question 1': 1.0, 'prev_action_listen': 1.0}]
2019-10-24 08:47:17 DEBUG rasa.core.policies.memoization - There is no memorised next action
2019-10-24 08:47:17 DEBUG rasa.core.policies.two_stage_fallback - NLU confidence threshold met, confidence of fallback action set to core threshold (0.3).
2019-10-24 08:47:17 DEBUG rasa.core.policies.ensemble - Predicted next action using policy_1_KerasPolicy
2019-10-24 08:47:17 DEBUG rasa.core.processor - Predicted next action 'utter_help' with confidence 0.60.
2019-10-24 08:47:17 DEBUG rasa.core.processor - Action 'utter_help' ended with events '['BotUttered(text: How can I help you ?, data: {"elements": null, "quick_replies": null, "buttons": null, "attachment": null, "image": null, "custom": null}, metadata: {})']'
2019-10-24 08:47:17 INFO socketio.server - emitting event "bot_uttered" to ab1ef0275ea240d1b1c77c32e8f86b21 [/]
2019-10-24 08:47:17 INFO engineio.server - ab1ef0275ea240d1b1c77c32e8f86b21: Sending packet MESSAGE data 2["bot_uttered",{"text":"How can I help you ?"}]
2