Hello,
While I’m in a form, and the bot detects an intent with a trigger action with a probability above the threshold, it reacts with the mapped action. Yet from what I understand from the documentation, FormPolicy should take priority over MappingPolicy. What can I do to change this ?
I use the following policies :
policies:
We document the policy priorities on the policies page here. As you stated, the FormPolicy takes priority over the MappingPolicy. If you enable verbose debugging with the --debug option, you will see the scores and process as the policy is run.
I suspect this could be an issue in the form but the logs will help you confirm what is going on.
This only happens to me since I upgraded to version 1.9.6 (while it worked in the previous version of rasa that I was using 1.7)
When I launch my form, I only want the exact content of the message regardless of the intent and I only want to fill in the “detail_category” slot. So I used “{“detail_category”: self.from_text(intent=None)}”.
Here is an example where I first send a message that will trigger an action “action_order_search”, which will launch the “form_order_search” form.
Your input -> show me my order list 2020-05-22 17:03:24 DEBUG rasa.core.tracker_store - Recreating tracker for id '6f8e5af849d1440b82404baa520fcea9' 2020-05-22 17:03:24 DEBUG rasa.core.processor - Received user message 'show me my order list' with intent '{'name': 'show_orders', 'confidence': 0.9576535820960999}' and entities '[]' 2020-05-22 17:03:24 DEBUG rasa.core.processor - Logged UserUtterance - tracker now has 440 events. 2020-05-22 17:03:24 DEBUG rasa.core.policies.form_policy - There is no active form 2020-05-22 17:03:24 DEBUG rasa.core.policies.fallback - NLU confidence threshold met, confidence of fallback action set to core threshold (0.7). 2020-05-22 17:03:24 DEBUG rasa.core.policies.mapping_policy - The predicted intent 'show_orders' is mapped to action 'action_order_search' in the domain. 2020-05-22 17:03:24 DEBUG rasa.core.policies.ensemble - Predicted next action using policy_2_MappingPolicy 2020-05-22 17:03:24 DEBUG rasa.core.processor - Predicted next action 'action_order_search' with confidence 1.00. 2020-05-22 17:03:24 DEBUG rasa.core.actions.action - Calling action endpoint to run action 'action_order_search'. 2020-05-22 17:03:24 DEBUG rasa.core.processor - Action 'action_order_search' ended with events '[BotUttered('form_order_search will be launch', {"elements": null, "quick_replies": null, "buttons": null, "attachment": null, "image": null, "custom": null}, {}, 1590159804.5780165), <rasa.core.events.FollowupAction object at 0x7f600f7f5b10>]'. 2020-05-22 17:03:24 DEBUG rasa.core.processor - Current slot values: detail_category: None form_rainbow_account: None list_orders: None order_id: None requested_slot: None sent_token: None soft_category: None spreadsheet_form: None user_changed_timezone: None verified_order_id: None verified_token: None 2020-05-22 17:03:24 DEBUG rasa.core.processor - Predicted next action 'form_order_search' with confidence 1.00. 2020-05-22 17:03:24 DEBUG rasa.core.actions.action - Calling action endpoint to run action 'form_order_search'. form_order_search will be launch 2020-05-22 17:03:24 DEBUG rasa.core.processor - Action 'form_order_search' ended with events '[<rasa.core.events.Form object at 0x7f600fb62510>, <rasa.core.events.SlotSet object at 0x7f600f714e90>]'. 2020-05-22 17:03:24 DEBUG rasa.core.processor - Current slot values: detail_category: None form_rainbow_account: None list_orders: None order_id: None requested_slot: detail_category sent_token: None soft_category: None spreadsheet_form: None user_changed_timezone: None verified_order_id: None verified_token: None 2020-05-22 17:03:24 DEBUG rasa.core.policies.form_policy - There is an active form 'form_order_search' 2020-05-22 17:03:24 DEBUG rasa.core.policies.fallback - NLU confidence threshold met, confidence of fallback action set to core threshold (0.7). 2020-05-22 17:03:24 DEBUG rasa.core.policies.mapping_policy - There is no mapped action for the predicted intent, 'show_orders'. 2020-05-22 17:03:24 DEBUG rasa.core.policies.ensemble - Predicted next action using policy_0_FormPolicy 2020-05-22 17:03:24 DEBUG rasa.core.processor - Predicted next action 'action_listen' with confidence 1.00. 2020-05-22 17:03:24 DEBUG rasa.core.processor - Action 'action_listen' ended with events '[]'. 2020-05-22 17:03:24 DEBUG rasa.core.lock_store - Deleted lock for conversation '6f8e5af849d1440b82404baa520fcea9'.
Then a “bonyour” message received with an intent “orders_backlog_summary” but with a confidence below my threshold. You can see in the logs that he chose the action predicted by the form_policy and that my slot has been filled:
> Your input -> bonyour
> 2020-05-22 17:04:31 DEBUG rasa.core.tracker_store - Recreating tracker for id '6f8e5af849d1440b82404baa520fcea9'
> 2020-05-22 17:04:31 DEBUG rasa.core.processor - Received user message 'bonyour' with intent '{'name': 'orders_backlog_summary', 'confidence': 0.37352967262268066}' and entities '[]'
> 2020-05-22 17:04:31 DEBUG rasa.core.processor - Logged UserUtterance - tracker now has 448 events.
> 2020-05-22 17:04:31 DEBUG rasa.core.policies.form_policy - There is an active form 'form_order_search'
> 2020-05-22 17:04:31 DEBUG rasa.core.policies.fallback - NLU confidence 0.37352967262268066 is lower than NLU threshold 0.60.
> 2020-05-22 17:04:31 DEBUG rasa.core.policies.mapping_policy - The predicted intent 'orders_backlog_summary' is mapped to action 'action_backlog_value' in the domain.
> 2020-05-22 17:04:31 DEBUG rasa.core.policies.ensemble - Predicted next action using policy_0_FormPolicy
> 2020-05-22 17:04:31 DEBUG rasa.core.processor - Predicted next action 'form_order_search' with confidence 1.00.
> 2020-05-22 17:04:31 DEBUG rasa.core.actions.action - Calling action endpoint to run action 'form_order_search'.
> 2020-05-22 17:04:31 DEBUG rasa.core.processor - Action 'form_order_search' ended with events '[BotUttered('bonyour', {"elements": null, "quick_replies": null, "buttons": null, "attachment": null, "image": null, "custom": null}, {}, 1590159871.9481802), <rasa.core.events.SlotSet object at 0x7f600f704f90>, <rasa.core.events.FollowupAction object at 0x7f600f704f10>, <rasa.core.events.Form object at 0x7f600f704310>, <rasa.core.events.SlotSet object at 0x7f600f704590>]'.
> 2020-05-22 17:04:31 DEBUG rasa.core.processor - Current slot values:
> detail_category: bonyour
> form_rainbow_account: None
> list_orders: None
> order_id: None
> requested_slot: None
> sent_token: None
> soft_category: None
> spreadsheet_form: None
> user_changed_timezone: None
> verified_order_id: None
> verified_token: None
> 2020-05-22 17:04:31 DEBUG rasa.core.processor - Predicted next action 'action_listen' with confidence 1.00.
> 2020-05-22 17:04:31 DEBUG rasa.core.processor - Action 'action_listen' ended with events '[]'.
> 2020-05-22 17:04:31 DEBUG rasa.core.lock_store - Deleted lock for conversation '6f8e5af849d1440b82404baa520fcea9'.
> bonyour
Here is a second example where I activate my form in the same way. This time, I send “backlog” instead of “bonyour” which is again understood by the intention “orders_backlog_summary” but with a confidence higher than my threshold. You can see in the logs and in the message it returns that it then chooses the action predicted by the mapping_policy which takes priority over the form_policy while the form is active :
> Your input -> backlog
> 2020-05-22 17:53:44 DEBUG rasa.core.tracker_store - Recreating tracker for id '6fcb29305e5f4a908a03a972f342f595'
> 2020-05-22 17:53:44 DEBUG rasa.core.processor - Received user message 'backlog' with intent '{'name': 'orders_backlog_summary', 'confidence': 0.9980136752128601}' and entities '[]'
> 2020-05-22 17:53:44 DEBUG rasa.core.processor - Logged UserUtterance - tracker now has 12 events.
> 2020-05-22 17:53:44 DEBUG rasa.core.policies.form_policy - There is an active form 'form_order_search'
> 2020-05-22 17:53:44 DEBUG rasa.core.policies.fallback - NLU confidence threshold met, confidence of fallback action set to core threshold (0.7).
> 2020-05-22 17:53:44 DEBUG rasa.core.policies.mapping_policy - The predicted intent 'orders_backlog_summary' is mapped to action 'action_backlog_value' in the domain.
> 2020-05-22 17:53:44 DEBUG rasa.core.policies.ensemble - Predicted next action using policy_2_MappingPolicy
> 2020-05-22 17:53:44 DEBUG rasa.core.processor - Predicted next action 'action_backlog_value' with confidence 1.00.
> 2020-05-22 17:53:44 DEBUG rasa.core.actions.action - Calling action endpoint to run action 'action_backlog_value'.
> 2020-05-22 17:53:44 DEBUG rasa.core.processor - Action 'action_backlog_value' ended with events '[BotUttered('the action to show backlog', {"elements": null, "quick_replies": null, "buttons": null, "attachment": null, "image": null, "custom": null}, {}, 1590162824.3366327)]'.
> 2020-05-22 17:53:44 DEBUG rasa.core.processor - Current slot values:
> detail_category: None
> form_rainbow_account: None
> list_orders: None
> order_id: None
> requested_slot: detail_category
> sent_token: None
> soft_category: None
> spreadsheet_form: None
> user_changed_timezone: None
> verified_order_id: None
> verified_token: None
> 2020-05-22 17:53:44 DEBUG rasa.core.policies.form_policy - There is an active form 'form_order_search'
> 2020-05-22 17:53:44 DEBUG rasa.core.policies.fallback - NLU confidence threshold met, confidence of fallback action set to core threshold (0.7).
> 2020-05-22 17:53:44 DEBUG rasa.core.policies.mapping_policy - The mapped action, 'action_backlog_value', for this intent, 'orders_backlog_summary', was executed last so MappingPolicy is returning to action_listen.
> 2020-05-22 17:53:44 DEBUG rasa.core.policies.ensemble - Predicted next action using policy_2_MappingPolicy
> 2020-05-22 17:53:44 DEBUG rasa.core.processor - Predicted next action 'action_listen' with confidence 1.00.
> 2020-05-22 17:53:44 DEBUG rasa.core.processor - Action 'action_listen' ended with events '[]'.
> 2020-05-22 17:53:44 DEBUG rasa.core.lock_store - Deleted lock for conversation '6fcb29305e5f4a908a03a972f342f595'.
> the action to show backlog
And I was wondering if there wasn’t a link with the following change when you moved to version 1.9.0. :
#5205: Fixed the bug when FormPolicy was overwriting MappingPolicy prediction (e.g. /restart ). Priorities for Mapping Policy and Form Policy are no longer linear: FormPolicy priority is 5, but its prediction is ignored if MappingPolicy is used for prediction.
Can I change this behavior? Because not being able to use the intentions mapped in the domain in the forms is a bit restrictive.
Frank,
Can you provide the stories related to this form and that use the intent examples you used.
Greg
I didn’t use stories because all my intent are mapped with the MappingPolicy.
Yes, i tried it, but no difference