Why mapping policy can have higher priority than form policy?

Rasa Open Source
1

rasa=1.10.2 rasa-sdk=1.10.1

config.yml:

...
policies:
  - name: MappingPolicy
  - name: FormPolicy
  - name: TEDPolicy
    max_history: 5
    epochs: 100
  - name: "FallbackPolicy"
    nlu_threshold: 0.7
    ambiguity_threshold: 0.1
    core_threshold: 0.7
    fallback_action_name: "action_not_clear_query"

domain.yml:

intents:
- calendar:
    triggers: calendar_form
- ticket:
    triggers: ticket_form
...
entities:
- datetime
...
slots:
  calendar_datetime:
    type: unfeaturized
    auto_fill: false
  ticket_datetime:
    type: unfeaturized
    auto_fill: false
...
forms:
- calendar_form
- ticket_form
...

actions.py:

...
class CalendarForm(FormAction):
    def name(self) -> Text:
        return "calendar_form"

    def slot_mappings(self) -> Dict[Text, Union[Dict, List[Dict]]]:
        return {
            "calendar_datetime": self.from_entity(entity="datetime")
        }
...
class TicketForm(FormAction):
    def name(self) -> Text:
        return "ticket_form"

    def slot_mappings(self) -> Dict[Text, Union[Dict, List[Dict]]]:
        return {
            "ticket_datetime": self.from_entity(entity="datetime"),
            ...
        }
...

Thank you for your wonderful work. I used your library to build a simple assistant. I used rasa=1.6.1 and rasa-sdk=1.6.1. Everything was good. I saw that you have updated your library. I tried your new version but had a problem.

rasa.core.processor  - Received user message 'tomorrow' with intent '{'name': 'calendar', 'confidence': 0.9955832958221436}' and entities '[{'entity': 'datetime', 'value': 'tomorrow', 'start': 0, 'end': 8, 'confidence': 1, 'extractor': 'ner_crf'}]'
rasa.core.policies.mapping_policy  - The predicted intent 'calendar' is mapped to  action 'calendar_form' in the domain.
rasa.core.policies.form_policy  - There is an active form 'ticket_form'
rasa.core.policies.fallback  - NLU confidence threshold met, confidence of fallback action set to core threshold (0.7).
rasa.core.policies.ensemble  - Predicted next action using policy_0_MappingPolicy
rasa.core.processor  - Predicted next action 'calendar_form' with confidence 1.00.

As you can see, MappingPolicy was chosen with a higher priority while there is an active form! It is stated here that FormPolicy has the highest priority. I saw that you changed the ensemble.py script and store form prediction separately in the _pick_best_policy function. But I don’t understand what you mean. Why give MappingPolicy a higher priority?

I think FormPolicy must have the highest priority. In my case, when we ask our user to fill ticket_datetime slot, we look for datetime in his/her response. We don’t care about its intent which can be ‘calendar’ or any other intents which are related to this kind of entity. So, it’s completely normal to see nlu detect another intent for user response. We just want datetime entity. If MappingPolicy has a higher priority, it will be difficult to complete the forms and we will not be able to fill their slots.

I just saw these two comments in a similar post:

that’s intended behavior, priority of mapping policy is lower due to implementation details (it has to be lower than fallback policy). However, mapping policy prediction will be taken over form policy prediction

Ok, but why?

if you have the cases when intents doesn’t always trigger the same action, then you shouldn’t use mapping policy

My intents always trigger the same action but the problem is the shared entities like datetime. If the user just mentions tomorrow without any verb or intent specific keyword, the predicted intent is uncertain. Predicted intent can be any intents that are related to this kind of entity.

2

the reason for mapping policy to be higher is because it supposed to be used to always trigger certain action if intent is detected and passed fallback check. I’d recommend to create stories like

* calendar:
    - calendar_form


* ticket:
    - ticket_form

instead of using MappingPolicy