How and when is MemoizationPolicy used?

Hi! I’m still trying to figure out how does MemoizationPolicy work, especially in which cases it decides about next action.

I have this and some other stories of different length and intents

## alarm activate all
* alarm.activate
  - slot{"x_selection_all": "all"}
  - alarm.activate

I had a problem when the input was correctly recognized as alarm.activate intent and the slot x_selection_all (text type) was filled, other than alarm.activate action was predicted, or fallback policy was applied. I do not have any other stories like this with intent alarm.activate and filled slot x_selection_all. According to documentation

The MemoizationPolicy just memorizes the conversations in your training data. It predicts the next action with confidence 1.0 if this exact conversation exists in the training data, otherwise it predicts None with confidence 0.0 .

I would expect that MemoizationPolicy will be used and predict alarm.activate action because the input fits this exact story.

What helped was setting augmentation_factor to 0 in TEDPolicy in config file:

policies:
  - name: MemoizationPolicy
  - name: TEDPolicy
    max_history: 10
    epochs: 20
    augmentation_factor: 0
  - name: MappingPolicy
  - name: FormPolicy
  - name: FallbackPolicy
    nlu_threshold: 0.8
    core_threshold: 0.4
    fallback_action_name: action_default_fallback

So probably not generating augmented stories helped TEDPolicy to be more accurate in predicting the next action.

However, I’m still trying to understand why MemoizationPolicy is not applied here? Or to be more precise why it predicts None with confidence 0.0 because otherwise it would be used - “The bot’s next action is then decided by the policy that predicts with the highest confidence.”.

Where are you setting the slot? Can you post the debug output (use the --debug switch)?

Every single time I had intent accuracy > fallback threshold, and had a fallback action, I had a “there is no memorized next action” message in the debug output. It was always due to a situation not predicted in the stories.

Thanks for your reply, @samscudder. The slot is set through entity extraction defined in nlu training data file:

## intent:alarm.activate
- activate alarm
- activate [all](x_selection_all) alarms
- etc...

For policies settings:

policies:
  - name: MemoizationPolicy
  - name: TEDPolicy
    max_history: 10
    epochs: 20
    augmentation_factor: 0
  - name: MappingPolicy
  - name: FormPolicy
  - name: FallbackPolicy
    nlu_threshold: 0.8
    core_threshold: 0.4
    fallback_action_name: action_default_fallback

and input “activate all alarms” I get this debug output:

Your input ->  activate all alarms                                                                                                                                                                           
2020-07-17 16:45:11 DEBUG    rasa.core.tracker_store  - Creating a new tracker for id '3ace16de573042f985236f6d1d8e3189'.
2020-07-17 16:45:11 DEBUG    rasa.core.processor  - Starting a new session for conversation ID '3ace16de573042f985236f6d1d8e3189'.
2020-07-17 16:45:11 DEBUG    rasa.core.processor  - Action 'action_session_start' ended with events '[<rasa.core.events.SessionStarted object at 0x7f57a016bdd8>, <rasa.core.events.ActionExecuted object at 0x7f57a016b400>]'.
2020-07-17 16:45:11 DEBUG    rasa.core.processor  - Current slot values: 
	x_selection_all: None
2020-07-17 16:45:12 DEBUG    rasa.nlu.classifiers.diet_classifier  - There is no trained model for 'ResponseSelector': The component is either not trained or didn't receive enough training data.
2020-07-17 16:45:12 DEBUG    rasa.nlu.selectors.response_selector  - Adding following selector key to message property: default
2020-07-17 16:45:12 DEBUG    rasa.core.processor  - Received user message 'activate all alarms' with intent '{'name': 'alarm.activate', 'confidence': 0.9985135197639465}' and entities '[{'entity': 'x_selection_all', 'start': 9, 'end': 12, 'value': 'all', 'extractor': 'DIETClassifier'}]'
2020-07-17 16:45:12 DEBUG    rasa.core.processor  - Current slot values: 
	x_selection_all: all
2020-07-17 16:45:12 DEBUG    rasa.core.processor  - Logged UserUtterance - tracker now has 5 events.
2020-07-17 16:45:12 DEBUG    rasa.core.policies.memoization  - Current tracker state [None, None, None, {}, {'slot_x_selection_all_0': 1.0, 'prev_action_listen': 1.0, 'intent_alarm.activate': 1.0, 'entity_x_selection_all': 1.0}]
2020-07-17 16:45:12 DEBUG    rasa.core.policies.memoization  - There is no memorised next action
2020-07-17 16:45:12 DEBUG    rasa.core.policies.form_policy  - There is no active form
2020-07-17 16:45:12 DEBUG    rasa.core.policies.fallback  - NLU confidence threshold met, confidence of fallback action set to core threshold (0.4).
2020-07-17 16:45:12 DEBUG    rasa.core.policies.ensemble  - Predicted next action using policy_1_TEDPolicy
2020-07-17 16:45:12 DEBUG    rasa.core.processor  - Predicted next action 'alarm.activate' with confidence 0.74.
2020-07-17 16:45:12 DEBUG    rasa.core.actions.action  - Calling action endpoint to run action 'alarm.activate'.

Which is okay but I’m still wondering why MemoizationPolicy wasn’t used to predict next action when the “conversation” exactly match my scenario? :thinking:

Did you see this error?

2020-07-17 16:45:12 DEBUG rasa.nlu.classifiers.diet_classifier - There is no trained model for 'ResponseSelector': The component is either not trained or didn't receive enough training data.

Yep, I saw it. But I think it’s not an error, at least not related to my question. According to the documentation, I assume that ResponseSelector is used to handle similar intents with responses.md file.

The ResponseSelector NLU component is designed to make it easier to handle dialogue elements like Small Talk and FAQ messages in a simple manner. By using the ResponseSelector, you only need one story to handle all FAQs, instead of adding new stories every time you want to increase your bot’s scope.

I used to have this kind of intents but I’ve removed them and left ResponseSelector in the pipeline. When I put back these intents (with responses.md file) or remove ResponseSelector from the pipeline, it doesn’t have any effect on choosing the next action by TEDPolicy and this DEBUG output is gone.

So I still don’t understand why MemoizationPolicy is not used to predict next action when conversation exactly matches that one scenario.

Like I mentioned, when you get there is no memorised next action, it couldn’t find a matching story, so it’s not matching you story for some reason.

The memoization policy also has a max_history, and checks the tracker state. Looking at your log:

2020-07-17 16:45:12 DEBUG    rasa.core.policies.memoization  - Current tracker state [None, None, None, {}, {'slot_x_selection_all_0': 1.0, 'prev_action_listen': 1.0, 'intent_alarm.activate': 1.0, 'entity_x_selection_all': 1.0}]
2020-07-17 16:45:12 DEBUG    rasa.core.policies.memoization  - There is no memorised next action

I’m not sure why you’ve got None, None, None, {} in the tracker history as this could be the problem. It could be that an action (or actions) you have are returning None instead of returning an empty list. I suggest checking to see if they return an event or [].

P.S. It shouldn’t be makiing any difference in this case, but (can’t recall where exactly), one of the rasa components used to split the intent name on _. It was hard coded, and I don’t know if they changed it (this was around version 1.4.6), so I’d use alarm_activate instead of alarm.activate.

I don’t know why tracker history contains None, None, None, {} when the conversation was started as a fresh session and no custom action was called. When the conversation continues, tracker history list is filled with regular state’s dictionaries such as {'slot_x_selection_all_0': 1.0, 'prev_action_listen': 1.0, 'intent_alarm.activate': 1.0, 'entity_x_selection_all': 1.0}. I will try to look closely what’s going on and if it’s a problem.

However, I probably figured out why the next action wasn’t predicted by the MemoizationPolicy even if the story structure matches the conversation. I changed story syntax to use intent with the entity:

## alarm activate all
* alarm.activate{"x_selection_all": "all"}
  - alarm.activate

instead of SlotSet event -slot{"x_selection_all": "all"} as in my first post and it works fine now. Although the slot x_selection_all is automatically filled from the entity of the intent, it’s probably not the same as SlotSet event and using a slot to distinguish story flow is not appropriate here.

Thank you for your help, @samscudder! :slightly_smiling_face: