Memoization Policy after Form Policy "Fail"

Rasa Open Source
1

Hi to all, I noticed that the Memoization Policy doesn’t work if contain a Form Policy with Follow Up action after submit:

## New Story
* slot{"department_id":NN}
    - slot{"department_id":NN}
    - utter_ask_fiscal_code
    - form_name
    - form{"name":"form_name"}
    - slot{"requested_slot":"fiscal_code"}
* fiscal_code{"fiscal_code":"XXXXXXXXXXX"}
    - slot{"fiscal_code":"XXXXXXXXXXX"}
    - form_name
    - slot{"fiscal_code":"XXXXXXXXXXX"}
    - form{"name":null}
    - slot{"requested_slot":null}
    - utter_ask_more
* intent_two
    - utter_to_intent_two

The follow up action is “utter_ask_more” and then with Memoization Policy with Max History = 1, the action_listen before intent_two was always predicted using Ted Policy (with high confidence). If i set bigger value for Max History the flow is always interrupted by action_listen and the following action “random” with low confidence, why? Are there error in story or something else?
Rasa version 1.9.2
Rasa X version 0.27.0

2

Sorry, I don’t understand what is the problem?

3

The problem is: the story written above with memoization policy (max history = 10 for instance) should recognize this same story, this doesn’t work, that is, the action_listen after utter_ask_more is predicted using TedPolicy so the actions following this one is not predicted with memoization policy sometime the prediction is wrong, to overcome this i setted max policy = 1 and this seem to work, so: Why this story is not predicted using memoization policy when the story is equal?

4

fiscal_code is the intent inside the form? Then it and following form_name action should be prefixed with form:. The best approach to find out is try interactive learning: Interactive Learning

5

Yes fiscal_code is the intent inside the form, I didn’t see this prefix, now i try some stories using it, however my story was composed using interactive learning inside Rasa X (version 0.27.0) i will try soon and inform you about the result :slight_smile:
Thank you so much

6 
## New Story
* slot{"department_id":NN}
    - slot{"department_id":NN}
    - utter_ask_fiscal_code
    - form_name
    - form{"name":"form_name"}
    - slot{"requested_slot":"fiscal_code"}
* form: fiscal_code{"fiscal_code":"XXXXXXXXXXX"}
    - slot{"fiscal_code":"XXXXXXXXXXX"}
    - form: form_name
    - slot{"fiscal_code":"XXXXXXXXXXX"}
    - form{"name":null}
    - slot{"requested_slot":null}
    - form: utter_ask_more
* intent_two
    - utter_to_intent_two

I tried with the story above but the result is the same, the action_listen after utter_ask_more is always predicted using TedPolicy and not MemoizationPolicy

7

It should be like that:

## New Story
    - slot{"department_id":NN}
    - utter_ask_fiscal_code
    - form_name
    - form{"name":"form_name"}
    - slot{"requested_slot":"fiscal_code"}
* form: fiscal_code{"fiscal_code":"XXXXXXXXXXX"}
    - slot{"fiscal_code":"XXXXXXXXXXX"}
    - form: form_name
    - slot{"fiscal_code":"XXXXXXXXXXX"}
    - form{"name":null}
    - slot{"requested_slot":null}
    - utter_ask_more
* intent_two
    - utter_to_intent_two
8

slot is the name of intent?

9

the story during prediction time probably contain more history than this snippet, so it doesn’t truly correspond to memorized version. Memoization policy is not there to be the main policy. It exists to ensure that if conversation goes exactly as training data, it will be followed, but training data contains mistakes, so it is ok to rely on ted predictions

10

The slot is the slot setted by the action “form_name”.
The story is the full story (i have many others).
I didn’t see issue on the story and i added the prefix form to utter_ask_more because is a followup action triggered by form_name. Is it incorrect?

11

yes, but it is triggered but it is not part of the form, so it will not be unfeaturized as smth inside the form