How does Memoization Policy with max history argument chooses its next action? Does it need to be up to max histroy identical to stories or just a sub part? I recognize that in my case MemoizationPolicy picts wrong next action although this is not part of story. When using max history = 2, in my case just the prev action might be encountered in stories.
Iām not sure i entirely understand your question, but if a history of 2 (or whatever your max history value is) is available, it takes that amount of history into consideration
Yea so the last 2 states have to coincide with one part of one of the stories or with one whole story?
1 Like
from what i noticed is one full story.
* welcome
- utter_welcome
* greet
- utter_greet
if your max history is 2, this story needs to have an exact match.
thatās my presumption
1 Like
Must this be part of any story or one whole story itself with an ID?
1 Like
@datistiquo Having a question regarding this as well. And maybe we could collaboratively work on a more extensive documentation of the entire policy part.
To my understanding after reading the featurizer and policies docs, it sounds to me like every single event
e.g. running a bot action, receiving a user message, setting slots
creates a new state. However, the memoization policy only takes into account the last max_history states at prediction time. An example: Letās take the following story
* welcome
- utter_welcome
* greet
- utter_greet
- utter_how_can_i_help
My tracker event history overall shows 11 events for this example: (only showing event types for length reasons)
1. user (user's welcome utterance)
2. action (utter_welcome action)
3. bot (bot's welcome utterance)
4. user (user's greet utterance)
5. action (utter_greet action)
6. bot (bot's greet utterance)
7. action (utter_how_can_i_help action)
8. bot (bot's how_can_i_help utterance)
9. action (listen)
However, we only care about the last (letās say 3) states at prediction, e.g. when predicting the action after utter_greet the featurized states will look something like:
[
{'prev_action_utter_welcome': 1.0, 'intent_welcome': 1.0}, # oldest featurized prediction state
{'prev_action_listen': 1.0, 'intent_greet': 1.0},
{'prev_utter_greet': 1.0, 'intent_greet': 1.0} # last featurized prediction state
]
These are essentially pairs of (last_user_message, last_bot_action) and could be read like:
- Initially, I responded with utter_welcome to userās intent_welcome (beginning of conversation)
- Then, I listened and and received intent_greet
- Just now, I responded with utter_greet to userās intent_greet
Additionally, the set slots and entities are also included and in these states and considered during prediction.
So for this example, youād already need a max_history=3 even though there are just 2 turns. Is this correct so far @akelad @souvikg10?
However, Iām still unclear about a few other memoization related things:
Augmentation Factor
I see that data is augmented generally before training and seems to not be treated differently depending on the policy that is being trained. If a story is only 3 prediction events long, but max_history is 5, then it will only be memorized if I randomly walked any of the augmented paths that include my story, correct?
This would essentially mean my augmentation factor should be so large, that almost any combination of story appears in the training data at least once. Shouldnāt I use no augmentation for Memoization though? Which would cause other ML policies to generalize worse though, right?
AugmentedMemoizationPolicy
Shouldnāt I almost always use this one instead? Since memoization takes slots into account, I typically donāt want any previously set slots to mess up my memoization, do I?
Maybe Iām getting it wrong, but letās say I have this story:
* thanks
- utter_thanks
But if the first thing my bot does is ask the user for its name and store it in a slot, then my training story (with slot name = None) and the real context (name = some name) will always differ, causing the memoization to never match this story, correct?
Inconsistent behavior due to max_history
If my max_history is small, what happens if the same last events are followed by multiple, different events? E.g. I could have
* intent1
- utter2
* intent3
- utter4
and
* intent5
- utter2
* intent3
- utter6
If max_history is so short that it only spans across utter2 & intent3, what would it do? Not predict anything? Predict utter4 or utter6 randomly?
Thanks a lot @akelad! Sorry for this very long post. Just saw many questions around this topic on the forum and lots of unclarity that I thought would be useful to address. Hope youāre the right person for this.
4 Likes
@smn-snkl I have provided a detailed solution for this, Kindly have a look.
@ashukrishna100 : Thanks for sharing your insights.
I was, however, rather looking for the algorithmic explanation so I can make sure I choose and configure my policies appropriately.
@smn-snkl That are questions I asked myself too in the past. Like: Need the whole story block matches the story or just a part of it to be used by MemoPolicy?
I think a state used for prediction is this featurized tracker state at one time which consists of intents, last actions, entitiesā¦And max_history means you have max_history of them using for prediction.
1.) memoization policy- this policy is used to remember the events as per story.
2.)keras policy: this policy is used to predict next action if the user asks something different from stories.
However,tracker depends on max_history parameters in case path of two/more stories are very similar. If we pass max_history parameter to memoization policy it compares the user path with story path and if it is same,predict the next action correctly.
If we pass max_history parameter to keras policy,it tunes our model for unknown events/not exactly same as stories.
Augmentation factor stitches stories and then the policies(keras and memoization) act on those stitched stories. You can find those sticthed stories in models/dialogue.
1 Like
[ {āprev_action_utter_welcomeā: 1.0, āintent_welcomeā: 1.0}, # oldest featurized prediction state {āprev_action_listenā: 1.0, āintent_greetā: 1.0}, {āprev_utter_greetā: 1.0, āintent_greetā: 1.0} # last featurized prediction state ]
Your list of tracker contains 3 elements i.e. your max history is 3, if u pass max history 30 the list will contain 30 elements
@datistiquo @ashukrishna100: The issue is that the exact story needs to match. Thus, if you increase the max_history to 30 as you @ashukrishna100 for example did, youāll run into quite some issues.
Let me show you a test I just performed. Letās say I only have this one story:
* greet
- utter_greet
With max_history=30 at the beginning of the conversation my tracker will be
[None, ... 28x None ..., None]
If I now say āHiā it will answer with āHiā as well because the featurized tracker state matches the training state:
[
None, ... 27x None ..., None,
{'intent_greet': 1.0, 'prev_action_listen': 1.0}
]
However, if I say āHiā again my tracker will look like this:
[
None, ... 26x None ..., None,
{'intent_greet': 1.0, 'prev_action_listen': 1.0},
{'intent_greet': 1.0, 'prev_utter_greet': 1.0}
]
This however will not match the memorized training state from above, thus leading to:
2018-12-13 15:05:22 DEBUG rasa_core.policies.memoization - There is no memorised next action
Therefore, I believe I only have two options in this scenario:
- Reduce max_history=1 will make sure I will always correctly classify this exact story. However, other longer stories potentially wonāt work anymore
- I use augmentation to make sure my short story from above is augmented in a way that stories like the second one (see below) will be memorized as well. However, the tricky thing is I have no real control over augmentation. Thus, I would need a really high augmentation factor to make sure my story appears at any position in a tracker state with max_history=30 the exact same way.
[None, ā¦ 26x None ā¦, None, {āintent_greetā: 1.0, āprev_action_listenā: 1.0}, {āintent_greetā: 1.0, āprev_utter_greetā: 1.0}]
So both options seem unpromising individually. Therefore, I believe the way to go is:
- Combine both: Use a small max_history, e.g. 2-5 to match very simple stories like the ones above. That will cover all your question - answer use cases. However, the higher your max_history, the larger your augmentation_factor should be. E.g. if I have two stories like the one above (letās say we also have *goodbye - utter_goodbye) and I set max_history=2 even though both stories have length 1, then I will definitely need augmentation to cover greet ā goodbye as well as goodbye ā greet appearing anywhere in the conversation. You know what I mean?
- Rely on an ML-based policy for everything else
What 1. allows you to do at least is that if augmentation is sufficiently large I detect all combinations of my stories so Iāll definitely find any combination of greet and goodbye, e.g. with max_history=2 and augmentation=50 no matter what I say greet ā greet, greet ā goodbye, goodbye ā goodbye, or goodbye ā greet all of them will match a training example.
That is also what my test with ~30 smalltalk stories showed:
- greet ā goodbye : There is a memorised next action ā45ā
- goodbye ā goodbye : There is a memorised next action ā0ā
- goodbye ā greet : There is a memorised next action ā46ā
However, I also already ran into the issue that greet ā greet was apparently not trained through augmentation, thus greet ā greet was predicted using the KerasPolicy ā¦
2018-12-13 15:46:51 DEBUG rasa_core.policies.memoization - Current tracker state [{'prev_action_listen': 1.0, 'intent_greet': 1.0}, {'prev_utter_greet': 1.0, 'intent_greet': 1.0}]
2018-12-13 15:46:51 DEBUG rasa_core.policies.memoization - There is no memorised next action
2018-12-13 15:46:51 DEBUG rasa_core.policies.form_policy - There is no active form
2018-12-13 15:46:51 DEBUG rasa_core.policies.ensemble - Predicted next action using policy_0_KerasPolicy
Which already shows the fallacy of the memoization policy. So for 30 simple stories, augmentation needs to be >> 50 already, potentially slowing down training a lot, especially when combined with a KerasPolicy that has max_history >> 3 as well.
2 Likes
That makes sense. And I also have been in entanglement with this too. Because when you have your happy paths of your story and user writes hiin between or you just create one additoinal new intent you donāt want to write all possible variations with this new intent? This would blow up exponentiallyā¦I hope Keras will learn that somehow by itself? But this I found difficult. If feel that like eg slots donāt have a much impact for prediction?
@smn-snkl the augmentaion factor is shared by all policies, right?
It would be nice if the devs like @akelad or @Juste clarify. I also miss in the docs the explanations of all basic components for Rasa. Many underlying stuff is very fuzzy.
@smn-snkl Do you mind chatting somehow privately (mail)?
I do agree on augmentation factor part. For bigger stories we can keep augmentation factor high which will further help in better prediction of next action but one thing is very clear,if we have similar stories then predicted actions become too ambiguous with probabilities with very little difference. For longer and similar stories we can keep augmentation factor and max_history both to be high.
Sorry about not getting back to you sooner, December has been a busy month.
Most of what you (@smn-snkl and @datistiquo) said makes sense! Increasing max_history to 30 would indeed not be ideal, because then if the conversation goes even slightly off course and you donāt have a story for it, it will derail.
The ideal max_history is of course up to what your stories look like, but 3-6 is generally enough. You can experiment with the evaluation script to see whether you need to increase it. @smn-snkl so a max history of 2 might be ok for a bit, but as soon as you have a bit more complicated stories youāll probably need to increase it. E.g. if you have these two stories:
## example1
* greet
- utter_greet
* greet
- utter_already_said_hi
## example2
* greet
- utter_greet
Youād need a max history of 3 here to take into account the previous utter_greet.
As for augmentation, it doesnāt need to grow exponentially large, the ML policies will take care of a lot of stuff. E.g. if you go off track from your stories, the KerasPolicy may jump in and then eventually when your conversation state matches with a conversation again, then the Memo policy can jump back in.
I hope that clarifies some things? Iām off for the holidays now, but will be happy to discuss this further when Iām back. Merry christmas!
1 Like
Thanks @akelad, thatās confirming my thoughts!
Sure. Let me know how I can reach out to you.
Hi all, just reopening this as we are tearing our hair out a bit with memoization. Specifically, an issue @smn-snkl raised earlier but seems to have dropped out - how memoization handles slot setting.
We have a fairly simple story, as an example:
- take_action
- utter_actions_menu
- create_livewire
- utter_confirm_livewire_intent
- utter_ask_subject
- select{āsubjectā: āClimate Changeā}
- utter_ask_livewire_content
The problem we are facing is that try whatever we can, no matter the combination of max history, aug factor, nothing, we just cannot get Memoization to handle that last step - actions are predicted right up until utter_ask_subject, then the user enters something, and it fails.
Weāve tried including explicitly the ā- slot{āsubjectā: āClimate Changeā}ā step, or removing it, and it makes no difference. We had a formAction action in there, made no difference. Have tried max histories from 2 to 30, no difference. Have reordered the story, and memoization works right up until it hits a slot fill, and then it fails, hence we think this is very likely to do with how the policy interacts with slot filling.
Anyone else run into same thing and solved it? @akelad Is this a known issue at all?
Thanks a ton,
Luke
Edit - what we have just realized is that it seems if the user types the exact same thing as in the training policy, then memoization (sometimes) recognizes it. But of course in almost all cases the user is not going to type the same content.
Anyone know if thereās a way to tell memoization to look at only what slot was filled, not the specific content that was filled - am wondering if maybe by passing a different state_featurizer? At present weāre just passing none. Is there a featurizer that ignores the content of slots?
Do you mean sometimes a different intent gets recognised?
@luke-jordan what kind of slots are you using? For some slot types the value matters, for others it only matters whether the slot is filled or not, see this