After having done some experimenting, it seems to me MemoizationPolicy has somewhat little impact on the bot in practice and is VERY easy for MemoizationPolicy to miss even literal story sequences. I’m not sure if I did something wrong, but AugmentedMemoization seems much more flexible. Could anyone confirm if my understanding is correct? Let me elaborate:
MemoizationPolicy matches only if history is exactly the same as some sampled conversations (if I understand correctly, conversations for training are composed by combining stories). So that means:
- For conversations that consist of more that one story, MemoizationPolicy depends on sampling to include the specific combination one hopes to have it recognize.
- Even if sampling includes your desired case, only clean transitions between stories can be recognized by MemoizationPolicy. Try to iniate a different story in the middle of another, and it won’t work.
- Since rules are not training data, right after rules, memoization won’t work.
- Since you need to match
max_historystates,max_historyis also the amount of turns it will take for MemoizationPolicy to just have a chance to work after messing up.
On the other side AugmentedMemoizationPolicy, as long as there’s no ambiguitiy:
- Does not depend on sampling to match transitions between separate stories.
- Can match on any individual story, no matter where it starts.
- Works after rules.
- Can match again after messing up without having to wait for
max_historyturns.
So AugmentedMemoizationPolicy could potentially pick up unintended ambiguous sequences if we’re not carefull, but on the whole makes bot behaviour closer to what we declare in stories. MemoizationPolicy kinda forces more responsibility onto TEDPolicy, and since Rasa is all about trying to generalize, maybe that’s why they use that as the default project policy. But honestly, as a beginner, it seems a lot less intuitive.