Okay, so let me first put forth what I did understand -
stories.md ( which has stories with checkpoints) first goes to generator module, which firstly glues together checkpoints yes? So, if we start with
story_1
- intent_1
- utter_1 > checkpoint_1
story_2
> checkpoint_1
- intent_2
- utter_2
story_3
> checkpoint_1
- intent_3
- utter_3
Then we should end up with two stories after this step yes? After this, the stories go for augmentation, yes? Assume I have no augmentation. Then the next place where the stories go is the feauturizer correct?
If I am correct so far, what I want to know is how does MaxHistoryFeaturizer then handle these stories to generate the final examples? For example, for this stories.md, how many examples are created for max_history=1, 2, 3, and 5 ?
More importantly how does it exactly slice it?
I think maybe it adds an extra start action? How does it pad exactly? Does it wraparound ( example : create a story like (intent_3, utter_3, intent_1, utter_1,etc)?
I’ve tried reading the code and making some guesses about how it does it, but then the final number of examples created always turns out to be different from the predicted :P. And the code will take me days to understand. So can anyone help me out here?