A better explanation of how the final examples are generated?

Okay, so let me first put forth what I did understand -

stories.md ( which has stories with checkpoints) first goes to generator module, which firstly glues together checkpoints yes? So, if we start with


  • intent_1
  • utter_1 > checkpoint_1


> checkpoint_1

  • intent_2
  • utter_2


> checkpoint_1

  • intent_3
  • utter_3

Then we should end up with two stories after this step yes? After this, the stories go for augmentation, yes? Assume I have no augmentation. Then the next place where the stories go is the feauturizer correct?

If I am correct so far, what I want to know is how does MaxHistoryFeaturizer then handle these stories to generate the final examples? For example, for this stories.md, how many examples are created for max_history=1, 2, 3, and 5 ?

More importantly how does it exactly slice it?

I think maybe it adds an extra start action? How does it pad exactly? Does it wraparound ( example : create a story like (intent_3, utter_3, intent_1, utter_1,etc)?

I’ve tried reading the code and making some guesses about how it does it, but then the final number of examples created always turns out to be different from the predicted :P. And the code will take me days to understand. So can anyone help me out here?