TED policy predicton not consistent

Hi All,

We are building a bot using rasa open source, but the next action prediction made by bot is inconsistent i.e. at time it selects wrong action, sometimes it selects the right action but with low confidence and hence goes to fallback, while other times it works fine on the same stories.

Interestingly, retraining the NLU w/o changing anything fixes the issue (although sometimes, I have to train it multiple times to make it behave as per the stories).

Below are some details -

Rasa Version: Rasa Version : 2.8.12 Minimum Compatible Version: 2.8.9 Rasa SDK Version : 2.8.2 Rasa X Version : None Python Version : 3.6.8 Operating System : Linux-4.18.0-240.10.1.el8_3.x86_64-x86_64-with-redhat-8.3-Ootpa

No errors when validated the NLU intents and stories using -
rasa data validate rasa data validate stories --max-history X

Attached the config.yml for policies. config.yml (2.2 KB)

Thanks! Akshay

Any pointers will be highly appreciated…