We have been working on an exciting new feature that allows you to check if your stories are consistent. Checkout this Colab notebook and let us know what you think, before we make it part of an official Rasa release. You find specific questions we’d like to hear your opinion on at the end of the notebook.
This tool is absolutely necessary, and it should be running whenever training happens. TBH it should not even be possible to write new stories in Rasa X without validation from this tool. When multiple members of a team are working together it is very easy for one to accidentally create conflicting stories.
Edit: One minor thing thing, maybe it has already been addressed. When the domain file is downloaded from Rasa X, it is named domain.yaml. The tool expects domain.yml so the file has to be renamed before the tool can find it.
With our new CI you can just add a line to the workflow:
This way nobody on your team can commit changes to your bot that break the stories
I think it would be too slow to run it constantly, however, at least when you have a large number of stories. Although… thinking about it, I could make it faster for a sort of continuous mode…