End to end story testing; how to read results of failed_test_stories.yml

Hi all,

I am hanging around for a while now without success regarding end to end testing. It all starts already how to interprete the results of the test run being written into failed_test_stories.yml. Currently the overall results show 3 of 30 test stories are correct, which results in having 27 in the failed_test_stories.yml. What I understood so far from the docs then is that one can expect the #tag to indicate where the test story deviates from the prediction. But here my problem starts - currently I have no hashtag at all. This forum post indicates that other users seem to share this experience as well https://forum.rasa.com/t/failed-stories/46003/29

Versions:

  • Rasa Version : 2.8.26
  • Minimum Compatible Version: 2.8.9
  • Rasa SDK Version : 2.8.6
  • Rasa X Version : None
  • Python Version : 3.7.13
  • Operating System : Linux-4.15.0-175-generic-x86_64-with-Ubuntu-18.04-bionic

I tested it also with Python 3.8 → same results