Failed Story in test stories

Hello, I’ve a problem with rasa test. I had some issues there with my project so I decided to make it simple - one intent, one story and one test story (based on this intent and the story). After running rasa test I get my test story in failed_test_stories.yml. After running rasa test --fail-on-prediction-errors I get WrongPredictionException, altough when a type the same sentence in rasa shell the prediction is correct. Is there any reasonably explanation of it? In story_report.json my intent travel_office_general_info has precision = 1.0. Does it have to do with action_listen which has also precision = 1.0 in story_report.json. I will be grateful for any help, any tips, ideas. Regards

Hi jalowiec,

I am hanging around with storytesting as well as you can see here:

I have currently two major topic with that:

  1. Nobody seems to know how to read the failed test stories file. For my understanding the hash-tag (#) should indicate wrong predictions, which gives you a hint where the story fails.

Problem here: I have quite often more rejected stories in the file than hash-tags. → Why is that ?

  1. I found differences between my CI/CD chain and my local test runs I am still in the middle of my investigation here, but honestly I have no clue what this might be, as all data- and config-files are the same …