Background:
We have a Rasa-x installation with CI/CD Github Action doing the rasa train & test. We produce our test files continuously with rasa-x.
Issue:
We have already a lot of test stories and when we make a fundamental change in our approach, we must regard all the test stories.
Currently we do:
- Check the failed_test_stories.yml → Search for “#” .
Evaluate if this is a bug or a change is ok. In example e.g. below we want in the future correct in line 4 and determine that masterdata_form is ok!
3- action: action_before_order_service_form
4- action: action_get_first_name # predicted: masterdata_form
5- action: action_get_last_name # predicted: utter_bot_stores_changes_in_d
- Then we have to lookup the wrong test story in (/tests/test_stories.yml) in order to change it accordingly – via editor a/o directly in the repo.
Another way to fix it could be, throw all outdated test stories way and creating new once with interactive training in the rasay UI.
Is there any best practice approach how to more efficiently do that as it really causes high manual effort ?