My problem is: when I add new stories to my stories stack, TED predicts right next actions but with confidence lower whan 0.4. And it causes action_default_fallback (I established core_fallback_threshold: 0.4)
When I train model with stories number.yml, pass.yml, service.yml, specification.yml prediction works great
When I train model with all of the previous stories and debt.yml in addition, TED predicts next action for intent ask_for_specification right, but with low confidence: 2021-06-04 16:34:17 DEBUG rasa.core.policies.ted_policy - TED predicted 'utter_ask_for_specification_old_customer' based on user intent with confidence 0.29507872462272644
This looks like your stories might be in conflict with each other. What do you get when you run rasa data validate and rasa data validate stories --max-history X? (Here X is the max_history that you’ve set for TED.)
I tried rasa data validate and rasa data validate stories (without max_history, cause I didnt specified it in config), and I found contradicting stories. Now I fixed them, and new debt.yml file is debt.yml (17.1 KB)
But the problem stayed,TED still fails to predict right new action, or predicts it with low confidence.
I just trained and tested with your data on Rasa 2.6.2 (I replaced the debt.yml with the new one you provided). TED gets all actions right upon testing on my machine, but the circuit breaker trips. This happens when you let rasa predict more than 10 actions in a row (no user messages in between). You can fix this by setting the environment variable: