When form is active and we send intent that is not expected, form deactivates normally, and Rasa sends right answer to the intent, if this intent listed in story.
But if we send intent listed in rule, form deactivates, rasa send right answer from the rule and then form activates again.
Here is the log with the “intent from rule” situation:
2021-07-28 08:15:56 DEBUG rasa.core.policies.memoization - Sender: '1234567890' -> There is no memorised next action
2021-07-28 08:15:56 DEBUG rasa.core.policies.ensemble - Sender: '1234567890' -> Execution of 'form_question' was rejected. Setting its confidence to 0.0 in all predictions.
2021-07-28 08:15:56 DEBUG rasa.core.policies.ensemble - Made prediction using user intent.
2021-07-28 08:15:56 DEBUG rasa.core.policies.ensemble - Added `DefinePrevUserUtteredFeaturization(False)` event.
2021-07-28 08:15:56 DEBUG rasa.core.policies.ensemble - Sender: '1234567890' -> Predicted next action using policy_0_RulePolicy.
2021-07-28 08:15:56 DEBUG rasa.core.processor - Sender: '1234567890' -> Predicted next action 'utter_no_stories' with confidence 1.00.
2021-07-28 08:15:56 DEBUG rasa.core.nlg.callback - Requesting NLG for utter_no_stories from https://.......
2021-07-28 08:15:56 DEBUG rasa.core.processor - Sender: '1234567890'-> Policy prediction ended with events '[<rasa.shared.core.events.DefinePrevUserUtteredFeaturization object at 0x000001835C7422B0>]'.
2021-07-28 08:15:56 DEBUG rasa.core.processor - Sender: '1234567890'-> Action 'utter_no_stories' ended with events '[BotUttered('', {"elements": null, "quick_replies": null, "buttons": null, "attachment": null, "image": null, "custom": {"metadata": {"ucid": "4736734673736332urfjhfhf333"}, "next_action": "transfer", "endpoint": "", "latest_message": {"intent": {"name": "balans", "confidence": "1.0"}}, "channel": "ivr", "agent": null}}, {"button": null, "element":null, "utter_action": "utter_no_stories"}, 1627449356.878203)]'.
2021-07-28 08:15:56 DEBUG rasa.core.policies.rule_policy - Current tracker state:
[state 0] slots: {'alarm-before': (1.0,), 'question1': (1.0,), 'question2': (1.0,), 'question3': (1.0,), 'question4': (1.0,), 'question5': (1.0,), 'pretense-alarm-before': (1.0,)}
[state 1] user intent: alarm_questions | previous action name: action_listen | slots: {'alarm-before': (1.0,), 'question1': (1.0,), 'question2': (1.0,), 'question3': (1.0,), 'question4': (1.0,), 'question5': (1.0,), 'pretense-alarm-before': (1.0,)}
[state 2] user intent: alarm_questions | previous action name: form_question | active loop: {'name': 'form_question'} | slots: {'alarm-before': (1.0,), 'question1': (1.0,), 'question2': (1.0,), 'question3': (1.0,), 'question4': (1.0,),'question5': (1.0,), 'pretense-alarm-before': (1.0,)}
[state 3] user intent: balans | previous action name: action_listen | active loop: {'name': 'form_question'} | slots: {'alarm-before': (1.0,), 'question1': (1.0,), 'question2': (1.0,), 'question3': (1.0,), 'question4': (1.0,), 'question5': (1.0,), 'pretense-alarm-before': (1.0,)}
[state 4] user intent: balans | previous action name: utter_no_stories | active loop: {'name': 'form_question'} | slots: {'alarm-before': (1.0,), 'question1': (1.0,), 'question2': (1.0,), 'question3': (1.0,), 'question4': (1.0,), 'question5': (1.0,), 'pretense-alarm-before': (1.0,)}
2021-07-28 08:15:56 DEBUG rasa.core.policies.rule_policy - Predicted loop 'form_question' by overwriting 'action_listen' predicted by general rule.
2021-07-28 08:15:56 DEBUG rasa.core.policies.ted_policy - TED predicted 'action_listen' based on user intent.
What can we do to prevent this behaviour?
Given the training data and domain you posted, can you share two test stories that will demonstrate the difference you mentioned? (I.e. one story where the interjection is handled with rules and one where it’s handled based on a training story.) I’d like to train Rasa Core on your data and then reproduce the issue.