How to continue stories when fallback action is triggered (without using OOO intent)

I am working on a use case of building a hybrid chatbot using rasa. I have integrated an QnA engine to answer all the questions whose intent score falls below certain threshold, for rest of the questions I have configured stories,responses and intents. The usual flow goes like this

User question --> rasa actions triggered as configured in stories–> the answers are served from response template. This is how a usual flow works when building a chatbot using rasa for my use case I configured a custom fallback action, such that whenever there is a question having intent below 0.6, a custom_fallback_action is called which gets the answer for this question from a QnA engine.

As per the documentation, after dispatching the response I have returned [UserUtteranceReverted] from custom_fallback_actions. Now the problem here is, stories gets stuck at this point (which is totally logical), but I want bot to continue with the story since my next action is utter_ask_anything_else, which ask users if they need more help.

I happened to solve this issue earlier by introducing out_of_scope intent and feeding around 1k utterances to NLU data. Now, the requirement is to get rid of this intent as it requires nlu utterances for its OOO intent training, as it cant scale to different domains. (You see, ironically, customers don’t know beforehand what are their business questions.)

Is there any approach to solve this problem. I have seen DialgoueFlow provides similar feature where it can redirect the question some other api, serve the answer and continue with the usual flow

Let me know if anyone needs further details Thanks Anupam

Generally speaking, keeping the out_of_scope intent would be a good idea and can be scaled as long as the training examples fall outside of the generalized bot domain.

Have you tried building the stories using interactive learning? It could be that the existing stories aren’t matching what’s happening inside the bot’s tracker, hence the sticking point.