Topic switching and story dependent confidence

Dear All :slight_smile: ,

While experimenting with different stories I saw that Rasa would easily switch topics, when an answer was not entirely as expected. Did any of you encounter a way to prevent the chatbot from switching away from a story? I was thinking of two possible scenarios and was wondering if some of you maybe already implemented something like this.

  1. Is there a way to limit the set of possible answers given? So let’s say the chatbot answered with a question and just three possible intent would be a valid user answer at this point in the story. Would it be possible to limit the intent recognition just to these intents?
  2. Or is there a simple/rasa native way to add “switching costs”, so that an intents that is not expected at this point in the story will have a reduced confidence?

Maybe it would also be better, to use different classifiers, in order to train for these intents, however, it seems like all classifiers rely on SVM, so I am not quite sure what use it might have to switch the classifier, or are there some classifiers who can be trained on the stories as well? Do you guys have any recommendations, for which scenarios which classiefiers work best? Have you also tried other (external, own) classifiers?

1 Like

Hi @terweh! Have you looked into the different policies? The memoization policy is useful if you want Core to follow your training stories exactly, while the two-stage fallback policy is helpful if you want to disambiguate what a user has said when your NLU confidence is low. I also recommend reading about how action selection works.

Before you start thinking about using different classifiers, are you sure you have enough reliably labeled NLU training data? Likewise, do your stories cover how your users interact with your assistant? Using Rasa X to learn from real conversations is a great way to generate realistic data and build a bot that can handle how your users actually interact with it. If you have not checked it out, I would recommend looking into it