Fallback intents for context-sensitive fallbacks

Hi everyone. I thought I would share an extension that I made to enable context-specific fallback/error handling. Context-specific error handling is really good for things like:

B:"for how many people should I make the reservation?"
U:"oh you are a clever little bot aren't you!"
B:"Sorry, didn't get that. For how many people?"
U:"sooo clever you are (as yoda would say)."
B:"No, I still didn't understand that. Just let me know for how many people please"
U: "Do you know any Chuck Norris jokes?"
B:"Sorry, I don't think I can help you. Please call 666-6666 to talk with a person. Bye!"

Google has a cool link about this sort of “fallback escalation” here: https://designguidelines.withgoogle.com/conversation/conversational-components/errors.html#errors-no-match

How to do this with rasa? The FallbackPolicy is not enough, because you can only specify one fallback utterance/action which can come across robotic. “Sorry, I didn’t understand”,“Sorry, I didn’t understand”,“Sorry, I didn’t understand”,“Sorry, I didn’t understand” etc…

My approach was to define a “Fallback intent” which can then be used as an intent in story definitions. The idea is the nlu would classify anything it didn’t understand (below a certain confidence threshold) as the “fallback” intent and give this information to the core. In order to do this, I had to chime into the nlu pipeline. The first step was defining a “Fallback intent filter” or interceptor. The code looks like this:

class FallbackIntentFilter(Component):
""" der fallback intent Filter ermöglicht es uns, alle Nutzer-Eingaben, die nicht einem Intent eindeutig
eingeordnet werden können, in einen 'fallback intent' unterzubringen.
Dieser fallback intent kann dann in dialog training (stories.md) verwendet werden,
um Kontext-abhängige error-handling zu ermöglichen
"""
def __init__(self,component_config=None,threshold=0.5, fallback_intent="fallback"):
    ## threshold -- wenn intent Erkennung unter threshold, setzte fallback intent als erkannter Intent.
    ## TODO threshold auf sinnvolle Wert setzten beim error-handling (0.1 <> 0.5)
    ## mit sehr wenig intents, soll dieser wert relativ hoch gesetzt sein
    ## fallback_intent -- name von intent. Dieser Name wird in training (stories.md) verwendet
    super().__init__(component_config)
    self.fb_threshold=threshold
    self.fallback_intent = fallback_intent

def process(self, message, **kwargs):
    # type: (Message, **Any) -> None
    if message.data['intent']['confidence'] < self.fb_threshold:
        fb_intent = {'name':self.fallback_intent,'confidence':self.fb_threshold}
        message.data['intent'] = fb_intent
        message.data['intent_ranking'].insert(0,fb_intent)

The “threshold” parameter says anything below 0.5 confidence should be classified as “fallback” intent (fallback_intent parameter specifies the name of the intent)

During processing, if the intent classification confidence in the message is below the threshold, the recognized intent is replaced with the “fallback” intent.

To use this component, add it to the nlu config pipeline in nlu_config.yml (using the module path and class name)

pipeline:
(featurizers and classifiers etc...)
- name: "components.fallback_intent_filter.FallbackIntentFilter"

It should come after your classifiers, so you can override the intent classification. After this, you can use the “fallback” intent in your stories like this:

## fallback 
start
- utter_greet
- utter_welcome_prompt
- utter_ask_specifics
* fallback
- utter_fallback
* fallback
- utter_escalate
* fallback
- utter_give_up
- utter_goodbye

You can use “fallback” just like any other intent, so you can tell your bot what to do when it sees a fallback intent in a very context-specific way, specifying different actions and utterances depending on the story context.

4 Likes

That’s nice! I thought that such out of scope intent already exists for NLU? But I see right now here that there is nothing about that how to do that for NLU http://rasa.com/docs/nlu/fallback/. It just writes:

You can use the confidence score to choose when to ignore Rasa NLU’s prediction and trigger fallback behaviour, for example asking the user to rephrase. If you are using Rasa Core, you can do this using a Fallback Policy.

But it says nothing about how to that?

I also see that you can use that for unsupported intents like here mentioned https://designguidelines.withgoogle.com/conversation/conversational-components/endings.html#endings-unsupported-intent, right? Because you can set an utter_fallback after the intent fallback for every context like previous intents and actions. I was wondering how you can do that with Rasa to say thing like Ups sorry I cannot send eCards. Since this fallback is specific to user input. Now, i know how to do that! But what if the user like in this example from google ask this send eCard in the first question or out of context. So there is no context to give a context specific fallback? You have to know what the users says without traiing this case.

Thank you John, this is indeed helpful. Just one question- Where, as is which file should I define class FallbackIntentFilter(Component): ? In my action file or somewhere else?

Thank you once again, Shruti

1 Like

Hi - glad it was useful for you! I just added a python file/module with the component class. You can structure this however you please, just as long as the name you use in the nlu config (for my case : "“components.fallback_intent_filter.FallbackIntentFilter”) points to the package/module/class that you define. Here is a screenshot of my project structure - but keep in mind you are free to structure your project as you see fit! This is just an example.

image

Hi John Loutzenhiser ,

Can i use it on form field. Please let me know how to use it. Also could you please paste your image again as i am not able to see it.

Thank you :slight_smile:

Hi John Loutzenhiser,

Could you please upload your picture again?

Also what will happens if there is no intent?

@Ghostvv this flow i was trying to tell you in my issue

I see, but this flow is included in the stories, right?

yes i have also mentioned these stories but in my case whenever fallback comes the mentioned stories fails to work