Hi everyone. I thought I would share an extension that I made to enable context-specific fallback/error handling. Context-specific error handling is really good for things like:
B:"for how many people should I make the reservation?"
U:"oh you are a clever little bot aren't you!"
B:"Sorry, didn't get that. For how many people?"
U:"sooo clever you are (as yoda would say)."
B:"No, I still didn't understand that. Just let me know for how many people please"
U: "Do you know any Chuck Norris jokes?"
B:"Sorry, I don't think I can help you. Please call 666-6666 to talk with a person. Bye!"
Google has a cool link about this sort of “fallback escalation” here: https://designguidelines.withgoogle.com/conversation/conversational-components/errors.html#errors-no-match
How to do this with rasa? The FallbackPolicy is not enough, because you can only specify one fallback utterance/action which can come across robotic. “Sorry, I didn’t understand”,“Sorry, I didn’t understand”,“Sorry, I didn’t understand”,“Sorry, I didn’t understand” etc…
My approach was to define a “Fallback intent” which can then be used as an intent in story definitions. The idea is the nlu would classify anything it didn’t understand (below a certain confidence threshold) as the “fallback” intent and give this information to the core. In order to do this, I had to chime into the nlu pipeline. The first step was defining a “Fallback intent filter” or interceptor. The code looks like this:
class FallbackIntentFilter(Component):
""" der fallback intent Filter ermöglicht es uns, alle Nutzer-Eingaben, die nicht einem Intent eindeutig
eingeordnet werden können, in einen 'fallback intent' unterzubringen.
Dieser fallback intent kann dann in dialog training (stories.md) verwendet werden,
um Kontext-abhängige error-handling zu ermöglichen
"""
def __init__(self,component_config=None,threshold=0.5, fallback_intent="fallback"):
## threshold -- wenn intent Erkennung unter threshold, setzte fallback intent als erkannter Intent.
## TODO threshold auf sinnvolle Wert setzten beim error-handling (0.1 <> 0.5)
## mit sehr wenig intents, soll dieser wert relativ hoch gesetzt sein
## fallback_intent -- name von intent. Dieser Name wird in training (stories.md) verwendet
super().__init__(component_config)
self.fb_threshold=threshold
self.fallback_intent = fallback_intent
def process(self, message, **kwargs):
# type: (Message, **Any) -> None
if message.data['intent']['confidence'] < self.fb_threshold:
fb_intent = {'name':self.fallback_intent,'confidence':self.fb_threshold}
message.data['intent'] = fb_intent
message.data['intent_ranking'].insert(0,fb_intent)
The “threshold” parameter says anything below 0.5 confidence should be classified as “fallback” intent (fallback_intent parameter specifies the name of the intent)
During processing, if the intent classification confidence in the message is below the threshold, the recognized intent is replaced with the “fallback” intent.
To use this component, add it to the nlu config pipeline in nlu_config.yml (using the module path and class name)
pipeline:
(featurizers and classifiers etc...)
- name: "components.fallback_intent_filter.FallbackIntentFilter"
It should come after your classifiers, so you can override the intent classification. After this, you can use the “fallback” intent in your stories like this:
## fallback
start
- utter_greet
- utter_welcome_prompt
- utter_ask_specifics
* fallback
- utter_fallback
* fallback
- utter_escalate
* fallback
- utter_give_up
- utter_goodbye
You can use “fallback” just like any other intent, so you can tell your bot what to do when it sees a fallback intent in a very context-specific way, specifying different actions and utterances depending on the story context.
