Out of scope story not triggered

I have two story:

- story: out_of_scope
  steps:
  - intent: nlu_fallback
  - action: utter_default
  - intent: nlu_fallback
  - action: utter_goodbye


- story: 2 times not repay
  steps:
    - intent: greet
    - action: utter_replyGreet
    - action: collection_form
    - active_loop: collection_form
    - intent: replyNo
    - action: utter_persuade1
    - action: collection_form
    - active_loop: collection_form
    - intent: replyNo
    - action: utter_persuade2
    - action: utter_goodbye

I get the sponse

stories:
- story: Story from Conversation ID af0a995bcbdd4ecdad18a4ea6bdacaae
  steps:
  - intent: greet
  - action: utter_replyGreet
  - action: collection_form
  - active_loop: collection_form
  - intent: replyNo
  - action: utter_persuade1
  - action: collection_form
  - intent: nlu_fallback  (out of scope)
  - action: action_default_fallback  (I expect **utter_default**)
  - intent: nlu_fallback
  - action: action_default_fallback  (I expect **utter_goodbye**)

can you help me ? thanks

Hi, action_default_fallback is always triggered when an action confidence is below the threshold. By default, it will send the response utter_default.

Since the action confidence is below the threshold, this means your model doesn’t know what to predict when it sees nlu_fallback. Normally, you would write a rule to cover behaviour for nlu_fallback like so:

rules:
- rule: Ask the user to rephrase whenever they send a message with low NLU confidence
  steps:
    - intent: nlu_fallback
    - action: utter_default

However, this will not work like the story above, because you seem to want to behave differently depending on whether the previous message was also nlu_fallback. You could handle this by writing a custom action that is triggered whenever nlu_fallback is detected. In this action, check whether this is the second time in a row. If not, use dispatcher to send utter_default, if so, use dispatcher to send utter_goodbye.

This might look something like below (DISCLAIMER: I did not test this code, there may be typos or other mistakes, this should just give you an idea of how a solution could look)

class ActionNLUFallback(Action):
    def name(self) -> Text:
        return "action_nlu_fallback"

    def run(
        self,
        dispatcher: CollectingDispatcher,
        tracker: Tracker,
        domain: Dict[Text, Any],
    ) -> List[EventType]:
        action_events = [e for e in tracker.events if e["event"] == "action"]
        if len(action_events) > 1 and action_events[-2].get("name") == "action_nlu_fallback":
              dispatcher.utter_message(template="utter_goodbye")
        else:
              dispatcher.utter_message(template="utter_default")

You will also need a rule like:

rules:
- rule: Perform nlu fallback action
  steps:
    - intent: nlu_fallback
    - action: action_nlu_fallbakck

And you will need to add the FallbackClassifier to your config:

pipeline:
# other components
- name: FallbackClassifier
  threshold: 0.7

Thank you for your reply. I will to try.

For the FallbackClassifier, I have other problems. When I add FallbackClassifier to config, then run rasa train. Will be an error

InvalidConfigException: The pipeline configuration contains errors. The component 'FallbackClassifier' requires 'IntentClassifier' to be placed before it in the pipeline. Please add the required components to the pipeline.

thanks

Hi! You need an IntentClassifier in the pipeline before the `FallbackClassifier:

Try adding:

pipeline:
# Featurizers and Tokenizers here
- DIETClassifier
- name: FallbackClassifier
  threshold: 0.7

You’ll also need other components in the pipeline, and some policies for Rasa core. If you’d like, I can take a look at your current config and make suggestions.

1 Like