Out of scope story not triggered

Rasa Open Source
1

I have two story:

- story: out_of_scope
  steps:
  - intent: nlu_fallback
  - action: utter_default
  - intent: nlu_fallback
  - action: utter_goodbye


- story: 2 times not repay
  steps:
    - intent: greet
    - action: utter_replyGreet
    - action: collection_form
    - active_loop: collection_form
    - intent: replyNo
    - action: utter_persuade1
    - action: collection_form
    - active_loop: collection_form
    - intent: replyNo
    - action: utter_persuade2
    - action: utter_goodbye

I get the sponse

stories:
- story: Story from Conversation ID af0a995bcbdd4ecdad18a4ea6bdacaae
  steps:
  - intent: greet
  - action: utter_replyGreet
  - action: collection_form
  - active_loop: collection_form
  - intent: replyNo
  - action: utter_persuade1
  - action: collection_form
  - intent: nlu_fallback  (out of scope)
  - action: action_default_fallback  (I expect **utter_default**)
  - intent: nlu_fallback
  - action: action_default_fallback  (I expect **utter_goodbye**)

can you help me ? thanks

2

Hi, action_default_fallback is always triggered when an action confidence is below the threshold. By default, it will send the response utter_default.

Since the action confidence is below the threshold, this means your model doesn’t know what to predict when it sees nlu_fallback. Normally, you would write a rule to cover behaviour for nlu_fallback like so:

rules:
- rule: Ask the user to rephrase whenever they send a message with low NLU confidence
  steps:
    - intent: nlu_fallback
    - action: utter_default

However, this will not work like the story above, because you seem to want to behave differently depending on whether the previous message was also nlu_fallback. You could handle this by writing a custom action that is triggered whenever nlu_fallback is detected. In this action, check whether this is the second time in a row. If not, use dispatcher to send utter_default, if so, use dispatcher to send utter_goodbye.

This might look something like below (DISCLAIMER: I did not test this code, there may be typos or other mistakes, this should just give you an idea of how a solution could look)

class ActionNLUFallback(Action):
    def name(self) -> Text:
        return "action_nlu_fallback"

    def run(
        self,
        dispatcher: CollectingDispatcher,
        tracker: Tracker,
        domain: Dict[Text, Any],
    ) -> List[EventType]:
        action_events = [e for e in tracker.events if e["event"] == "action"]
        if len(action_events) > 1 and action_events[-2].get("name") == "action_nlu_fallback":
              dispatcher.utter_message(template="utter_goodbye")
        else:
              dispatcher.utter_message(template="utter_default")

You will also need a rule like:

rules:
- rule: Perform nlu fallback action
  steps:
    - intent: nlu_fallback
    - action: action_nlu_fallbakck

And you will need to add the FallbackClassifier to your config:

pipeline:
# other components
- name: FallbackClassifier
  threshold: 0.7
3

Thank you for your reply. I will to try.

For the FallbackClassifier, I have other problems. When I add FallbackClassifier to config, then run rasa train. Will be an error

InvalidConfigException: The pipeline configuration contains errors. The component 'FallbackClassifier' requires 'IntentClassifier' to be placed before it in the pipeline. Please add the required components to the pipeline.

thanks

4

Hi! You need an IntentClassifier in the pipeline before the `FallbackClassifier:

Try adding:

pipeline:
# Featurizers and Tokenizers here
- DIETClassifier
- name: FallbackClassifier
  threshold: 0.7

You’ll also need other components in the pipeline, and some policies for Rasa core. If you’d like, I can take a look at your current config and make suggestions.

2 Likes