Intent other than nlu_falback, triggering action_custom_fallback

Rasa Open Source
1

I have defined a rule that nlu_fallback intent should trigger action_custom_fallback. Nowhere else in my stories and rules have I specified that some other intent should also trigger the custom_fallback action. Still for some reason, many times an intent other than nlu_fallback triggers the action_custom_fallback, when some distance is traversed well into the conversation with the bot.

I tried varying the fallback threshold from 0.3 to 0.5 and 0.3 back again. Still no avail.

Here is my config.yml

# Configuration for Rasa NLU.
# https://rasa.com/docs/rasa/nlu/components/
language: en

pipeline:
# # No configuration for the NLU pipeline was provided. The following default pipeline was used to train your model.
# # If you'd like to customize it, uncomment and adjust the pipeline.
# # See https://rasa.com/docs/rasa/tuning-your-model for more information.
  - name: WhitespaceTokenizer
  - name: RegexEntityExtractor
  - name: LexicalSyntacticFeaturizer
  - name: CountVectorsFeaturizer
    analyzer: char_wb
    min_ngram: 1
    max_ngram: 4
  - name: DIETClassifier
    epochs: 100
    constrain_similarities: true
    random_seed: 1
  - name: EntitySynonymMapper
  - name: ResponseSelector
    epochs: 100
    constrain_similarities: true
    random_seed: 1
  # - name: DucklingHTTPExtractor
  #   url: http://0.0.0.0:8000
  #   dimensions: ["account_number","number","mobile_number"]  
  #   timezone: "India/Kolkata"
  #   timeout: 3
  - name: FallbackClassifier
    threshold: 0.3
  #   ambiguity_threshold: 0.1
  # - name: SpacyNLP
  #   model: "en_core_web_sm"
  # - name: SpacyTokenizer
  # - name: SpacyFeaturizer
  #   pooling: mean
  # - name: RegexFeaturizer
  # - name: CRFEntityExtractor
  # - name: regex.RegexEntityExtractor
  # - name: EntitySynonymMapper
  # - name: "SklearnIntentClassifier"

policies:
# # No configuration for policies was provided. The following default policies were used to train your model.
# # If you'd like to customize them, uncomment and adjust the policies.
# # See https://rasa.com/docs/rasa/policies for more information.
  - name: AugmentedMemoizationPolicy
  - name: TEDPolicy
    max_history: 5
    epochs: 100
    random_seed: 1
    # model_confidence: linear_norm
    constrain_similarities: true
  - name: RulePolicy
    core_fallback_threshold: 0.3
    core_fallback_action_name: "action_custom_fallback"
    enable_fallback_prediction: True

I also tried giving multiple rules for nlu_fallback intent with various slot conditions:

- rule: Ask the user to rephrase whenever they send a message with low NLU confidence | is_authenticated - False
  condition:
  - slot_was_set:
    - is_authenticated: False
  steps:
  - intent: nlu_fallback
  - action: action_custom_fallback

- rule: Ask the user to rephrase whenever they send a message with low NLU confidence | is_authenticated - True | is_activated - N
  condition:
  - slot_was_set:
    - is_authenticated: True
    - is_activated: N
  steps:
  - intent: nlu_fallback
  - action: action_custom_fallback

- rule: Ask the user to rephrase whenever they send a message with low NLU confidence | is_authenticated - True | is_activated - Y
  condition:
  - slot_was_set:
    - is_authenticated: True
    - is_activated: Y
  steps:
  - intent: nlu_fallback
  - action: action_custom_fallback

- rule: Ask the user to rephrase whenever they send a message with low NLU confidence | is_authenticated - True | is_activated - SUSPEND
  condition:
  - slot_was_set:
    - is_authenticated: True
    - is_activated: SUSPEND
  steps:
  - intent: nlu_fallback
  - action: action_custom_fallback

Even this doesn’t overcome the problem.

Also, it is difficult to predict when the bot will be triggering the action_custom_fallback, when any intent is recognised. This makes life hell for developers and testers alike.

The rasa model logs also don’t say much when this situation arises. Any help will be appreciated

2

Review the debug log to see what is happening with the nlu & dialog prediction.