Fallback issues and policy advice

Rasa Open Source
1

I recently started having an issue where the default fallback intent will be triggered and the bot just repeats the default utterance, even when I use statements that are clearly in the NLU. The fallback seems to be more easily triggered, too.

I’ve pasted my Config.yml below. I’ve read the fallback policy documents and options for different AugmentedMemoizationPolicy, but i’m still pretty confused.

My bot should be designed for long conversations and go between different story types. The most common intent for my bot is inform.

I’m open to using a two-stage fallback policy, but confess that I don’t understand how this could help my particular situation very well.

cc @SamS

Open to advice on how to improve it. thank you!

# Configuration for Rasa NLU.
# https://rasa.com/docs/rasa/nlu/components/
language: en
pipeline:
# # No configuration for the NLU pipeline was provided. The following default pipeline was used to train your model.
# # If you'd like to customize it, uncomment and adjust the pipeline.
# # See https://rasa.com/docs/rasa/tuning-your-model for more information.n
   - name: WhitespaceTokenizer
   - name: RegexFeaturizer
   - name: LexicalSyntacticFeaturizer
   - name: CountVectorsFeaturizer
   - name: CountVectorsFeaturizer
 analyzer: char_wb
 min_ngram: 1
 max_ngram: 4
   - name: DIETClassifier
 epochs: 10
 constrain_similarities: true
   - name: EntitySynonymMapper
   - name: ResponseSelector
 epochs: 10
 constrain_similarities: true
   - name: FallbackClassifier
 threshold: 0.05
 ambiguity_threshold: 0.05
policies:
# # No configuration for policies was provided. The following default policies were used to train your model.
# # If you'd like to customize them, uncomment and adjust the policies.
# # See https://rasa.com/docs/rasa/policies for more information.
   - name: MemoizationPolicy
   - name: TEDPolicy
 max_history: 5
 epochs: 10
 constrain_similarities: true
   - name: RulePolicy
   - name: RulePolicy
  # Confidence threshold for the `core_fallback_action_name` to apply.
  # The action will apply if no other action was predicted with
  # a confidence >= core_fallback_threshold
 core_fallback_threshold: 0.05
 core_fallback_action_name: "action_default_fallback"
 enable_fallback_prediction: True
2

@travelmail26 Heya! Can you tell why threshold is 0.05?

Question: Above policy is working fine with the bot/user conversation? But sometime when you update the code, train it. and run it; it start triggering you default fallback? Right.

@travelmail26 ? Yes or No?

3

I wanted the fallback to have a low threshold and figured maybe this would fix the issue, but maybe i’m wrong.

4

@travelmail26 This is fine for your use:

# Configuration for Rasa NLU.
# https://rasa.com/docs/rasa/nlu/components/
language: en
pipeline:
  - name: WhitespaceTokenizer
  - name: RegexFeaturizer
  - name: LexicalSyntacticFeaturizer
  - name: CountVectorsFeaturizer
  - name: CountVectorsFeaturizer
    analyzer: "char_wb"
    min_ngram: 1
    max_ngram: 4
  - name: DIETClassifier
    epochs: 100
    ranking_length: 5
    constrain_similarities: true
    model_confidence: linear_norm
  - name: EntitySynonymMapper
  - name: ResponseSelector
    epochs: 100
    constrain_similarities: true
    model_confidence: linear_norm

# Configuration for Rasa Core.
# https://rasa.com/docs/rasa/core/policies/
policies:
  - name: MemoizationPolicy
  - name: TEDPolicy
    max_history: 5
    epochs: 100
  - name: RulePolicy
    nlu_threshold: 0.4
    core_threshold: 0.4
    fallback_action_name: "action_default_fallback"

@travelmail26 good luck with your error.

1 Like