NLU predicts nlu_fallback even the confidence score of some other intent is higher than the threshhold

Rasa Open Source
1

Hi, when I added the nlu_fallback to my rasa config, it didn’t worked as expectedly. NLU predicts nlu_fallback even when the confidence score of some other intent is higher than the threshhold. Can anyone help? Thanks!

the 2 test case as follows:

  1. the nlu result of the config without nlu_fallback: njUdY0dnby

  2. the nlu result of the config with nlu_fallback (threshold=0.7): 4LE3a1tQXP

the rasa version: 2.7.1

my rasa config:

language: zh
pipeline:
#- name: WhitespaceTokenizer
  - name: JiebaTokenizer
    intent_tokenization_flag: True
    intent_split_symbol: "+"
  - name: RegexFeaturizer
  - name: LexicalSyntacticFeaturizer
  - name: CountVectorsFeaturizer
  - name: CountVectorsFeaturizer
    analyzer: char_wb
    min_ngram: 1
    max_ngram: 4
  - name: DIETClassifier
    epochs: 100
    model_confidence: softmax
    constrain_similarities: true
  # - name: FallbackClassifier
  #   threshold: 0.7
  - name: EntitySynonymMapper
  - name: ResponseSelector
    epochs: 100
    constrain_similarities: true
  - name: ResponseSelector
    epochs: 100
    retrieval_intent: faq
  - name: ResponseSelector
    epochs: 100
    retrieval_intent: chitchat
# Configuration for Rasa Core.
# https://rasa.com/docs/rasa/core/policies/
policies:
  - name: MemoizationPolicy
  - name: TEDPolicy
    max_history: 5
    epochs: 100
    constrain_similarities: true
  - name: RulePolicy
1 Like
2

Is there anyone meeting this problem? All help and suggestions would really be appreiciated.