Combining Keyword classification with DIETClassifier and Fallback

Rasa Open Source
1

Hey There

I have an issue regarding the usage of multiple intent classifiers. My goal: Whenever a keyword is detected (from the nlu samples), I want the NLU to predict the accoring intent. Whenever no keyword is detected, I want the DIETClassifier to predict the intent based on the nlu data. If the confidence of the DIETClassifier is low, I want the FallbackClassifier to trigger.

My current pipeline looks like this:

  - name: WhitespaceTokenizer
  - name: RegexFeaturizer
  - name: LexicalSyntacticFeaturizer
  - name: CountVectorsFeaturizer
  - name: CountVectorsFeaturizer
    analyzer: char_wb
    min_ngram: 1
    max_ngram: 4
  - name: DIETClassifier
    entity_recognition: false
    epochs: 100
  - name: KeywordIntentClassifier
    case_sensitive: False
  - name: EntitySynonymMapper
  - name: ResponseSelector
    epochs: 100
  - name: FallbackClassifier
    threshold: 0.3
    ambiguity_threshold: 0.1

In the NLU data, I for example have the intent “greet” where one of the example sentences is “hey” If I type in “hey”, greet is correctly predicted. If I, however, type “heyy”, the prediction is the following: image

What surprises me is the fact that greet has a confidence of 97% but the actual prediction is fallback. What exactly am I missing here? Is the ordering of my components off?

Thank you in advance!