Fallback help

Rasa Open Source
1

Hello, after rasa init, i added the following to my yml.

domainy.yml:

  utter_fallback:
    - text: "I am sorry, I do not understand that."

and

rules.yml

  - rule: Say Fallback message with uncertain NLU
    steps:
      - intent: nlu_fallback
      - action: utter_fallback

the training data is default data.the config im using is the default config ( all lines are commented out). After that , i use rasa shell and input random text like 1+2+4+2 or sadsadsadsa. but the chatbot does not utter_fallback. why ?

2 
Rasa Version      :         3.6.20
Minimum Compatible Version: 3.5.0  
Rasa SDK Version  :         3.6.2  
Python Version    :         3.10.10
Operating System  :         Windows-10-10.0.19044-SP0

This is my current data:

domain.yml

version: "3.1"

intents:
  - introduce
  - ask_entry_requirements
  - say_nationality

entities:
  - nationality

slots:
  nationality:
    type: text
    mappings:
      - type: from_entity
        entity: nationality

responses:
  utter_introduce:
    - text: "Hey, How can i help you today? You can select one of the topics below:"
      buttons:
        - title: "Entry Requirements"
        - title: "Fees"
        - title: "Schedule"
        - title: "Procedure"
        - title: "Program Details"

  utter_ask_nationality:
    - text: "Are you a Malaysian or International applicant?"

  utter_entry_requirements:
    - text: "The minimum entry requirements for {nationality} are as follows...."

actions:
  - action_fallback_and_store_message

session_config:
  session_expiration_time: 60
  carry_over_slots_to_new_session: true

nlu.yml

version: "3.1"

nlu:
  - intent: introduce
    examples: |
      - hey
      - hello
      - hi
      - hello there
      - good morning
      - good evening
      - hey there
      - let's go
      - hey dude
      - goodmorning
      - goodevening
      - good afternoon

  - intent: ask_entry_requirements
    examples: |
      - entry requirements
      - minimum entry requirements
      - admission requirements
      - minimum admission requirements
      - what is the entry requirements?
      - wat is the entry requirements?
      - what's the entry requirements?
      - the minimum entry requirements are?
      - the minimum entry requirement is ?
      - what is the min entry requirements?
      - wat is the min admission requirements?

  - intent: say_nationality
    examples: |
      - [malaysian](nationality)
      - [malaysian](nationality) student
      - [malaysian](nationality) applicant
      - I am a [malaysian](nationality)
      - am a [malaysian](nationality) student
      - a [malaysian](nationality) applicant
      - Im [malaysian](nationality)
      - Im a [malaysian](nationality) applicant
      - I'm [malaysian](nationality)
      - [international](nationality)
      - [international](nationality) student
      - [international](nationality) applicant
      - I am an [international](nationality) student
      - I am an [international](nationality) applicant
      - am an [international](nationality) student
      - Im an [international](nationality) student
      - an [international](nationality) applicant

  - synonym: malaysian
    examples: |
      - local
      - malaysia
      - m'sian

stories.yml

version: "3.1"

stories:
  - story: introduction
    steps:
      - intent: introduce
      - action: utter_introduce

  - story: Ask entry requirements for malaysian applicant
    steps:
      - intent: ask_entry_requirements
      - action: utter_ask_nationality
      - intent: say_nationality
      - slot_was_set:
          - nationality: "malaysian"
      - action: utter_entry_requirements

rules.yml

version: "3.1"

rules:
  - rule: Say Fallback message with uncertain NLU
    steps:
      - intent: nlu_fallback
      - action: action_fallback_and_store_message

config.yml

pipeline:
  - name: WhitespaceTokenizer
  - name: RegexFeaturizer
  - name: LexicalSyntacticFeaturizer
  - name: CountVectorsFeaturizer
  - name: CountVectorsFeaturizer
    analyzer: char_wb
    min_ngram: 1
    max_ngram: 4
  - name: DIETClassifier
    epochs: 100
    constrain_similarities: true
  - name: EntitySynonymMapper
  - name: ResponseSelector
    epochs: 100
    constrain_similarities: true
  - name: FallbackClassifier
    threshold: 0.7
    ambiguity_threshold: 0.1

# Configuration for Rasa Core.
# https://rasa.com/docs/rasa/core/policies/
policies:
  - name: MemoizationPolicy
  - name: RulePolicy
  - name: UnexpecTEDIntentPolicy
    max_history: 5
    epochs: 100
  - name: TEDPolicy
    max_history: 5
    epochs: 100
    constrain_similarities: true

using rasa shell --debug these are some logs i get

2024-05-27 00:44:22 DEBUG rasa.core.processor - [debug ] processor.message.parse parse_data_entities= parse_data_intent={‘name’: ‘introduce’, ‘confidence’: 0.999078631401062} parse_data_text=dsadsadwdw2232

2024-05-27 00:46:26 DEBUG rasa.core.processor - [debug ] processor.message.parse parse_data_entities= parse_data_intent={‘name’: ‘say_nationality’, ‘confidence’: 0.8710447549819946} parse_data_text=call you tell me my name?

2024-05-27 00:47:10 DEBUG rasa.core.processor - [debug ] processor.message.parse parse_data_entities= parse_data_intent={‘name’: ‘ask_entry_requirements’, ‘confidence’: 0.9983372688293457} parse_data_text=who is the us president?

As you can see the confidence level for random text that i dont have in my nlu is very high causing nlu_fallback to not be trigger. why is this happening ? how can fix this ? I want my nlu_fallback to be properly triggered when user input text that are not in the nlu such as random words or numbers