I added few intents and chitchat in my training data and trained the bot. When I started testing the bot, other than the defined intents in nlu.yml file, any other input whether unknown or gibberish or some random words will get predicted as chitchat intent with confidence as 1.0 or 0.99. I even added two stage fall back logic in config.yml, rules.yml and custom actions. For fallback to happen, prediction confidence has to be lower than 1.0 or 0.99.
Hi, could you please share your Rasa version and YAML files (config.yml, domain.yml, nlu.yml, rules.yml) that lead to the issue?
Please post the actual code/logs/stacktrace text into the channel instead of the screenshots. That way we can more easily copy/paste the code and help you debug. You can use markdown syntax ``` to create code blocks.
version: "2.0"
nlu:
- intent: greet
examples: |
- Hi
- Hello
- Hi
- hello
- hi
- heeeeeeellllllllooooooooooooo
- hai
- Hai
- Hi How are you?
- how are you?
- how are you
- How are you?
- How are you
- I am doing good, how are you?
- I am doing good how are you?
- I am doing good how are you
- I am doing good, How are you?
version: "2.0"
rules:
- rule: to greet the customer
steps:
- intent: greet
- action: utter_greet
- rule: Implemtation of Two Stage FallBack
steps:
- intent: nlu_fallback
- action: action_two_stage_fallback
- active_loop: action_two_stage_fallback
Domain.yml
version: '2.0'
config:
store_entities_as_slots: true
session_config:
session_expiration_time: 480
carry_over_slots_to_new_session: true
intents:
- affirm
- greet
- ask_how_contribute
- ask_question_in_forum
- ask_when_next_event
- ask_which_events
- ask_why_contribute
- book_demo
- broken
- bye
- canthelp
- chitchat:
use_entities: []
- contact_sales
- deny
- enter_data:
use_entities: []
- explain
- faq:
use_entities:
- product
- nlu_fallback
- trigger_response_selector:
use_entities: false
responses:
utter_greet:
- text: "Hi, How are you doing?"
- text: "Its good to hear from you"
- text: "Always looking forward to say you a Hi"
utter_ask_feedback:
- buttons:
- payload: '/feedback{{"feedback_value": "positive"}}'
title: 👍
- payload: '/feedback{{"feedback_value": "negative"}}'
title: 👎
text: How is this conversation going?
utter_ask_feedback_message:
- text: I'm sorry I couldn't help you. Please let me know if you have any suggestions for how I can improve.
utter_ask_for_nlu_specifics:
- text: Anything in particular you'd like to know about, e.g. intent classification or entity recognition?
utter_ask_goal:
- text: If you're new to Rasa, I can help you get started! If you've already started building an assistant, you can also ask me about the different parts of Rasa 🐦
utter_ask_if_problem:
- text: Did you encounter any problems during installation?
utter_ask_job_function:
- text: What's your job? 🕴
utter_ask_x_local_server:
- text: Are you trying to install Rasa X on your local machine or on a server?
utter_ask_migration:
- text: Are you migrating from another tool?
utter_ask_name:
- text: What's your name?
utter_ask_person_name:
- text: What's your name?
actions:
- action_default_ask_affirmation
- utter_chitchat
- utter_faq
- utter_out_of_scope
- utter_greet
I am using rasa V2.8.10. I wanted to start with a only one intent in the bot as there are more work in the custom actions at this point of the project. Basically this is for MVP.
Thanks for sharing the files. Is this your complete nlu.yml or did something accidentally get cut when copying it? It only contains training examples for one intent (greet), which for me leads to an error when training the pipeline:
2021-11-25 11:32:53 ERROR rasa.nlu.classifiers.diet_classifier - Cannot train 'DIETClassifier'. Need at least 2 different intent classes. Skipping training of classifier.
Hi, I tried training with the chitchat.yml data and the nlu.yml you provided and saw a similar behaviour. The classifier looks very skewed towards the chitchat class. The reason could be that the training data contains far more chitchat examples, which cover a very diverse set of topics and the examples for greet are partly overlapping with the chitchat/ask_howdoing section. So anything that is not clearly greet will be classified as chitchat with high confidence.
This skew in performance can also be seen in the test output, running
rasa test nlu --nlu data/nlu/ --cross-validation --runs 1 --folds 2
