I am working on Rasa 2.4. The use case is a chatbot that is handling intents and sub intents. I’ve removed the default rasa stories, responses and intents. Yet the system is classifying it as mood unhappy sometimes. I’ve tried working on different systems and deleted all temp files yet the issue persists. Please find the output, nlu, config etc below.
I need help removing those intents from the system and instead triggering fallback.
Output:
Your input> → Hello Hi. I am Sevak at your service. May I know your name please? Your input → Rahul How may I help you today Rahul? Your input → I want food I’m sorry, I didn’t quite understand that. Could you rephrase? Your input → what is cdc? Change Data Capture; It is a tool by IBM. Replicate data from source to target and vice-versa. Your input → what is nested jobplan Jobplan ID which is mapped in Nested Jobplan ID field against the task, which has to be performed by the user. Your input → what colour dress are you wearing? > d:\anaconda\envs\sevakiaf\lib\site-packages\rasa\shared\utils\io.py:96: UserWarning: Interpreter parsed an intent ‘mood_unhappy’ which is not defined in the domain. Please make sure all intents are listed in the domain. More info at Domain Bye, Have a nice day
Output: <rasa shell --debug>
> Your input -> how is the weather today?
> 2021-04-27 13:19:38 DEBUG rasa.core.lock_store - Issuing ticket for conversation 'fa39d6f36a2e4101b8633a12ba6a3adb'.
> 2021-04-27 13:19:38 DEBUG rasa.core.lock_store - Acquiring lock for conversation 'fa39d6f36a2e4101b8633a12ba6a3adb'.
> 2021-04-27 13:19:38 DEBUG rasa.core.lock_store - Acquired lock for conversation 'fa39d6f36a2e4101b8633a12ba6a3adb'.
> 2021-04-27 13:19:38 DEBUG rasa.core.tracker_store - Recreating tracker for id 'fa39d6f36a2e4101b8633a12ba6a3adb'
> 2021-04-27 13:19:38 DEBUG rasa.nlu.selectors.response_selector - Adding following selector key to message property: faq
> 2021-04-27 13:19:38 DEBUG ***rasa.core.processor - Received user message 'how is the weather today?' with intent '{'id': 1248384363710024904, 'name': 'mood_unhappy', 'confidence': 0.30179139971733093}' and entities '[]'***
> d:\anaconda\envs\sevakiaf\lib\site-packages\rasa\shared\utils\io.py:96: UserWarning: Interpreter parsed an intent 'mood_unhappy' which is not defined in the domain. Please make sure all intents are listed in the domain.
> More info at https://rasa.com/docs/rasa/domain
> 2021-04-27 13:19:38 DEBUG rasa.core.processor - Logged UserUtterance - tracker now has 14 events.
> 2021-04-27 13:19:38 DEBUG rasa.core.policies.memoization - Current tracker state:
> [state 0] user intent: faq | previous action name: action_listen
> [state 1] user intent: faq | previous action name: utter_faq
> [state 2] user intent: deny | previous action name: action_listen
> [state 3] user intent: deny | previous action name: utter_goodbye
> [state 4] user intent: mood_unhappy | previous action name: action_listen
> 2021-04-27 13:19:38 DEBUG rasa.core.policies.memoization - There is no memorised next action
> 2021-04-27 13:19:38 DEBUG rasa.core.policies.rule_policy - Current tracker state:
> [state 1] user intent: faq | previous action name: action_listen
> [state 2] user intent: faq | previous action name: utter_faq
> [state 3] user intent: deny | previous action name: action_listen
> [state 4] user intent: deny | previous action name: utter_goodbye
> [state 5] user text: how is the weather today? | previous action name: action_listen
> 2021-04-27 13:19:38 DEBUG rasa.core.policies.rule_policy - There is no applicable rule.
> 2021-04-27 13:19:38 DEBUG rasa.core.policies.rule_policy - Current tracker state:
> [state 1] user intent: faq | previous action name: action_listen
> [state 2] user intent: faq | previous action name: utter_faq
> [state 3] user intent: deny | previous action name: action_listen
> [state 4] user intent: deny | previous action name: utter_goodbye
> [state 5] user intent: mood_unhappy | previous action name: action_listen
> 2021-04-27 13:19:38 DEBUG rasa.core.policies.rule_policy - There is no applicable rule.
> 2021-04-27 13:19:38 DEBUG rasa.core.policies.ensemble - Made prediction using user intent.
> 2021-04-27 13:19:38 DEBUG rasa.core.policies.ensemble - Added `DefinePrevUserUtteredFeaturization(False)` event.
> 2021-04-27 13:19:38 DEBUG rasa.core.policies.ensemble - Predicted next action using policy_1_TEDPolicy.
> 2021-04-27 13:19:38 DEBUG rasa.core.processor - Predicted next action 'utter_goodbye' with confidence 0.61.
> 2021-04-27 13:19:38 DEBUG rasa.core.processor - Policy prediction ended with events '[<rasa.shared.core.events.DefinePrevUserUtteredFeaturization object at 0x000001FD0894C948>]'.
> 2021-04-27 13:19:38 DEBUG rasa.core.processor - Action 'utter_goodbye' ended with events '[BotUttered('Bye, Have a nice day', {"elements": null, "quick_replies": null, "buttons": null, "attachment": null, "image": null, "custom": null}, {"utter_action": "utter_goodbye"}, 1619509778.8535676)]'.
> 2021-04-27 13:19:38 DEBUG rasa.core.policies.memoization - Current tracker state:
> [state 0] user intent: faq | previous action name: utter_faq
> [state 1] user intent: deny | previous action name: action_listen
> [state 2] user intent: deny | previous action name: utter_goodbye
> [state 3] user intent: mood_unhappy | previous action name: action_listen
> [state 4] user intent: mood_unhappy | previous action name: utter_goodbye
> 2021-04-27 13:19:38 DEBUG rasa.core.policies.memoization - There is no memorised next action
> 2021-04-27 13:19:38 DEBUG rasa.core.policies.rule_policy - Current tracker state:
> [state 1] user intent: faq | previous action name: action_listen
> [state 2] user intent: faq | previous action name: utter_faq
> [state 3] user intent: deny | previous action name: action_listen
> [state 4] user intent: deny | previous action name: utter_goodbye
> [state 5] user intent: mood_unhappy | previous action name: action_listen
> [state 6] user intent: mood_unhappy | previous action name: utter_goodbye
> 2021-04-27 13:19:38 DEBUG rasa.core.policies.rule_policy - There is no applicable rule.
> 2021-04-27 13:19:38 DEBUG rasa.core.policies.ensemble - Predicted next action using policy_1_TEDPolicy.
> 2021-04-27 13:19:38 DEBUG rasa.core.processor - Predicted next action 'action_listen' with confidence 0.99.
> 2021-04-27 13:19:38 DEBUG rasa.core.processor - Policy prediction ended with events '[]'.
> 2021-04-27 13:19:38 DEBUG rasa.core.processor - Action 'action_listen' ended with events '[]'.
> 2021-04-27 13:19:38 DEBUG rasa.core.lock_store - Deleted lock for conversation 'fa39d6f36a2e4101b8633a12ba6a3adb'.
> Bye, Have a nice day
Domain:
> intents:
>
> - greet
>
> - goodbye
>
> - affirm
>
> - deny
>
> - bot_challenge
>
> - got_solution
>
> - help_categories
>
> - introduction:
>
> use_entities:
>
> - name
>
> - faq
>
> entities:
>
> - name
>
> slots:
>
> name:
>
> type: text
>
> # this slot will not influence the predictions
>
> # of the dialogue policies
>
> influence_conversation: false
>
> actions:
>
> - utter_greet
>
> - utter_how_can_i_help
>
> - utter_how_can_i_help_no_name
>
> - utter_please_rephrase
>
> - utter_did_that_help
>
> - utter_can_i_do_anything_more
>
> - utter_ask_query
>
> - utter_goodbye
>
> - utter_iamabot
>
> - utter_introduction
>
> - utter_not_got_solution
>
>
>
> forms:
>
> intro_form:
>
> name:
>
> - type: from_entity
>
> entity: name
>
> intent: introduction
>
>
>
> responses:
>
> utter_default:
>
> - text: "Sorry I didn't get that. Can you put it in different words?"
>
> utter_greet:
>
> - text: "Hi. I am Sevak at your service. May I know your name please?"
>
> # How can I help
>
> utter_how_can_i_help:
>
> - text: "How may I help you today {name}?"
>
> utter_how_can_i_help_no_name:
>
> - text: "Very well. How may I help you today?"
>
> utter_please_rephrase:
>
> - text: I'm sorry, I didn't quite understand that. Could you rephrase?
>
>
>
> utter_did_that_help:
>
> - text: "Did that help you?"
>
> # Anything more to assist
>
> utter_can_i_do_anything_more:
>
> - text: Can I help you with something else?
>
> - text: Is there any other query that I can help you with?
>
> utter_ask_query:
>
> - text: "Please, state your query"
>
> - text: "Please, state your question"
>
> utter_goodbye:
>
> - text: "Bye, Have a nice day"
>
> utter_iamabot:
>
> - text: "I am a bot, powered by Rasa."
>
>
>
> utter_help_categories:
>
> - text: "I can help you in over hundreds of queries. They can be related to Issues & Returns, Common Data Management, Item information, tools, assets, DC RDC queries, extension and anticipation, work order concession, flight log book, dr/pwr, jobplan, snags, reports, model (cm), weekly training plans, preventive maintainance among others."
>
> # greeting and progress to the person's purpose
>
> utter_introduction:
>
> - text: "Pleased to be acquainted with you {name}."
>
> utter_not_got_solution:
>
> - text: "I am sorry, that is the extent of my knowledge. If you need more assistance please contact a human representative."
>
> + around 220 sub intents responses of faq
nlu:
version: “2.0”
nlu:
intent: greet
examples: |
hey
hello
hi
hello there
good morning
good evening
moin
hey there
let’s go
hey dude
goodmorning
goodevening
good afternoon
intent: goodbye
examples: |
good afternoon
cu
good by
cee you later
good night
bye
goodbye
have a nice day
see you around
bye bye
see you later
intent: affirm
examples: |
yes
y
indeed
of course
that sounds good
correct
intent: deny
examples: |
no
n
never
I don’t think so
don’t like that
no way
not really
i would rather not disclose
not really
intent: bot_challenge
examples: |
are you a bot?
are you a human?
am I talking to a bot?
am I talking to a human?
intent: got_solution
examples: |
Sure… Got it.
I understand.
Thanks.
Ok
okay thanks
cool
that was helpful
intent: not_got_solution
examples: |
That does not answer my question.
It is still not clear
I don’t understand
Can you elaborate more?
What do you mean by that?
intent: help_categories
examples: |
Please help me understand, what all can you tell me about?
What are the things I can ask you?
Can you tell me what topics you are informed about?
What are the ways in which you can help me?
How can you help me?
intent: introduction
examples: |
- around 220 sub intents of FAQ
config: > # Configuration for Rasa NLU.
> > # https://rasa.com/docs/rasa/nlu/components/ > > language: en > > pipeline: > > # No configuration for the NLU pipeline was provided. The following default pipeline was used to train your model. > > # If you'd like to customize it, uncomment and adjust the pipeline. > > # See https://rasa.com/docs/rasa/tuning-your-model for more information. > > - name: WhitespaceTokenizer > > # - name: RegexFeaturizer > > # - name: LexicalSyntacticFeaturizer > > - name: LanguageModelFeaturizer > > model_name: "bert" > > model_weights: "rasa/LaBSE" > > cache_dir: null > > # - name: CountVectorsFeaturizer > > # - name: CountVectorsFeaturizer > > # analyzer: char_wb > > # min_ngram: 1 > > # max_ngram: 4 > > - name: DIETClassifier > > epochs: 150 > > constrain_similarities: true > > - name: EntitySynonymMapper > > - name: ResponseSelector > > epochs: 100 > > retrieval_intent: faq > > constrain_similarities: true > > - name: FallbackClassifier > > threshold: 0.3 > > ambiguity_threshold: 0.3 > > # Configuration for Rasa Core. > > # https://rasa.com/docs/rasa/core/policies/ > > policies: > > # No configuration for policies was provided. The following default policies were used to train your model. > > # If you'd like to customize them, uncomment and adjust the policies. > > # See https://rasa.com/docs/rasa/policies for more information. > > - name: MemoizationPolicy > > - name: TEDPolicy > > max_history: 5 > > epochs: 150 > > constrain_similarities: true > > - name: RulePolicy > > core_fallback_threshold: 0.25 > > core_fallback_action_name: "action_default_fallback" > > enable_fallback_prediction: True
rules
> > version: "2.0" > > rules: > > # - rule: Say goodbye anytime the user says goodbye > > # steps: > > # - intent: goodbye > > # - action: utter_goodbye > > - rule: Say 'I am a bot' anytime the user challenges > > steps: > > - intent: bot_challenge > > - action: utter_iamabot > > - rule: what are the things you can help me with > > steps: > > - intent: help_categories > > - action: utter_help_categories > > # - rule: respond to FAQs > > # steps: > > # - intent: faq > > # - action: utter_faq > > # - rule: when someone tells their name > > # steps: > > # - intent: introduction > > # - action: intro_form > > # - action: utter_how_can_i_help > > - rule: Ask the user to rephrase whenever they send a message with low NLU confidence > > steps: > > - intent: nlu_fallback > > - action: utter_please_rephrase > > # some other fallback option > > # additional rule in intent
stories:
stories version: "2.0"
story: faq with chit chat
steps:
intent: greet
action: utter_greet
intent: introduction
entities:
- name : “Douglas”
action: intro_form
action: utter_how_can_i_help
intent: faq
action: utter_faq
intent: got_solution
action: utter_can_i_do_anything_more
intent: goodbye
action: utter_goodbye
story: faq with chit chat name deny
steps:
intent: greet
action: utter_greet
intent: deny
action: utter_how_can_i_help_no_name
intent: faq
action: utter_faq
intent: got_solution
action: utter_can_i_do_anything_more
intent: deny
action: utter_goodbye
story: faq with chit chat name deny not got solution
steps:
intent: greet
action: utter_greet
intent: deny
action: utter_how_can_i_help_no_name
intent: faq
action: utter_faq
intent: not_got_solution
action: utter_not_got_solution
action: utter_can_i_do_anything_more
intent: deny
action: utter_goodbye
story: faq without greet
steps:
intent: introduction
entities:
- name : “Douglas”
action: intro_form
action: utter_how_can_i_help
intent: faq
action: utter_faq
intent: got_solution
action: utter_can_i_do_anything_more
intent: goodbye
action: utter_goodbye
story: faq without greet intent affirm
steps:
intent: introduction
entities:
- name : “Douglas”
action: intro_form
action: utter_how_can_i_help
intent: faq
action: utter_faq
intent: got_solution
action: utter_can_i_do_anything_more
intent: affirm
intent: faq
action: utter_faq
intent: goodbye
action: utter_goodbye
story: faq without name
steps:
intent: greet
action: utter_greet
intent: faq
action: utter_faq
intent: got_solution
action: utter_can_i_do_anything_more
intent: affirm
intent: goodbye
action: utter_goodbye
story: faq without name and utter_can_i_do_anything_more
steps:
intent: greet
action: utter_greet
intent: faq
action: utter_faq
intent: goodbye
action: utter_goodbye
story: faq without utter_can_i_do_anything_more
steps:
intent: greet
action: utter_greet
intent: introduction
entities:
- name : “Douglas”
action: intro_form
action: utter_how_can_i_help
intent: faq
action: utter_faq
intent: goodbye
action: utter_goodbye
story: faq with introduction
steps:
intent: greet
action: utter_greet
intent: introduction
entities:
- name : “Douglas”
action: intro_form
action: utter_how_can_i_help
story: faq without bye
steps:
intent: greet
action: utter_greet
intent: introduction
entities:
- name : “Douglas”
action: intro_form
action: utter_how_can_i_help
intent: faq
action: utter_faq
intent: got_solution
action: utter_can_i_do_anything_more
story: faq with introduction only
steps:
intent: introduction
entities:
- name : “Douglas”
action: intro_form
action: utter_how_can_i_help
story: faq with introduction and bye
steps:
intent: introduction
entities:
- name : “Douglas”
action: intro_form
action: utter_how_can_i_help
intent: faq
action: utter_faq
intent: goodbye
action: utter_goodbye
story: faq with got_solution and affirm
steps:
intent: got_solution
action: utter_can_i_do_anything_more
intent: affirm
action: utter_ask_query
intent: faq
action: utter_faq
intent: goodbye
action: utter_goodbye
story: faq with got_solution and deny
steps:
intent: got_solution
action: utter_can_i_do_anything_more
intent: deny
action: utter_goodbye
story: faq questions
steps:
intent: faq
action: utter_faq
story: faq questions with bye
steps:
intent: faq
action: utter_faq
intent: goodbye
action: utter_goodbye
story: faq questions with bye help categories
steps:
intent: help_categories
action: utter_help_categories
intent: faq
action: utter_faq
intent: goodbye
action: utter_goodbye
story: faq with chit chat help categories
steps:
intent: greet
action: utter_greet
intent: introduction
entities:
- name : “Douglas”
action: intro_form
action: utter_how_can_i_help
intent: help_categories
action: utter_help_categories
intent: faq
action: utter_faq
intent: got_solution
action: utter_can_i_do_anything_more
intent: goodbye
action: utter_goodbye