Predicted next action is always "Chitchat"

Rasa Open Source
1

Hi, I have a problem with my chitchats

I’m using small talk following the instructions from Chitchat and FAQs

Chitchat works as expected but when I’m trying to say something not related like “I want to eat hamburguer”

image

Your input ->  i want to eat hamburguer                                         
2020-07-13 15:28:44 DEBUG    rasa.core.tracker_store  - Creating a new tracker for id 'ae3725d1a6f4498c86048c35dad969b8'.
2020-07-13 15:28:44 DEBUG    rasa.core.processor  - Starting a new session for conversation ID 'ae3725d1a6f4498c86048c35dad969b8'.
2020-07-13 15:28:44 DEBUG    rasa.core.processor  - Action 'action_session_start' ended with events '[<rasa.core.events.SessionStarted object at 0x7f72ac3a3198>, <rasa.core.events.ActionExecuted object at 0x7f72ac3a3160>]'.
2020-07-13 15:28:44 DEBUG    rasa.core.processor  - Current slot values: 
	nombre_alicia: None
2020-07-13 15:28:44 DEBUG    rasa.nlu.selectors.response_selector  - Adding following selector key to message property: chitchat
2020-07-13 15:28:44 DEBUG    rasa.core.processor  - Received user message 'i want to eat hamburguer' with intent '{'name': 'chitchat', 'confidence': 0.9999862909317017}' and entities '[]'
2020-07-13 15:28:44 DEBUG    rasa.core.processor  - Logged UserUtterance - tracker now has 4 events.
2020-07-13 15:28:44 DEBUG    rasa.core.policies.memoization  - Current tracker state [None, None, None, {}, {'prev_action_listen': 1.0, 'intent_chitchat': 1.0}]
2020-07-13 15:28:44 DEBUG    rasa.core.policies.memoization  - There is a memorised next action 'respond_chitchat'
2020-07-13 15:28:44 DEBUG    rasa.core.policies.fallback  - NLU confidence threshold met, confidence of fallback action set to core threshold (0.3).
2020-07-13 15:28:44 DEBUG    rasa.core.policies.ensemble  - Predicted next action using policy_0_MemoizationPolicy
2020-07-13 15:28:44 DEBUG    rasa.core.processor  - Predicted next action 'respond_chitchat' with confidence 1.00.
2020-07-13 15:28:44 DEBUG    rasa.core.actions.action  - Picking response from selector of type chitchat
2020-07-13 15:28:44 DEBUG    rasa.core.processor  - Action 'respond_chitchat' ended with events '[BotUttered('Me llamo Alicia, guapo', {"elements": null, "quick_replies": null, "buttons": null, "attachment": null, "image": null, "custom": null}, {}, 1594668524.4719737)]'.
2020-07-13 15:28:44 DEBUG    rasa.core.processor  - Current slot values: 
	nombre_alicia: None
2020-07-13 15:28:44 DEBUG    rasa.core.policies.memoization  - Current tracker state [None, None, {}, {'prev_action_listen': 1.0, 'intent_chitchat': 1.0}, {'prev_respond_chitchat': 1.0, 'intent_chitchat': 1.0}]
2020-07-13 15:28:44 DEBUG    rasa.core.policies.memoization  - There is a memorised next action 'action_listen'
2020-07-13 15:28:44 DEBUG    rasa.core.policies.mapping_policy  - There is no mapped action for the predicted intent, 'chitchat'.
2020-07-13 15:28:44 DEBUG    rasa.core.policies.fallback  - NLU confidence threshold met, confidence of fallback action set to core threshold (0.3).
2020-07-13 15:28:44 DEBUG    rasa.core.policies.ensemble  - Predicted next action using policy_0_MemoizationPolicy
2020-07-13 15:28:44 DEBUG    rasa.core.processor  - Predicted next action 'action_listen' with confidence 1.00.
2020-07-13 15:28:44 DEBUG    rasa.core.processor  - Action 'action_listen' ended with events '[]'.
2020-07-13 15:28:44 DEBUG    rasa.core.lock_store  - Deleted lock for conversation 'ae3725d1a6f4498c86048c35dad969b8'.
Me llamo Alicia, guapo
Your input ->

As you can see It predicted chitchat with 0.999 confidence instead of fallback

This is my chitchat intents

## intent:chitchat/como_te_llamas
- Como te llamas?
- Cual es tu nombre?
- Quiero saber tu nombre

## intent:chitchat/eres_genial
- Eres genial
- Eres la mejor
- Eres asombrosa

My only two stories

## saludar
* saludos
  - actions_saludar

## chitchat
* chitchat
    - respond_chitchat

and this is my config

language: es
pipeline:
  - name: WhitespaceTokenizer
  - name: RegexFeaturizer
  - name: LexicalSyntacticFeaturizer
  - name: CountVectorsFeaturizer
  - name: CountVectorsFeaturizer
    analyzer: "char_wb"
    min_ngram: 1
    max_ngram: 4
  - name: DIETClassifier
    epochs: 100
  - name: EntitySynonymMapper
  - name: ResponseSelector
    epochs: 100
    retrieval_intent: chitchat
    scale_loss: false

# Configuration for Rasa Core.
# https://rasa.com/docs/rasa/core/policies/
policies:
  - name: MemoizationPolicy
  - name: TEDPolicy
    max_history: 5
    epochs: 100
  - name: MappingPolicy
  - name: TwoStageFallbackPolicy
    nlu_threshold: 0.7
    ambiguity_threshold: 0.1
    core_threshold: 0.7
    fallback_core_action_name: "action_default_fallback"
    fallback_nlu_action_name: "action_default_fallback"
    deny_suggestion_intent_name: "out_of_scope"

Please help I have been trying to solve this problem for two days

2

What other intents do you have? It might be that you need to provide more training examples for the individual intents.

3

@Tanja I am also facing the same issue that all the inputs other than few trained intents going to chitchat only. I also added two stage fallback logic in the custom actions and config. I expect a lower confidence for unknow or gibberish input but always 1.0 or 0.99 for chitchat.

image
image2256×1504 364 KB