Rasa X and action_default_fallback

Rasa Open Source
1

Hello,

I’ve succesfully shared my bot, but the problem is it will break it self after a few messages. The problem is that it will predict action_default_fallback even though it can correctly identify the intent, even with high confidence.

I have not used the action, so I dont understand where it’s coming from.

I tried going into debug mode on the command interface to see if there was a meaningful error. This was the output:

2021-05-12 19:29:52 DEBUG rasa.core.lock_store - Issuing ticket for conversation '15a51c34df1d42d58914d4caae7a647f'.2021-05-12 19:29:52

DEBUG rasa.core.lock_store - Acquiring lock for conversation '15a51c34df1d42d58914d4caae7a647f'.2021-05-12 19:29:52

DEBUG rasa.core.lock_store - Acquired lock for conversation '15a51c34df1d42d58914d4caae7a647f'. 2021-05-12 19:29:52 DEBUG rasa.core.tracker_store - Recreating tracker for id '15a51c34df1d42d58914d4caae7a647f' `2021-05-12 19:29:52

DEBUG rasa.core.processor - Received user message 'jeg forstår ikke hvad der menes med for lidt?' with intent '{'id': -7753130457771574260, 'name': 'Slider', 'confidence': 0.9963321089744568}' and entities '[]' 2021-05-12 19:29:52 DEBUG rasa.core.processor - Logged UserUtterance - tracker now has 11 events. 2021-05-12 19:29:52 DEBUG rasa.core.policies.memoization - Current tracker state:

[state 1] user intent: Slider | previous action name: action_listen

[state 2] user intent: Slider | previous action name: utter_Slider

[state 3] user intent: Slider | previous action name: utter_Assistance

[state 4] user intent: Slider | previous action name: action_listen

2021-05-12 19:29:52 DEBUG rasa.core.policies.memoization - There is no memorised next action

2021-05-12 19:29:52 DEBUG rasa.core.policies.rule_policy - Current tracker state:

[state 1] user intent: Slider | previous action name: action_listen

[state 2] user intent: Slider | previous action name: utter_Slider

[state 3] user intent: Slider | previous action name: utter_Assistance

[state 4] user text: jeg forstår ikke hvad der menes med for lidt? | previous action name: action_listen

2021-05-12 19:29:52 DEBUG rasa.core.policies.rule_policy - There is no applicable rule.

2021-05-12 19:29:52 DEBUG rasa.core.policies.rule_policy - Current tracker state:

[state 1] user intent: Slider | previous action name: action_listen

[state 2] user intent: Slider | previous action name: utter_Slider

[state 3] user intent: Slider | previous action name: utter_Assistance

[state 4] user intent: Slider | previous action name: action_listen

2021-05-12 19:29:52 DEBUG rasa.core.policies.rule_policy - There is no applicable rule.

2021-05-12 19:29:52 DEBUG rasa.core.policies.ensemble - Made prediction using user intent.

2021-05-12 19:29:52 DEBUG rasa.core.policies.ensemble - Added DefinePrevUserUtteredFeaturization(False) event.

2021-05-12 19:29:52 DEBUG rasa.core.policies.ensemble - Predicted next action using policy_2_RulePolicy.

2021-05-12 19:29:52 DEBUG rasa.core.processor - Predicted next action 'action_default_fallback' with confidence 0.30.

2021-05-12 19:29:52 DEBUG rasa.core.actions.action - Calling action endpoint to run action 'action_default_fallback'.

What am i doing wrong?

Thanks in advance

2

Hi @Blaabjerg96 action_default_fallback is a default action used by the Fallback Classifier, have you enabled this in your config.yml file? Although in your case I think this action is predicted by the RulePolicy.

The debug log is showing how it’s trying to predict next action using the policies in the config but it cannot find an applicable action in stories or rules - could you please share your config, stories and rules files with me?

3

Thank you for the quick reply.

They are as follows:

the config file:

language: da

# # No configuration for the NLU pipeline was provided. The following default pipeline was used to train your model. # # If you'd like to customize it, uncomment and adjust the pipeline. # # See https://rasa.com/docs/rasa/tuning-your-model for more information.

pipeline:

- name: SpacyNLP

model: da_core_news_md

- name: SpacyTokenizer

- name: SpacyEntityExtractor

- name: SpacyFeaturizer pooling: mean

- name: CountVectorsFeaturizer

analyzer: char_wb

min_ngram: 1

max_ngram: 2

- name: DIETClassifier

- name: FallbackClassifier

threshold: 0.8

ambiguity_threshold: 0.1

epochs: 1

# Configuration for Rasa Core.

# https://rasa.com/docs/rasa/core/policies/

policies:

# # No configuration for policies was provided. The following default policies were used to train your model.

# # If you'd like to customize them, uncomment and adjust the policies.

# # See https://rasa.com/docs/rasa/policies for more information.

- name: MemoizationPolicy

- name: TEDPolicy

# max_history: 5

# epochs: 100

# constrain_similarities: true

- name: RulePolicy

Stories:

version: “2.0”

stories:

  • story: happy path

    steps:

    • intent: greet

    • action: utter_greet

    • intent: mood_great

    • action: utter_happy

  • story: sad path 1

    steps:

    • intent: greet

    • action: utter_greet

    • intent: mood_unhappy

    • action: utter_cheer_up

    • action: utter_did_that_help

    • intent: affirm

    • action: utter_happy

  • story: sad path 2

    steps:

    • intent: greet

    • action: utter_greet

    • intent: mood_unhappy

    • action: utter_cheer_up

    • action: utter_did_that_help

    • intent: deny

    • action: utter_goodbye

  • story: Hello Dansk

    steps:

    • intent: Hilsen

    • action: utter_greet

  • story: Data

    steps:

    • intent: Datasikkerhed

    • action: utter_data

  • story: Betafunktioner

    steps:

    • intent: Betafunktioner

    • action: utter_beta

  • story: DreamPlanVsBank

    steps:

    • intent: DreamplanVsBank

    • action: utter_DreamplanVsBank

  • story: Kontakt bank

    steps:

    • intent: KontaktBank

    • action: utter_KontaktBank

  • story: AndreTingAfValue

    steps:

    • intent: AndreTingAfValue

    • action: utter_AndreTingAfValue

  • story: HvordanGorJeg

    steps:

    • intent: HvordanGorJeg

    • action: utter_HvordanGorJeg

  • story: DrommeFunktion

    steps:

    • intent: Dromme

    • action: utter_Dromme

  • story: Slider

    steps:

    • intent: Slider

    • action: utter_Slider

    • action: utter_Assistance

  • story: AndreDreamPlan

    steps:

    • intent: AndreDreamPlan

    • action: utter_Andre

    • action: utter_Assistance

  • story: Forud

    steps:

    • intent: Forudsatninger

    • action: utter_Forud

  • story: Fallback

    steps:

    • intent: nlu_fallback

    • action: utter_fallback

Rules:

version: “2.0”

rules:

  • rule: Say goodbye anytime the user says goodbye

    steps:

    • intent: goodbye

    • action: utter_goodbye

  • rule: Say ‘I am a bot’ anytime the user challenges

    steps:

    • intent: bot_challenge

    • action: utter_iamabot

  • rule: fallback regler

    steps:

    • intent: nlu_fallback

    • action: utter_fallback

I hope it’s somewhat readable

4

Hi @Blaabjerg96 Thanks for providing the contents of your files.

Returning to the debug log, did the bot identify correctly the user message as intent Slider? And the next expected action should have been utter_Slider according to the story Slider?

One initial recommendation I have is perhaps it could be a good idea to transfer these shorter stories to rules if you always expect intent Slider to be followed by utter_Slider and utter_Assistance actions. Let me know if that works.

5

Ah yea that makes sense - I will try that and see if that works out

6

Unfortunately it does not seem to work, and to answer your other question. It did correctly assume actions.

It seems to be an old problem Link. Don’t know if it has been fixed though.

Even tried adding a max history factor to my MemoizationPolicy also tried with AugmentedMemoizationPolicy nothing worked unfortunately.

So update: I ran the bot in rasa shell mode - it worked fine, but does not seem to work in Rasa X, is this helpful to maybe find the error?

7

I have now fixed the error, it was probably something to do with the above mentioned changes. However the bug was still remaining until I changed language in the config file, and changed it back to the original language. I have a post regarding this issue here