Utter_default triggered despite nlu intent recognition is really high

Rasa Open Source
1

Hi rasa community,

After training my model I run a rasa shell nlu to check that my intent recognition and entities extraction go well, so far so good, all my intents are correctly recognized. However when running rasa shell, the intent is not recognized anymore and utter_default response is triggered.

Find below the result of rasa shell nlu:

“intent_ranking”: [ { “id”: -5679577555852856409, “name”: “getInfoProcess”, “confidence”: 0.999994158744812 }, { “id”: -5418671895374280530, “name”: “joke”, “confidence”: 2.2008860014466336e-06 }, { “id”: -8154774908068738538, “name”: “greet”, “confidence”: 1.3680216852662852e-06 }

Any idea how to modify this behavior so my intent is recognized?

Best,

ch.

1 Like
2

Hello @Chrisprollsite

What is the output of rasa --version for you? Do I understand you correctly: when you run rasa shell nlu and type hello you get intent greet with confidence > 0.99, but when you type the exact same hello in rasa shell --debug, you get a different intent or a different confidence?

The utter_default may just be triggered because Rasa doesn’t know what to do next, given the conversation history, even though the intent was classified correctly. In this case, you should add more stories to your training data, so Rasa can learn. You may also change settings of the fallback action: Fallback and Human Handoff

3

Yes that is exactly, nlu intent is correct but it seems the next action isn’t triggered properly, maybe it also has something to do with TED policy:

image
image1465×187 11.8 KB

4

Hi @Chrisprollsite can you please provide more training examples in nlu.yml file? Hopefully your action server is running and you mention the action name in domain.yml file?

@Chrisprollsite please do share the config.yml file

5

Hi @nik202 Yes, all actions are mentioned in my domain.yml. Please find attached my nlu.yml and config.yml file config.yml (1.7 KB) nlu.yml (7.7 KB)

6

@Chrisprollsite Can you please uncommented polices in config.yml file please

policies:
# No configuration for policies was provided. The following default policies were used to train your model.
# If you'd like to customize them, uncomment and adjust the policies.
# See https://rasa.com/docs/rasa/policies for more information.
  - name: MemoizationPolicy
  - name: RulePolicy
  - name: UnexpecTEDIntentPolicy
    max_history: 5
    epochs: 100
  - name: TEDPolicy
    max_history: 5
    epochs: 100
    constrain_similarities: true

Try train the model again. I hope this will solve you issue. Try delete all older models and re-train and run.

@Chrisprollsite please share the domain.yml file ?

7

Will retrain the model with uncommented policies, meanwhile here is my domain file domain.yml (2.7 KB)

8

@Chrisprollsite please check this for use_entities : Domain

9

mmmh, weird behavior, I specifically ignored “areaName” for intent “getData” but it still extracts “areaName” when running the model:

image

image
image1458×59 6.8 KB

10

@Chrisprollsite delete older model please.

11

@nik202 I removed all other models and rasa loads properly the right model (the latest trained). And it still doesn’t work

12

@Chrisprollsite Is it working? I not get you.

13

@nik202 , removing unexpecTEDpolicy from my config file did help to figure out the proper next action. As mentioned in the rasa doc, this feature is still under development

14

@Chrisprollsite Great! please close this thread as a solution for others.

15

@Chrisprollsite Nice, but I shared you the idea :wink: Utter_default triggered despite nlu intent recognition is really high - #6 by nik202 Good Luck! Happy learning!.

16

If that makes you happy :wink:

17

@Chrisprollsite I am always happy, its good my suggestion helped you. Enjoy rasa!

18

I am facing similar issues. I have correctly added intent in domain , nlu , stories but still some of my buttons doesn’t work , I have tried using the rasa interactive but still not much of help. I don’t know what is going on , the intent must be taken correctly as it is coming from the buttons itself . Please help

19

It gives blank result