Raise core confidence

Rasa Open Source
1

Sometimes the NLU identifies correctly what intent I want but Core gives utter_default. I already provided a lot of data to core, how can I improve it even more?

1 Like
2

Hi @devdeca! If NLU incorrectly identifies your intent, how do you expect Core to take the correct action?

3

Hello, @tyd. ‘incorrectly’ was a typo, I edited it now. Thank you.

4

@devdeca can you share your stories and config file with me?

5

@tyd I cannot share the stories file but to describe it, for each intent there is a story with a single question and response, then there is lots of stories with multiple intents and responses.

The config file is like that:

# Configuration for Rasa NLU.
# https://rasa.com/docs/rasa/nlu/components/
language: pt
pipeline:
- name: "WhitespaceTokenizer"
- name: "RegexFeaturizer"
- name: "CRFEntityExtractor"
- name: "EntitySynonymMapper"
- name: "DucklingHTTPExtractor"
  url: "{link to server}"
  dimensions: ["time"]
- name: "CountVectorsFeaturizer"
- name: "CountVectorsFeaturizer"
  analyzer: "char_wb"
  min_ngram: 1
  max_ngram: 4
- name: "EmbeddingIntentClassifier"

# Configuration for Rasa Core.
# https://rasa.com/docs/rasa/core/policies/
policies:
  - name: MemoizationPolicy
    max_history: 1
  - name: KerasPolicy
    epochs: 1000
    max_history: 1
  - name: MappingPolicy
  - name: FallbackPolicy
    nlu_threshold: 0.4
    core_threshold: 0.3
    fallback_action_name: 'utter_default'
7

Why are you using a max_history of 1?

8

Because before I did it, the model wouldn’t recognize anything that didn’t look exactly like a existing story from the file. (Related problem can be seen here: Bot not recognizing intent second time).

9

If you set your max_history to 1, Rasa Core can only predict based on the most recent conversation turn. max_history controls how much dialogue history the model looks at to decide which action to take next. When it is 1, it only looks at the most immediate dialogue history.

In your case, the FallbackPolicy triggers utter_default when either NLU confidence is below 0.4 or Core confidence is below 0.3. You said that the intent is correctly predicted, so my guess is that your Core prediction is under that threshold.