RASA NLU pretrained embeddings not detecting intent even if the input is straight forward

I am using the “pretrained_embeddings_spacy” pipeline in the config. And my intents(nlu.md) is,

intent:greet

  • hey
  • hello
  • good morning
  • good evening
  • hey there

intent:project_usecase

intent:client_info

intent:person_enquiry

intent:full_form

intent:owner

And config.yml is, language: “en”

pipeline:

  • name: “SpacyNLP”
  • name: “SpacyTokenizer”
  • name: “SpacyFeaturizer”
  • name: “RegexFeaturizer”
  • name: “CRFEntityExtractor”
  • name: “EntitySynonymMapper”
  • name: “SklearnIntentClassifier”

I trained the model using, “rasa train nlu” And it got trained successfully.

Then I try to analyse the model by “rasa shell nlu”

When I tried to enter my input, leader of pv first { “intent”: { “name”: “project_usecase”, “confidence”: 0.36016004367122345 }, “entities”: [ { “start”: 10, “end”: 18, “value”: “pv first”, “entity”: “project_name”, “confidence”: 0.9780724954649112, “extractor”: “CRFEntityExtractor” } ],

It is detecting absolutely wrong intent, more over my input is the same sentence which I had mentioned in the nlu.md (“leader of pv first”).

What’s happening here, can someone explain.

I would guess you have too few examples, and they are too similar. “pv first” is in every single intent. Try making at lease 10 unique phrases for each intent, and write different entities.

Understood. But even for this case, where “Kalyanasundaram” is specifically in a single intent it is classified wrongly as, who is kalyanasundaram { “intent”: { “name”: “full_form”, “confidence”: 0.3468138762898333 }, “entities”: [ { “start”: 7, “end”: 22, “value”: “kalyanasundaram”, “entity”: “person”, “confidence”: 0.8666814391059177, “extractor”: “CRFEntityExtractor” } ]

Here the intent is classified as “full_form”, meanwhile I haven’t specified the word “kalyanasundaram” under that intent. Or else is it a mistake from my side, do I want to change the config file?

I’m running into this issue as well, and it seems to have started once I updated rasa and began training with and using 1.4.0a1 rasa/rasa:latest-full

My intent confidence is now very low, even if there is only one exact match in the NLU data. and the two-stage fallback policy threshold is never met so it just keeps asking … “did you mean?..” I’m not sure why things are different now.

My issue was resolved upon discovering conflicting entries in nlu.md, carry on! :slight_smile:

Hi @nicholasbulka, thanks for your response. Still I didn’t understood on how you resolved the issue. Can you please give some more detail as of which RASA version you are using now, and if possible share your nlu.md data.

Any others who had faced this issue can help me out!! Thanks…

you have tagged examples an entity pv_first and at some places pv first, if you give spaces, it is tokenised so maybe try to fix the consistency of your training data.

check the ambiguity threshold in your config.yml policies declaration. This was throwing my “greet_bot” intent for a loop because I guess I had “hi” in another intent.

  • name: TwoStageFallbackPolicy nlu_threshold: 0.1 core_threshold: 0.1 ambiguity_threshold: 0.01