If I provide random input text like “…” , “???”,"-=-%$^" or “qwerty”, Rasa NLU returns very good confidence with any intent. Such words are not part of any of the training data in any intent, yet NLU returns intents and confidence level being higher than threshold. Can you please guide how to fix this ? I believe such words should be with lower confidence and go in side fallback intent.

with the server, I get: curl 'localhost:5000/parse?q=qwerty&project=current&model=nlu' | python -m json.tool % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 165 0 165 0 0 138 0 --:--:-- 0:00:01 --:--:-- 138 { "entities": [], "intent": { "confidence": 0.0, "name": null }, "intent_ranking": [], "model": "nlu", "project": "current", "text": "qwerty" }

it worked in virtual environment

Then, you do not use 0.13.1 in your main environment

The main environment also shows the same version but not working as above mentioned. When I created a virtual environment it works as above mentioned.

Did you try to reinstall rasa_nlu?

I am facing a slightly different issue.When i input single words that has never been used in the data i get zero confidence.When i ask for some random genuine question out of my bot’s context it classifies it under the intent “inform”(inform here is my intent for question asked to my bot’s domain) with high confidence (70-80%). I am using tensor flow embedding pipeline.(tensorflow 0.10.0)

I face a smimiliar issue. If I just use single words like at which are conatined more often conatined in one intent, then it is classified as it. That is of course fine! But it has very high confidence like 0.97. I rather like that it is low for an fallback treshold. Why is there such an high confidence when you just train senetcnes with many words and single words like at. So, it seems tensorflow embedding misses here some relative importance aspects (relative importance agaibst count of trained words) ?

@Abir 70-80 is actually relatively low, you should be able to handle that with the fallback policy. @datistiquo hm, if the word at is in your training data a lot, then i’m not surprised that the confidence is quite high. it doesn’t explicitly take into account the relative importance agaibst count of trained words

yes … but then what i was facing was , there were many contextual questions that were predicted with a confidence with (70-80%) . So you understand if i simply put a fallback with say 0.75 it would filter those contextual questions as well. But no worries i found a fix.

NLU detects random input with wrong intent and high confidence

Rasa Open Source

harshitazilen (Harshit Azilen) August 31, 2018, 11:40am 26

I just upgraded it. Seems now I will need to reinstall it.

Topic		Replies	Views
NLU detects random input with wrong intent that is none with zero confidence Rasa Open Source	3	1074	March 8, 2019
Random input - intent classified with high confidence Rasa Open Source	5	701	December 22, 2020
Rasa classifies random input as intents with high probability Rasa Open Source	24	1794	April 20, 2023
Rasa NLU predicting random intent Rasa Open Source	14	1754	November 15, 2019
Wrong Confidence Rasa Open Source	1	336	February 26, 2020

NLU detects random input with wrong intent and high confidence

Related topics