Intentions and confidence

Rasa Open Source
1

Is there any way to reduce the confidence on incoming words if I know that the ML is super inaccurate?

input> can’t right now

message_intent: c_features message_confidence: .6839

 intent: c_features
  examples: |
    - Features
    - C features
    - What are the features of C?
    - Advantages of C
    - C Programming Features

I don’t actually understand how the confidence is so high when in fact no words even closely match… but, nevertheless; how can I for manually edit this confidence in python?

2

I don’t think there’s anything out-of-box, so I’ll handle it with some custom Python code… But I am curious why the confidence is so high for this specific scenario.

3

Hello Joel,

Thank you for your question! I would recommend to try to train your bot so it does give you the correct intent prediction instead of changing the confidence by brute force (with custom Python code).

Some tips on debugging :

  • try to run with --debug flag to see what NLU is doing (as already mentioned here also)
  • try to add some more examples under the c_features intent (10-20)
  • align your intent classifier with the rest of your pipeline (config.yml) or share it and I can also have a look
  • make sure that your intents are not too specific and each one represents a broad intent that the user is trying to do

The last two are also mentioned in this video from Rachael in section common errors (min 4:29).

Also, some resources that might be useful for you are the following :

  • Here you can find a very insightful Rasa blog post about intent classification
  • Here is a very good Rasa blog post that can give you more insights on your NLU pipeline
  • Confidence explained by Rachael here
  • This post gives an insightful explanation on accuracy