This tutorial on the TwoStageFallbackPolicy had a section on empirically finding a good NLU Threshold.
I’m just wondering if there is a similar method to find a good Core Threshold.
the same principles should apply, but the simplest is to run through some test conversations and see what kind of threshold works best for you. Unfortunately we don’t yet have a script that’s as automated as for NLU. If you were up for creating one, that would be an awesome contribution!
I’d like to help.
Anyone here can mentor me? I’m not sure what kind of tests to run.
As taught on the tutorial you’ve linked, you can run some test with trainning/test data or doing cross validation so you can see the distributions of intent prediction confidence. The better the NLU model, blue bars will go to the right and red ones to the left. If a value (threshold) of 0.50 makes a good separation between these colors, that will indicate that when nlu model predict some intent with confidence higher than threshold, you will get it right.
Results will be deeply dependent on your pipeline defined and dataset.
Hi @amn41 My customers often avoid the widget button and I would like to have the widget window automatically open when they arrive on my page. Is it possible to have the window pop up when someone goes to my website? How do I customize my widget so that the window is displayed instead of the button? I am using rasa webchat.