Im trying to fine tune the core_threshold for the fallback policy and knowing whats the confidence of prediction of a particular action would greatly help.
Is there a way to get it ?
Thanks!
Hello, @vivekanon. I’m currently using rasa shell with debug to check the threshold:
rasa shell --debug
This shows the confidence and which Policy is used to return the next action. Hope it helps. 
1 Like
Yeah that sure helps, but im forced to interact on the terminal. Is there no way to get it for a deployed bot ? Im interacting over an api and would like to see the confidence of the predictions in this setup.
Also, in rasa shell , the fallback actions are shown to be predicted at a confidence always equal to core_threshold .
Any idea why that is ?
I don’t know how to do this but I’m interested too, if anyone can answer it.
About confidence in different policies: FallbackPolicy is set to fire whenever core_threshold is below the minimum in your settings. So if it’s fired, it means the other dialogue policies weren’t able to predict intent higher than the minimum.
We are currently facing this issue in our project now and I guess there are many things we should explore to improve it. The ones we are trying to tackle are: better policy setup, testing with different pipelines, more stories and adding entities (we’re currently trying to predict without entities yet, only intents). Let me know what you think you’re going to do too
Also, here in documentation you can see how to change this core_threshold.
Yeah so i believe about what core_threshold is supposed to do.
But then, if you chek the tracker, the fallback action event has a confidence always equal to the core_threshold . That’s strange.