Threshold too much all or nothing

Rasa Open Source
#1

I am making a bot to make reservations and where you can ask anything else. When it is a message to do with the reservation i want to handle the story. If its any other question i need to forward is to the fallback action. This any other question can be about really anything.

When i use thresholds of 0.3 all messages are accepted as part of the story. When i use thresholds of 0.6 all messages are accepted as part of the fallback action. I feel there is no sweet spot.

Can anyone tell me what the best way is to tune this.

#2

Hey @longshot,

Are you talking about the Core threshold or NLU threshold?

At this point my recommendation is to ask yourself what kind of queries the bot can deal with, and train a list of intents on those kinds of queries. After that it’s as simple as writing stories with those intents, using a checkpoint after the “anything else?” prompt

You might also want to write a bunch of out-of-scope intents in order to trigger fallback correctly.

#3

Thank you for your recommendations. I will try them all and let you know about the results.