We are trying out Rasa core and using the nlu part of it extensively for our intent matching for user queries. What we have experienced is that we are getting low scores for the intent with the exact matching user queries vs intent samples.
When we pass the intents like “goodbye” or “I need the latest version of the toolkit installed on my PC” we are getting lower scores for our model parse api and correspondingly for our core:
For “goodbye” :
"intent_ranking": [
{
"name": "intent_Greet",
"confidence": 0.3711083233
},
{
"name": "intent_Bye",
"confidence": 0.1221811026
}
]
For “I need the latest version of the toolkit installed on my PC” :
"intent_ranking": [
{
"name": "intent_Q37",
"confidence": 0.3810080886
}
}
What we expected was that for exact query matches the scores would be higher from the Rasa nlu model. Could you suggest any configuration changes for achieving higher scores for exact matches, or any training techniques which we can focus on to achieve this.
At present we have close to 156 intents and 4000+ intent samples, for training the rasa nlu, and we would not be able to share the data with you as a whole. The configuration and the training samples for the specific intents for which we are getting low scores are attached nlu.md (936 Bytes) config.yml (548 Bytes)