I am using RASA NLU for intent identification and entities are custom and having few doubts on training pattern. Please find the details below.
If your question consist like 6 words, suppose if we have mapped all words to single custom entity or majority of words are mapped to entity and my question is will NLU consider even entity words are also to identify intents or it will ignore if we mapped them as entity.
2 . And also generally, NLU will prefer only some kind of lengthy statements or we can add like below example.
RAM Failure as an Intent and statement is very directly RAM down. Do you think, any wrong here or it’s absolutely fine and my question is it compulsory we need to make the statement is lengthy like My system is not working and may be RAM problem etc. Can you please comment on that and if first example also fine means RAM down and then if mapped two words are an entity and am interested how the training will perform and will i loose anything in future or it’s not a general practice.
Is it correct, if we have good number of examples then we can use n Number of intents and not needed to limit and we can extend like 10 or 100 intents also?
And if our dataset is imbalanced how to handle them while the training.
How to train the custom entities, example if we have mapped few words are entities and in future even some near by words are also won’t recognize and how to resolve them smartly because if we do everything as dataset and it would be like one to one pattern and where is learning scope on NLU.
Thanks and let me know if you need any other information and also am not sure whether any questions are irrelevant to you but these came with curiosity
intent classification and entity recognition are completely separate – all words including entity words are used in intent classification!
it doesn’t prefer short or lengthy statements – you should include both if you think it’s probably your users could use either one.
Yes, if you have a good number of examples you can increase your intents – but the goal shouldn’t be to have a bunch of intents – usually if you have lots of intents this is a signal that some of your intents should be merged and entities used to really define the difference in followup action after input
not sure what you mean by this, i think the machine learning model will take care of that for you e.g. if you are picking up names, you don’t need to inlclude every possible name as an example for your bot to realize that the blank in “my name is ___” to be classified as a name
Thanks for detailed answers,few are clear and i have some more doubts on the above answers.
We are facing one problem i.e Entities Are Not Generalizing, I assume this is because of lack of training data for entities and found below point
Provide enough examples (> 20) per entity so that the conditional random field can generalize and pick up the data
This means for any each custom entity, we should train more than 20 ways of each entity?
What are the tools used in kitchen.
Example: Tools is one custom entity and do we need to train them differently like automation process, fastest way and auto tools etc representing the tools entity?
And also If possible can you share me how custom entities are recognizing in detail like article or tutorial or any code snippet.
And above example, you are talking about example: “my name is sharath” and my question is i have mapped name entity values like Sharath, Bharath and Suresh etc using lookup tables or regular expressions or any custom entities and suppose new name found Kiran, will it detect automatically?
Coming to Intents design, as you mentioned we can increased if we have good number of examples but the goal shouldn’t be bunch of intents. In ideal cases, how many intents will be better and i understood it’s not possible practically to balance them all into equal count.
From NLU point of view, we need to include in pipeline like “batch_strategy: sequence” or “Balanced batching”?
Initially trained intents having confidence score is little high and after adding new intents it’s degrading, is it because of new intents and also how to improve confidence score or how to handle this issues?
This means for any each custom entity, we should train more than 20 ways of each entity?
yes
And above example, you are talking about example: “my name is sharath” and my question is i have mapped name entity values like Sharath, Bharath and Suresh etc using lookup tables or regular expressions or any custom entities and suppose new name found Kiran, will it detect automatically?
Yes if there is enough training data, though names are notably a hard task for entity extraction because there are so many that don’t follow a particular pattern
Coming to Intents design, as you mentioned we can increased if we have good number of examples but the goal shouldn’t be bunch of intents. In ideal cases, how many intents will be better and i understood it’s not possible practically to balance them all into equal count.
number of intents depends entirely on the domain, but if you find two intents are being confused often, they likely should be merged
From NLU point of view, we need to include in pipeline like “batch_strategy: sequence” or “Balanced batching”?
Balanced batching (default on latest rasa) is a good solution for unequal intents. It will require more memory to train, if your dataset is relatively small this shouldn’t be an issue
Initially trained intents having confidence score is little high and after adding new intents it’s degrading, is it because of new intents and also how to improve confidence score or how to handle this issues?
This is due to intent crossover as described, you are probably adding new intents that are too similar to one another.
** I am very much interested to see Entity extraction mechanism, can you share some forum link or tutorial or code snippet for entity extraction how exactly happens in RASA**
New intents are very different and i observed only one difference count wise new intents having more statements because they are generic ones and old intents are domain specific
can you share some tips about improving confidence score
e.g. if you are picking up names, you don’t need to inlclude every possible name as an example for your bot to realize that the blank in “my name is ___” to be classified as a name