Hi @prasad01dalavi. I would guess that 10-12 examples per intent when you have 2000 intents is simply not enough + I would doubt that you really need 2000 intents and chances are that some intents simply have the examples which are too similar which results in a model making lots of mistakes. For example, just looking at the example you provided I already see that the examples you used to train both intents are very similar in terms of vocabulary and sentence structure. I would suggest diversifying them a bit more and testing the performance then
Thanks Juste. I agree we may need more examples. but about the 2000 intents are because they are different questions and answers. and you are very right that model is getting confused for the section part. as below:
## intent:intent_number_1972
- what is section 170 of IPC?
- find something on section 170 of IPC?
- show me something on section 170 of IPC?
- I need something about section 170 of IPC?
- I'm looking for something about section 170 of IPC?
- I'd like to see something about section 170 of IPC?
- find something about section 170 of IPC?
- search for something about section 170 of IPC?
- get me something about section 170 of IPC?
- something about section 170 of IPC?
- I need something on section 170 of IPC?
- I want to see something on section 170 of IPC?
- I would like to see something on section 170 of IPC?
- I want you to show me section 170 of IPC?
- I want something about section 170 of IPC?
- I would like to see section 170 of IPC?
## intent:intent_number_1974
- what is section 171 of IPC?
- find something on section 171 of IPC?
- show me something on section 171 of IPC?
- I need something about section 171 of IPC?
- I'm looking for something about section 171 of IPC?
- I'd like to see something about section 171 of IPC?
- find something about section 171 of IPC?
- search for something about section 171 of IPC?
- get me something about section 171 of IPC?
- something about section 171 of IPC?
- I need something on section 171 of IPC?
- I want to see something on section 171 of IPC?
- I would like to see something on section 171 of IPC?
- I want you to show me section 171 of IPC?
- I want something about section 171 of IPC?
- I would like to see section 171 of IPC?
and many more sections are there. How do i make it different. It is as it is. User will ask like that only. Its working fine for other intents but completely fails for section thing as its lot similar to each other.
Question 2:
Will you please tell what is the best Test Configuration I used here. there are 4 tests i listed down. so that i will go with that only. because more augmentation is taking lot of time for training and also the epochs.
Queston 3:
Spacy is not a good choice as you have mentioned in pipelines that for more than 1000 examples we need supervised.
And I would like to appreciate your work as team and individual as I can see how clearly you explained it on youtube.
You may have better luck treating this as a NER problem and trying to extract the section number from the text and have some conversation flow/database query action based on that.
In the examples in your initial post you could do something similar, or also just do a keyword lookup (though not actually sure how to do that in Rasa short of a custom action that just searches the input string)
HI @prasad01dalavi. You should definitely go the NER route for this use case. It will be almost impossible to build a good classification model when your examples for different intents are completely the same except for the different entity which is present in the sentence. In the example you included, you should have just one intent, for example intent_number and write your examples as follows:
- what is section [171](number) of IPC?
If you use duckling component to extract the numbers you will not need to do any data labelling.
By going this route you will need way less data to achieve much better results for you assistant. I am almost sure, that you will be able to cut your data at least in half by better designing the training examples (using entities instead of just intents).
Happy to see it worked. The locale and timezone you used should work for Berlin