When performing intent classification, the DIET takes all the training examples under ask_faq/ and uses them for predicting if the user message belongs to the ask faq_category.
So basically if I have ask_faq/vacation with 20 training example questions and another ask_faq/sick_leave with another 20 training example questions, it would ignore the sick_leave and vacation tags and combines the training examples to predict the ask_faq.
Am I correct?
If the faq category is detected, the faq response selection (Information Retieval model) gets the same weights from DIET and it adjusts them further on the tag level (sick_leave) by working out the similarity between the training question and the training response and fits them under a label/tag (sick_leave).
What’s the benefits of having DIET in front of Response Selector when I have a use case of information retrieval on FAQ? Even for chitchat we can add them as Q-A pairs with the FAQ. I don’t see the intuition of having a whole model just to detect an intent ‘are you a bot’ or ‘greet’.
Is there a way to use the Response Selector directly? Or how to change DIET into the Response Selector? Is it possible to have a pipeline with Response Selector only or is there any experimental version of such pipeline designed specifically for information retrieval?
Any hints would be helpful and ideas on how to turn this into a pure IR model without intent classification would be helpful.
Thank you!