I am currently trying to train the DIETClassifier with different intents containing several examples. The issue I want to solve, that if the user enters an intent having the same context of one of the trained classes/intents, but actually it doesn’t really fall under it, so I don’t want the model to generate a high confidence for that intent just because it has similar context.
For example:
Assuming in the training examples we have intent called (Flu treatment)along with different examples (i.e. what is the treatment for flu?)
Suppose that the user enters (what is the treatment for tooth pain?)
In this case, I want the model to generate a lower confidence, since the (tooth pain) is not included in any of the examples
I tried to specify the main keywords where I want the model to be trained on using the entity, so for example I annotated the flu as flu
But it seems it is still it doesn’t help, maybe because it it for sure that the embedding and the context of the trained entity will be the same of the main keywords in the recognized intent …
Hi @DinaAlBassam, I think there’s a misunderstanding here about how DIET predicts intents and entities. It predicts both, but they do not depend on each other, in either direction. So, whether or not a flu entity is extracted will have no bearing on what intent is predicted.
From your examples, it sounds like you want to be able to distinguish between similar sentences with different entities. In this case, you could have one intent, e.g. treatment and then define an entity ailment or something like that. Then your data would look something like this:
## intent:treatment
- What is the treatment for [flu](ailment)?
- what is the treatment for [tooth pain](ailment)?
You can then use the extracted entity value to set a slot and influence the next action prediction, or use the value of the slot in a custom action to e.g. look up the appropriate treatment.
If you only include examples for e.g. “treatment for flu”, your model will almost certainly predict a lower confidence for the treatment intent when given an unseen example like “treatment for tooth pain”. You don’t need to train it to predict lower confidence for wrong things, you just need to provide lots of examples of the right things.
Does that make sense?