Detecting multiple entities of the same type

Rasa Open Source
1

I want to detect multiple entities of the same type in the sentence:

Order 2 bananas 3 oranges and 5 apples Order 5 cases of spinach 16 ounces 7 kgs of fuji apples 12 packs of milk

Entities above are:

  1. Count - 2/3/5/7
  2. Products - bananas/oranges/spinach 16 ounces/fuji apples
  3. Units - cases/kgs/packs

While I can train the model really well to pick n different types of entities I am facing issues with having multiple entities of multiple types in the same utterance.

I have tried to generate training data using chatito but the permutation and combinations run into 10s of millions.

Any suggestions around this?

1 Like
2

Hi @spiralswimmer, when you day that you are having issues, do you mean that the NLU is not extracting all the entities correctly from the user message?

3

Yes that is correct. The behaviour is inconsistent. It is extracting them sometimes and other times it is not. It kind of depends on what the entities are. Some combinations are okay while some others are not and extract less than the total number of entities.

1 Like
5

Have you got the answer how to fix this issue? I have the same issue but don’t know how to fix it.

6

@spiralswimmer Have you found a solution to this yet? I am having a similar issue with certain pairs of entities are extracted but other are not.

7

This sounds like a case where a lookup table could be useful. Part of the problem with many entities in the same sentence like this is that the possible contexts for each entity are very many, and both sentence context and the word itself inform CRFEntityExtractor (<Rasa1.7)/DIETClassifier(>Rasa1.8) .

Also, in the case of numbers, you could use a pretrained extractor like DucklingHTTPExtractor or SpacyEntityExtractor instead, which should be a lot more accurate (with no requirement for training data). Same for other dimensions supported by duckling or spacy; if you can use them, it’s often more reliable than providing your own training data.