In my chatbot project I am currently using a lookup table and synonyms together to improve entity detection for one specific entity.
This entity is called categories and simply consists of about 20 product categories relevant to the users. Now I want to make it as robust as possible detecting these categories so I try to include a bunch of synonyms with different spellings, partial synonyms etc.
I learned that you have to include those synonyms in the lookup table aswell so I did so. But I still encounter some issues with the entity matching especially when those categories consist of multiple words.
Example: One category is called “Connected Life Services”. One synonym I included is “life services”. I included this synonym in the lookup table, as well as in the nlu data as:
synonym:Connected Life Services
- life services
Now if I talk to my bot and I use the synonym life services in a sentence that I have similar training data for, it goes ahead and detects “life” as the entity instead of “life services” and then does not correct this via synonyms and so the entity is mapped as “life” instead of “Connected Life Services”.
My guess is that the lookup table words get matched individually and so it mappes “life”, but then searches for “life” in synonyms and since it is only mentioned as “life services” doesnt find anything so it doesnt get replaced with the correct entity.
Is there any way to go about this differently than just having those single words as synonyms aswell? In this example “life” is a bit too generalistic to be added to lookup tables and synonyms, so I would prefer if it only gets matched in connection with the services part.