Add examples with synonyms

Do I need to add exampels with synonyms in my training data ?

I have the following training data :

{
    "text": "Si je vais chez un psychologue, est-ce que vous me remboursez ?",
    "intent": "remboursement_medecine_douce",
    "entities": [
      {
	    "start": 19,
    	    "end": 30,
    	    "value": "psychologue",
    	    "entity": "specialite_medecine_douce"
      }
    ]
},

with this synonym :

{
    "value": "psychologue",
    "synonyms": [
      "psychologie",
      "psy"
    ]
}

So if I use the exact training sentence, the NLU will detect “psychologue” but if I use the sentence with “psy” instead of “psychologue”, the NLU won’t extract it. I know there is the following statement in the doc : " Entities must be properly classified before they can be replaced with the synonym value." but I don’t understand the utility of the entity_synonyms field if I have to create an example for each synonyms.

So what is the correct use of synonyms ? In the entity doc (Training Data Format) there is two options one with two training sentences and one using entity_synonyms. The second alone doesn’t seems to work, do I need to implement the first option too ?

Thank you in advance :slight_smile:

2 Likes

Hi @huberrom, in short, yes you do need training sentences with different values of your synonyms. Probably not all of them, but at least some of them. The idea behind synonyms is that the value of your extracted entities is mapped to a specific value (e.g. if you want to account for misspellings or different variations of a word) so you can then use it properly in your custom actions/slots/etc.

We do have something called lookup tables as well: Training Data Format, though not sure if that’s something you should use in this case

Thank you for your answer.

I know the idea behind the synonyms and that exactly why I’m using them (because I need to request a CMS with the value, but I also want my bot to understand variations of the word), but I thought at first that maybe the NLU will train with synonyms and understand them without having to use them in training sentences. For example, rasa-ui is creating variations of training sentences with the differents synonyms (e.g if you have one training phrases and 3 synonyms, you’ll end up with 4 training phrases, the original one and the three variants).

But thank you for your explanation, I’ll just add some examples in my training.

Unfortunatly I can’t use lookup tables since it’s not supported by rasa-ui yet, and I don’t think it’s useful in my case