Rasa unable to recognize unseen entities

Rasa Open Source
1

I am building a paper search bot. And i have an entity “paper_type” and slot “paper_type” which stores the category of paper. But the problem with the bot is it recognizes only those paper_type which are mentioned in the nlu data under intent paper_category. It is unable to search for any new paper.

I built another bot for weather. It have an entity “location” and slot “location”. The problem with this bot is the same as above but although in some cases it is able to recognize the loction which is not mentioned in the nlu data but it only recognize the first part of the location. eg. if the location is “connaught place” and if this is not present in nlu data then it fills the slot location with “connaught” only.

It also creating unwanted synonyms like

synonym:manchester

  • Manchester

and i don’t want this

I have sufficient amount of data in nlu for the intents paper_category and weather_location. And as I increase the data it didn’t learn but just memorize the values of those slots.

I also have more than 130 stories.

I think all it is doing is just memorizing.

What to do?

and i also want to remove the unwanted synonyms

2

You can use lookup tables. I think you can delete the unwanted synonyms in the nlu training data files.

3

I went through the blog, it’s really great but it doesn’t solve my problem because i don’t have a dataset for the entity paper_type and neither i can create it. I don’t know beforehand what the possible paper_types are.

image
image2550×1112 232 KB
The api i am using won’t give types of paper, it just search a database for a given paper_type, if there is a paper under the category of that paper type then it returns result otherwise returns 0 result.

For the case of weather bot their are billions of location across the world, I can’t use lookup table for them also.

For the case of synonyms I deleted them from my nlu.md file a lot of times but they appears again and again. So I deleted the synonym mapper from the config.yml because i think they are appearing because of them but synonyms still appears in my data file

4

Hey @GauravSharma were you able to solve this problem? I’m facing the same issue