I’m trying to use rasa nlu along with OpenWeatherMap to create a weather bot. Thus, I’d need rasa to return location entities. I’m using SpacyEntityExtractor with the trf model as DietClassifier is having trouble correctly identifying locations, and I don’t have the time to enter every city in every country into a dictionary.
The issue I’m facing is similar to this:
The extractor recognizes and properly extracts “China” in “what is the weather in China”, but not “china” in “what is the weather in china”
I know that Spacy extracts entities based on contextual clues, such as capitalization, but it is probably annoying for users to use proper punctuation in online messaging. Does anyone have a solution or a workaround?
Edit: It’s worth mentioning that I’ve tried using the truecaser package as well, but it couldn’t recognize lesser known cities.
Same here, I am using the reminder bot example and there I tried using the training data while chating with bot but still it wasn’t able to capture name from that sentence,
Fyi: I did download spacy’s md English model and load it in config
Rasa version: 2.3.4
Rasa x: 0.37.0
SpaCy, generally, works well enough for entities but it is highly sensitive to capitalisation. This has also been demonstrated in names of people too. Also note that spaCy completely ignores the data in Rasa’s yml files. Rasa assumes that spaCy is pre-trained.
@Horizon733 it’s not really a matter of “turning something on”. It’s a matter of, this spaCy model is pre-trained and the only way to change it is to train spaCy on a specific task. This would mean training your own spaCy model.
Did you see our video/blogpost on name detection? The video was shared in the previous message but this blogpost highlights some alternatives that would work for names of people.
@koaning Thank you for the link to the blogpost! After reading through it, I’ve decided that lookup tables would best solve my issue, and thus have compiled a lookup table with most major cities throughout the world as well as countries that I needed
To anyone who has this issue, thinks this method will help them as well, and wants a copy of my lookup tables, here’s the github link! I hope it helps you in some way or form.
We currently host a similar repository for intent examples. It even has a fancy streamlit app here. I’m wondering if it makes sense for us to host something similar for entities. Your repo has some useful ones but we might also be able to merge it with the baby-name examples currently hosted in rasa-nlu-examples.
I think it’d be a pretty cool idea to have a repo of entities, especially the more “famous” ones like book titles, names, geographical locations etc.
I’m very new to github and I’m not too familiar with how to pull, push or merge stuff, but if you guys think it’d be nice to have some examples of cities in your repos, I’ll try my best to learn how to pull/push files and how to do merge requests!
It’s worth noting though that since it’s from the Lite version of GeoCities, it only contains major cities, but I hope it helps nonetheless!
)