Not able to identify locations from lookup table

My bot is not able to identify a few locations which are mentioned in the lookup file. I tried different nlu pipelines : CRF entity extractor is able to identify locations when provided in sentences ( I want to travel to NYC) and DIET cannot. But both of them are not able to identify locations provided alone.

For example:
user : search trains
bot : enter Source location
user : NYC

For such cases, the location should be provided without any neighboring words. Few locations in such cases are misclassified to another intent. Is it a problem of lookup connection or features for the language. I tried using with and without Spacy featurizer for Telugu language.

I have added lots of samples under the related intent. Annotations are proper. I have also tried using roles for Entities. Reduced the size of lookup file.

Is there any way to connect lookup file, how to solve this?

What version of Rasa are you using? We’re making a new change in Rasa 2.0. In the pre-2.0 versions we offer a lookup table to add a feature to the entity-extractor. This feature will help nudge an algorithm in the right direction but it does not offer a guarantee. In the new version of Rasa we will have a RegexEntityDetector as opposed to a RegexFeaturizer. This new RegexEntityDetector should be able to fetch them directly as opposed to merely adding features.

1 Like

I have experimented with different pipelines and policies in rasa versions 1.8.3, 1.10.8, and 1.10.12.

Is it possible to say, when can we expect this new version?

@koaning a nice move by Rasa, introducing RegexEntityDetector. Using lookup tables as a feature was a pain for the community. Thanks

1 Like

I think the 1st release candidate is due this/next week. I cannot promise anything more detailed than this for now.