Hello Rasa Community, I’m developing a Chatbot and I’m currently designing a form where my bot has to extract different entities starting by first and last names. I’ve managed to extract first names through combining multiple strategies (models & lookup table) and the results are quiet good. Now I need to extract family names, but I have no idea how to do this, models don’t perform well and I obviously can’t build a lookup table for this. Did someone manage to do it ? Any ideas ? Thanks.

There are some more ideas that come to mind. Have you tried fr_core_news_md and fr_core_news_lg? These models may have better performance because they are trained on more data . It also deserves mentioning that soon-ish spaCy 3.0 will be officially released and that Rasa will start to support the new French transformer models that are inside. These might also help out here. Part of me is wondering if it might be simpler to just brute force it by adding a larger database. A quick google resulted in this website . We have a fancy new RegexEntityExtractor in Rasa 2.0 that might be of help here too.

Hi @forwitai , my name is Vincent. I’m working on a library that contains NLU components that should make it easier for non-English users to make assistants with Rasa. The project is called rasa nlu examples and it can be found here; Rasa NLU Examples . I don’t speak Arabic but I’d love learn more about the problems that you’re encountering. My guess is that there might be something to improve at the embedding part of your project. Unfortunately Duckling doesn’t work for Arabic and I fear that spaCy support is also modest at the moment. I can’t speak Arabic unfortunately, but I’m trying to support for tools to rasa nlu examples cover non-English languages like Arabic. In particular I’ve added support for the following tools for Arabic; the stanza tokeniser the fasttext and bytepair embeddings If you can share a config.yml pipeline that you’ve tried locally and perhaps part of your nlu.yml file then I might be able to help you in more detail. Also, I’ll be speaking at PyData Riyahd this Wednesday. My goal is to have a talk with folks to see if there’s more tools that I can make available for Arabic. Feel free to join in to exchange ideas!

I’ve also been reminded that there are Huggingface models for Arabic that might be worth trying out. It’s been suggested that "kuisailab/albert-base-arabic" ( github link ) might be worth to try and I think that we support it natively via our huggingface component . You might be able to use whatlies to play around with these embedding beforehand. from whatlies.language import HFTransformersLanguage from whatlies.transformers import Pca, Umap hf_lang = HFTransformersLanguage("kuisailab/albert-base-arabic") text = [ "قط","الكلب","الفأر","رجل","النساء","ملك","ملكة" ] hf_lang[text].transform(Pca(2)).plot_interactive(annot=True) If you replace text with words and names, you should see that the names appear in a different cluster.

Hi @koaning , thank you so much for your answer ! Actually I’m not dealing with Arabic letters, but rather Arabic names written in latin letters, for example “Maryam”, “Alaa”, “Samir”, etc. Those names and the french/english names. My Chatbot is trained in french, I’ve managed to extract first names with a combination of CRFEntityExtractor, Spacy and lookup tables. But for last names, it’s much harder ! Last names can be composed of 1, 2 or more words, they are very different depending on the country and you can’t just enumerate them in a lookup table, that’s where I’m stuck !

I’m wondering if you might be able to “apply a hack” then. A first name is usually followed by a last name, no? You might be able to write a custom NLU component that looks at the current tokens/entities and appends where needed. Got a small chunk of example data? Also, what spaCy model are you using? What exactly are you doing in your lookup tables? Another thing to point out; is there a reason why you’re not using a form?

I’m actually using a form and trying to extract first and last names inside ! The bot first asks what’s your first name ? then what’s your last name ? and is supposed to extract both separately. Concerning Spacy, I’m using “fr_core_news_sm”, it gives good results with french and english first names. I use a lookup table to extract arabic names written in latin letters, I’ve build a sort of database with around 3000 names. Here’s an example of my nlu data for first names : Je m’appelle Sarah Mon prénom est Maria Sophie Ok, voici mon prénom : Camélia Meriem Je m’appelle Malika And here’s an example of my nlu data for last names : Smith C’est Williams Mon nom est Alaoui Le voici : El Kamali C’est Abadi Saqqaf mon nom est Al Andaloussi Regarding your idea of asking for the first and last name at the same time, the user might say his name is “Will Smith” or “Smith Will”, his last name might be composed of 2 or 3 words, so I wouldn’t know how to locate his last name, especially if the first letters are not capital, even pos tagging wouldn’t help. I hope my case is clearer now to you, any advice is more than welcome !

There are some more ideas that come to mind. Have you tried fr_core_news_md and fr_core_news_lg ? These models may have better performance because they are trained on more data . It also deserves mentioning that soon-ish spaCy 3.0 will be officially released and that Rasa will start to support the new French transformer models that are inside. These might also help out here. I did try fr_core_news_md and it gives pretty much similar results as fr_core_new_sm in my case. Part of me is wondering if it might be simpler to just brute force it by adding a larger database. A quick google resulted in this website . We have a fancy new RegexEntityExtractor in Rasa 2.0 that might be of help here too. I think that would depend on the origin of the surnames, I’ll give it a shot ! Thanks a lot @koaning !

Pragmatically though, I might solve part of your issue by introducing a conversation design. You might be able to design stories like; [bot] What is your name? [person] Vincent D. Wamerdam [bot] Just to check, is your name Vincent D. ? If it isn't please write your first and last name again. [person] Vincent Wamerdam [bot] Just to check, is your name Vincent Warmerdam? If it isn't please write your first and last name again. [person] This is correct [bot] ... continues conversation... Again, this is me thinking out loud. But something like this feels “safe”. I’m assuming that it is very important to get the name 100% right so it might be best to add an extra step. This way the user can also catch any spelling errors.

I’ve thought about confirming the first and last names too, but I was afraid it would be a little inconvenient for the user !

It’s certainly a fair concern, but at the same time, I also imagine that a misspelled name in a database can be an even larger concern.

You’re probably right, I will try to apply your advices. Thank you so much Vincent, I really appreciate your help !

Family name extraction

Rasa Open Source

forwitai (Enthusiast) November 27, 2020, 10:18am 21

Done !

Topic		Replies	Views
How to extract person names from user input? Rasa Open Source	12	3235	October 27, 2020
Get Person names as entity from user input Rasa Open Source	3	1086	February 1, 2021
Can I Extract Persons Name by not using SpaCy's "PERSON" entity, Because It has some problems Rasa Open Source	1	476	October 7, 2021
Name entity not extracting Rasa Open Source	22	5334	September 17, 2020
Extracting name entity from text Rasa Open Source	1	3401	October 22, 2019

Family name extraction

Related topics