Entity extraction duplicated using duckling and ner_crf pipeline

Rasa Open Source
1

Hi, I need help with a challenging situation I am facing. I made a pipeline where I am using ner_crf to extract words like cities (locations) and it was the way I found to extract that properly. the caveat of using this with duckling is: I am also extracting time and phone-number for example, in the action_listen I am writing a phone contact like this my number is +34 609 543 888

? The bot wants to run 'action_listen', correct?  Yes                                                                                                                         
? Next user input (Ctr-c to abort): +34 609 543 888                                                                                                                           
? Is the NLU classification for '[+34 609 543 888](phone-number:(+34) 609543888)' with intent 'contact' correct?  (Y/n)

the slot only capture “phone-number:(+34)” and ignore 609543888) piece.

ideas on how to fix this situation?

The other challenging situation is extracting **time**

for some reason (I don’t know) the use of both ner_crf and duckling duplicate the extraction :frowning:

el dia [10 de febrero a las 9 am](time)[10 de febrero a las 9 am](time:2019-02-10T09:00:00.000-06:00)

the first extraction is ner_crf the second is the familiar duckling extraction.

Any clue about how I can prevent this duplication and getting the wrong phone-number? I remove the ner_crf from my pipeline and everything works great with the exception of the location extraction, which stop to work. Then I had to place it again.

I will appreciate you input folks. Thanks

2

for the duplication issue,

you don’t need to annotate entities in your training data that you want to catch with duckling, case time

Perhaps due to annotation of phone number and time, the ner CRF captures the info places them into the slot

try to remove the annotations of entities that you want to capture from duckling

3

Thank you @souvikg10 I will give a try removing the entities from my nlu.md I will let you know

4

@souvikg10 , I did the changes. Basically, I removed all entities from my NLU training data which are captured by duckling or defined in their dimension parameter. After that, I trained and test the bot, NO MORE DUPLICATION a BIG THANKS

I wasn’t aware and I probably misread the docs.

Thank you very much for your help

1 Like