Entity extraction of custom entities -- symptoms

Rasa Open Source
1

Hello,

Rasa 2.0 question on extracting entities. I have an intent for when the user informs the assistant of their symptoms:

  • intent: enumerate_symptoms

    examples: |

    • i have [sore throat] (symptoms)

    • im experiencing [shortness of breath] (symptoms)

    • it’s [difficult to breathe] (symptoms)

    • i feel [pain in my chest] (symptoms)

and a lookup table in my nlu.yml file to improve entity extraction:

  • lookup: symptoms

    examples: |

    • shortness of breath

    • difficulty breathing

    • loss of speech

    • loss of mobility

    • confusion

    • chest pain

    • fever

    • cough

    • tiredness

    • loss of taste

    • loss of smell

    • sore throat

    • headache

    • aches

    …

but my assistant can’t seem to extract the entities with the config that I have:

pipeline:

  • name: WhitespaceTokenizer

  • name: SpacyTokenizer

  • name: RegexFeaturizer

  • name: LexicalSyntacticFeaturizer

  • name: CountVectorsFeaturizer

  • name: CountVectorsFeaturizer

    analyzer: “char_wb”

    min_ngram: 1

    max_ngram: 4

  • name: DucklingEntityExtractor

    url: “http://localhost:8000”

    dimensions: [“time”, “phone-number”]

  • name: CRFEntityExtractor

    features: [

    [“low”, “title”, “upper”],

    [

    "bias",

"low",

"prefix5",

"prefix2",

"suffix5",

"suffix3",

"suffix2",

"upper",

"title",

"digit",

"pattern",

    ],

    [“low”, “title”, “upper”],

    ]

    split_entities_by_comma:

    symptoms: True

  • name: EntitySynonymMapper

  • name: “SpacyNLP”

    model: “en_core_web_md”

  • name: “SpacyEntityExtractor”

    dimensions: [“PERSON”, “GPE”]

  • name: DIETClassifier

    entity_recognition: False

    epochs: 100

image
image710×616 15.7 KB

I’m wondering if there’s anything wrong with my pipeline? Thanks!

2

Yes @chiqui_hm . You have set the entity_recognition false under DIET classifier.

Remove that and it will work.

3

Hello, @ChikkaUdayaSai ! I changed it to True but still cannot extract the entities.

4

You can remove that entire line that contains entity_recognition. BTW, have you trained your bot after making the changes. If not, do that.

Let me know once you have finished them.

5

Hi @ChikkaUdayaSai. yes, I re-trained the model but still not working. is this what you meant by “removing the entire line that contains entity_recognition”?

  • name: DIETClassifier

    epochs: 100

If yes, it still didn’t extract the entity.

6

Hi @chiqui_hm,

Are you receiving any warnings when you run rasa train --debug? It could help if you share the output of that call.

7

Hi @kearnsw the model was trained successfully. there’s just this note:

c:\users\clarisse\documents\c19_venv\venv\lib\site-packages\rasa\utils\tensorflow\model_data_utils.py:197: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify ‘dtype=object’ when creating the ndarray attribute_features = {MASK: [np.array(attribute_masks)]} 2020-12-17 20:32:59 DEBUG rasa.utils.tensorflow.models - Building tensorflow train graph… 2020-12-17 20:33:08 DEBUG rasa.utils.tensorflow.models - Finished building tensorflow train graph. Epochs: 0%| | 0/100 [00:00<?, ?it/s]c:\users\clarisse\documents\c19_venv\venv\lib\site-packages\rasa\utils\tensorflow\model_data.py:587: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify ‘dtype=object’ when creating the ndarray final_data[key][sub_key].append(np.concatenate(np.array(f)))

8

Hello Everyone, I am facing the same issue…instead of following all the steps, Entities are not getting extracted. please help