Hi I facing some issue in entity capturing. Does not capture incorrect words as entity

Rasa Open Source
1

My model in trained on sentence like he cooks food. When I test my model on sentence like he cook food, it does not capture the entities ‘cook’ without s. can you help me how to resolve this issue. How to take care of incorrect spelling sentences.

Here is my Config file.

language: “en”

pipeline:

  • name: “nlp_spacy” model: “en”
  • name: “tokenizer_spacy”
  • name: “ner_spacy”
  • name: “ner_duckling_http” dimensions:
    • “NUMBER”
  • name: “ner_crf” BILOU_flag: true features:

    features for word before token

    • [“low”, “title”, “upper”, “pos”, “pos2”]

    features of token itself

    • [“bias”, “low”, “upper”, “title”, “digit”, “pos”, “pos2”, “pattern”]

    features for word after the token we want to tag

    • [“low”, “title”, “upper”, “pos”, “pos2”] max_iterations: 50 L1_c: 1 L2_c: 1e-3
  • name: “ner_synonyms”
  • name: “intent_featurizer_count_vectors”
  • name: “intent_classifier_tensorflow_embedding”
2

you’d have to add a training example like that

3

Do I have to train on both sentences ?

  • he cooks food.
  • he cook food

Then it captures cook as entity ?

What about wrong spelling words ? Does it capture ? I think it has nothing with spelling. I do not know in my case it does not capture entity of my sentence.

4

Little advice, verbs are not entities, nouns are

verbs usually indicates the intent of the user, (I eat mangoes, I run 5 miles everyday) - eating and running are my intents while mangoes or 5 miles are my entities. When you do classification take note of these

5

can you give me any reference that verbs cannot be a verb ? that would be helpful.

6

I don’t understand what you mean, it is not a question of singular or plural.

When you create your classifier , it is important to understand grammatically what it means, after all it is natural language, not a system one

Your intents basically is the verb of the sentence " what is the user intends to do"

However entities are facts that supports your statement ( a user request )

An example would be

Souvik is going to Beijing 
(Noun - Souvik) 
(Verb - to go) 
( Noun - Beijing)

so your intent is to_go

Entity is Souvik and Beijing

This is a more constructed way of doing text classification and entity detection

7

yep that’s a good suggestion @souvikg10 @azizullah2017 could you provide an example of your training data?

8

Intent : sales Example : replace the airfilter of my car break pads Test Case : replace the airfilter of my car break pad

In test case it does not get the ’ break pad ’ as parts entity @akelad @souvikg10

my pipline:

language: “en”

pipeline:

  • name: “nlp_spacy” model: “en”

  • name: “tokenizer_spacy”

  • name: “ner_spacy”

  • name: “ner_duckling” dimensions:

    • “number”

  • name: “ner_crf” BILOU_flag: true features:

    • [“low”, “title”, “upper”, “pos”, “pos2”]

    • [“bias”, “low”, “upper”, “title”, “digit”, “pos”, “pos2”, “pattern”]

    • [“low”, “title”, “upper”, “pos”, “pos2”] max_iterations: 50 L1_c: 1 L2_c: 1e-3

  • name: “ner_synonyms”

  • name: “intent_featurizer_count_vectors”

  • name: “intent_classifier_tensorflow_embedding”