Unable to pick name or location from my lookup table

[atwine (Mugume Twinamatsiko Atwine)]

Hello, I have my file as below but am not able to pick information from the lookup table. I want to pick different names and locations.

Issue:

When I train the model its not able to use the lookup tables am not aware what mistake I am making. Please advise.

Here is my rasa version:

Rasa Version      :         2.8.1
Minimum Compatible Version: 2.8.0
Rasa SDK Version  :         2.8.1
Rasa X Version    :         0.39.3
Python Version    :         3.7.5
Operating System  :         Darwin-19.6.0-x86_64-i386-64bit
Python Path       :         /opt/anaconda3/envs/rasaenv/bin/python

This is my config.yml file

language: en
pipeline:
- name: WhitespaceTokenizer
- name: RegexFeaturizer
  case_sensitive: False 
- name: RegexEntityExtractor
  case_sensitive: False
  use_lookup_tables: True
  use_regexes: True 
- name: LexicalSyntacticFeaturizer
- name: CountVectorsFeaturizer
- name: CountVectorsFeaturizer
  analyzer: char_wb
  min_ngram: 2
  max_ngram: 4
- name: DIETClassifier
  epochs: 200
  entity_recognition: False 
- name: EntitySynonymMapper
- name: ResponseSelector
  epochs: 200
- name: ResponseSelector
  epochs: 200
  retrieval_intent: faq
- name: ResponseSelector
  epochs: 200
  retrieval_intent: chitchat
# - name: DucklingEntityExtractor
#   dimensions:
#   - email
#   - number
#   - language
#   - location
#   - name
#   url: http://localhost:8000 
- name: FallbackClassifier
  threshold: 0.2
  ambiguity_threshold: 0.2
policies:
- name: MemoizationPolicy
- name: TEDPolicy
  max_history: 10
  epochs: 200
- name: RulePolicy

I am attaching the lookup files called name and location and nlu file.nlu.yml (14.8 KB) location.yml (1.6 KB) name.yml (18.5 KB)

[fkoerner (Felicia Koerner)]

Hi, what does your file structure look like? Perhaps the lookup tables aren’t being picked up? The location.yml file needs to be in the same folder (can be in a sub-folder of this folder) as specified using --data. If you haven’t specified anything, the default location is data. For example:

____data
    |___nlu.yaml
    |___lookups
        |___location.yml
        |___names.yml

[atwine (Mugume Twinamatsiko Atwine)]

Hello @fkoerner

Thanks for your response, the file structure is as below.

@fkoerner

____data
    |___nlu.yaml
    |___location.yml
    |___names.yml

[sdhaker2]

Hi @fkoerner i have same yml file locations in my data. So, i want to know how to define lookup files and where to define (like in nlu/domain). and I am using 2.8.9 version of rasa.

[fkoerner (Felicia Koerner)]

@atwine are you seeing any error messages? How do you know that the model isn’t using the lookup tables? Have you tested it using rasa test nlu?

[fkoerner (Felicia Koerner)]

@sdhaker2, you will need a RegexEntityExtractor in your pipeline, and also to define the entities in your domain (see here). You will need a file with the lookups, in the same directory as your nlu data. The format is described here

[sdhaker2]

Thank you @fkoerner for replying. I had mentioned lookup examples(like recipe name, ingredient) in nlu but didn’t mention lookup files because i dont where to mention those. So, please guide me where can i add lookup files.

[sdhaker2]

@fkoerner okay I will try with RegexEntityExtractor, btw currently I am using RegexFeaturizer. And yes I have already mentioned all entities in domain.

[atwine (Mugume Twinamatsiko Atwine)]

Hello @fkoerner

I found the solution, it was in the way I set up my pipelines, I decided to use spacy pipeline and crfentityextractor and it worked.

This is what it looks like now.

# Spacy Lookuptable pipeline
language: en
pipeline:
- name: SpacyNLP
  model: en_core_web_md
- name: SpacyTokenizer 
- name: RegexFeaturizer
  case_sensitive: False 
  use_word_boundaries: true #new 
- name: LexicalSyntacticFeaturizer
- name: CRFEntityExtractor
- name: CountVectorsFeaturizer
- name: CountVectorsFeaturizer
  analyzer: char_wb
  min_ngram: 2
  max_ngram: 4
- name: DIETClassifier
  epochs: 200
  entity_recognition: False #this should be turned off otherwise we will be getting entities twice
- name: EntitySynonymMapper
- name: ResponseSelector
  epochs: 200
- name: ResponseSelector
  epochs: 200
  retrieval_intent: faq
- name: ResponseSelector
  epochs: 200
  retrieval_intent: chitchat
- name: FallbackClassifier
  threshold: 0.2
  ambiguity_threshold: 0.1
policies:
- name: MemoizationPolicy
- name: TEDPolicy
  max_history: 10
  epochs: 200
- name: RulePolicy

[sdhaker2]

hey, @atwine where did you add those lookup files in code? I have looked at your nlu file but there you didn’t mention lookup files. I just wanna know how to mention lookup files in the code part. I already store those files under “data/lookups/name.yml”. It will be very helpful if you share a snapshot of your code where you mention lookup files.

[atwine (Mugume Twinamatsiko Atwine)]

@sdhaker2

Happy to help, I am attaching my current nlu for you to see.

I have two lookup tables, one for name and location.

nlu.yml (32.4 KB)

[fkoerner (Felicia Koerner)]

@atwine, very glad to hear it Happy bot building!

@sdhaker2, you don’t need to specifically point to the lookups file anywhere. As long as it is in the same location as the rest of your nlu data, and it has the proper format, it should be recognized just like any other nlu data. If you’re stuck on this, please share how you’ve set up your file structure. @atwine has a good example of how the format should look (the key is lookup)