How to provide the lookup entities in .txt format

Hey I am getting the same issue. Its not recognizing any entities from the lookup txt file. Is there any solution or any other way of doing this?

try adding more examples from your .txt file to your nlu data. Sometimes it needs more examples to pick that table. I used ner_spacy for this purpose, it tends to serve me well.

@Akshit Any idea the least number of examples I should add? And also, How do I use ner_spacy? I mean where and how do I have to use it?

Sorry for late reply. I don’t know an exact amount as it depends on how much examples you have in table. I had a table of 50-60 names and I added 8-9 examples in my nlu data and it worked fine. For ner_spacy, you add it in your pipeline and It works very well with certain entities but I personally like duckling because of it’s date filter.

Thanks @Akshit. One more question, can we use both regex and lookup for the same entity ?

Do any one know how to attach .txt file with lookup in rasa 2.0

We can represent lookup tables like this in Rasa 2.0.

 - lookup: location
   files: |
     - location.txt
2 Likes

Hi sorry to interrupt the thread. I am having issues with providing lookup tables in .txt format. I tried the code as advised by @ChikkaUdayaSai but I get this error.

Any help will be greatly appreciated.

Thanks!

this leads to exception

in data/nlu.yml:3:
      Key 'lookup' was not defined. Path: '/nlu/0'
  in data/nlu.yml:3:
      Key 'files' was not defined. Path: '/nlu/0'

Hi @Jason73, and @susajsnair.

You no longer need to point to the file in nlu.yml, instead you can include the files in the same folder that your nlu.yml file is in (default is data), and rasa will search the subdirectories in data recursively.

@fkoerner can you provide a small example! if so it will be helpful. Thanks

Sure! For example, your lookup table could be a file like:

fruits.yml:

version: "2.0"
nlu:
  - lookup: fruits
    examples: |
      - apples
      - bananas
      - pears

Your nlu.yml should contain some examples that include the lookup table items. You don’t have to include every single lookup table item, just a few of them. You also do not have to mention fruits.yml or the lookup table in the nlu.yml file. Including it in the data directory is enough.

nlu.yml:

version: "2.0"
nlu:
- intent: ask_for_fruit
  examples: |
     - I'd like to buy some [apples](fruits)
     - Can I purchase some [pears](fruits)?

Finally, your directory structure should look like this:

data/nlu.yml
data/fruits.yml

Does that clear things up @susajsnair?

4 Likes

Thank you so much. You are awesome @fkoerner

1 Like

Thank you @susajsnair, glad I could help! :slight_smile:

1 Like

@fkoerner I follow the step you mentioned. But it seems the lookup table is not working. Could you help me?

course_code.yml:

image

nlu.yml:

image

Directory structure:

image

config.yml:

image

Hi @HaiiDD-creator, what exactly isn’t working? Are you getting an error message? Are the entities not getting picked up? Or something else?

I have DIETClassifier in my pipeline and since lookup works only with RegexEntityExtractor, I added RegexEntityExtractor into the Pipeline, But when I run RASA it always gives me an error (warning exactly, wihch eventually leading to error) due to conflict of entities extracted from two extractor. So, then i removed DIETClassifier, but my usual entity extraction isn’t working well. Then I made the lookup and training examples mutually exclusive. It’s working fine but I am not sure if I am doing right or is there any better way to do it. Please help me to find a better solution

I was making lookup for colors, and I didn’t add the colors in lookup table, that were already mentioned in training examples

if I add “blue” in my training example, and if my user input is " I would like to place an order for blu shoes" DIETClassifier could recognize “Blu” as color entity, but RegexEntityExtractor fails to do so

I would like to have an entity extractor or combination of extractors such that it works well with unseen data and lookup as well

Hi,

I’m Vincent and I maintain the rasa nlu examples project. I just added an issue on Github to explore ways of addressing this. I’m thinking about adding a NLU component that can do a bit of post-processing on all the detected entities. The working title for the component is EntityOrchestrator but there’s a couple of different ways of going about it.

If you’d like to give feedback on what would/would not work for you, I’d be all ears!

Hey! I want to use .txt instead of .yml files. What should I do?

Hi, you’ll need to convert the files to yml. See here for a script to help