FutureWarning: Directly including lookup tables as a list is deprecated since Rasa 1.6. regex_pattern = self._generate_lookup_regex(table)

Rasa Open Source
1

I am using Rasa version 1.10.9 and i want to use the lookup table feature for entity recognition. My config.py file:

language: en
pipeline:
  - name: SpacyNLP
    model: en_core_web_md
    case_sensitive: False
  - name: SpacyTokenizer
    intent_tokenization_flag: False
    intent_split_symbol: "_"
  - name: SpacyFeaturizer
  - name: RegexFeaturizer
  - name: LexicalSyntacticFeaturizer
  - name: CountVectorsFeaturizer
  - name: CountVectorsFeaturizer
    analyzer: "char_wb"
    min_ngram: 1
    max_ngram: 4
  - name: DucklingHTTPExtractor
    url: "http://localhost:8000"
    dimensions: ["number"]
  - name: RegexFeaturizer
  - name: CRFEntityExtractor
    features: [
               ["low", "title", "upper"],
               ["bias", "low", "prefix5", "prefix2", "suffix5", "suffix3","suffix2", "upper", "title", "digit", "pattern"],
               ["low", "title", "upper"]
              ]
  - name: DIETClassifier
    epochs: 100
  - name: EntitySynonymMapper
  - name: ResponseSelector
    epochs: 100

My nlu file contains:

## lookup:food
- data/food_list.txt

First issue is that why is it giving me this future warning when i train rasa nlu: FutureWarning: Directly including lookup tables as a list is deprecated since Rasa 1.6. regex_pattern = self._generate_lookup_regex(table)

Is there a new way to specify the lookup table since version 1.6?

and further after training i do rasa shell nlu and i give “Chicken Chilli Maggie” which is present in my lookup list, DietClassifier does entity recognition and not the CRFEntityExtractor and recognizes only “Chicken Chilli” and not the whole entity

How to make rasa use the lookup list? Am i missing something? Also how can Rasa give preference to one entity extractor over other? like in this case i want preference given to entity recognized by CRFEntityExtractor and if it misses some entity then fallback on DietClassifier for entity extraction. Is it possible?

2

I think you need to remove the - in front of your lookup table file. Can you try the following?

## lookup:food
data/food_list.txt

Does it work?

If you have multiple entity extractors in the pipeline, the entities detected by them are merged. So all are considered to be equally important.

3

Yes I did that… it worked after that

4

In Rasa 2.0 the lookup table example (NLU Training Data) is of putting lookup table directly into the NLU training file. Is there a way to refer to the lookup table as a file name, like it was in Rasa 1.x ?

5

Hey @lgrinberg

In 2.0 the lookup tables don’t refer to file names anymore. You can put then as a separate file with only lookup tables inside, and leave a comment in another file if that’s necessary though.

6

So what would be the format to put a reference to the lookup table file inside nlu.md? How do I tell the NLU model to use a particular lookup table for a particular entity?

7

Did you get solution?