# Lookup Table doesn't work

**URL:** <https://forum.rasa.com/t/lookup-table-doesnt-work/11039>\
**Category:** Rasa Open Source\
**Created:** [June 5, 2019, 6:48pm UTC](https://forum.rasa.com/t/lookup-table-doesnt-work/11039 "2019-06-05T18:48:11Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![sfurao](https://avatars.discourse-cdn.com/v4/letter/s/8491ac/32.png) [@sfurao](https://forum.rasa.com/u/sfurao)\
**Post date:** [June 5, 2019, 6:48pm UTC](https://forum.rasa.com/t/lookup-table-doesnt-work/11039/1 "2019-06-05T18:48:11Z")

</div>

Hi, im trying to use lookup tables but I’m not having sucess. This is my pipeline and data.

 ![rasaforum](https://europe1.discourse-cdn.com/flex013/uploads/rasa/original/2X/b/b2b5e8ea5da278ff7f23868816679e5b86733533.png)

Exemple of “lt\_filme” lookup file (filme.txt):

filme

filmes

um filme

o filme

de filmes

dos filmes

os filmes

When i parse q=movie de comédia i obtain this response from rasa nlu:

{

```
"intent": {

    "name": "A1M_SearchByGenre - Content&SearchMEO&V9.0",

    "confidence": 0.933992862701416

},

"entities": [

    {

        "start": 0,

        "end": 5,

        "value": "movie",

        "entity": "lt_filme",

        "confidence": 0.3769318939068408,

        "extractor": "CRFEntityExtractor"

    },

```

but movie is not even in the dataset or lookup files.

Someone can help or have the same problem? Thanks.

---

<div class="post-metadata">

**Author:** ![srikar\_1996](https://avatars.discourse-cdn.com/v4/letter/s/e274bd/32.png) [@srikar\_1996](https://forum.rasa.com/u/srikar_1996)\
**Post date:** [June 6, 2019, 12:42am UTC](https://forum.rasa.com/t/lookup-table-doesnt-work/11039/2 "2019-06-06T00:42:28Z")

</div>

Hello,

You seem to have very less training data. Adding more data would help. Make sure to add a few examples in the training data which are in the lookup table.

Also, as you can see, the confidence of **lt\_filme** is very less (0.37) and since **movie de comédia** is similar to **filme de comédia** , the CRF is identifying it as belonging to **lt\_filme** entity.

---

<div class="post-metadata">

**Author:** ![sfurao](https://avatars.discourse-cdn.com/v4/letter/s/8491ac/32.png) [@sfurao](https://forum.rasa.com/u/sfurao)\
**Post date:** [June 6, 2019, 9:17am UTC](https://forum.rasa.com/t/lookup-table-doesnt-work/11039/3 "2019-06-06T09:17:30Z")

</div>

Hi @srikar_1996 thanks for answering. I already tried with larger data but got the same result. I always put some of the lookup synonyms in the training phrases. So you are saying that it does not extract only when the word matches with the words in the lookup file but if it is a similar word it also extracts but with less confidence ?

For example when i parse q=“filme de dramático” i obtain this response from rasa nlu: ![print1000](https://europe1.discourse-cdn.com/flex013/uploads/rasa/original/2X/8/815bdd37e5bf03d03a3397854c961101290ff662.png)

and when i try q=“filmes de dramático”

 ![Capture1001](https://europe1.discourse-cdn.com/flex013/uploads/rasa/original/2X/6/683f532e09a5a1f3257216125c930d6611fae3f4.png)

These results do not seem to make sense because i have this words in the lookup files, and the extractor some times do not extract the word or extract with low confidence score.

Drama lookup file: ![Capturedarama](https://europe1.discourse-cdn.com/flex013/uploads/rasa/original/2X/6/6719e880ce622eff3a3e2e093b2870259825176d.png)

---

<div class="post-metadata">

**Author:** ![srikar\_1996](https://avatars.discourse-cdn.com/v4/letter/s/e274bd/32.png) [@srikar\_1996](https://forum.rasa.com/u/srikar_1996)\
**Post date:** [June 6, 2019, 10:14am UTC](https://forum.rasa.com/t/lookup-table-doesnt-work/11039/4 "2019-06-06T10:14:21Z")

</div>

> [@sfurao](#):
>
> So you are saying that it does not extract only when the word matches with the words in the lookup file but if it is a similar word it also extracts but with less confidence ?

Yes, something like that. This happens with my application as well but I do not use a lookup table. I’m not entirely sure if it’s the same case with lookup tables.

> [@sfurao](#):
>
> These results do not seem to make sense because i have this words in the lookup files, and the extractor some times do not extract the word or extract with low confidence score.

You have multiple entities which are very similar, probably that is the reason the bot extracts with less confidence. For example, lt\_dramat, lt\_comedia have a similar structure.

Can you show your complete nlu file?