RASA NLU not capturing synonyms correctly, resulting in wrong slot value

Rasa Open Source
1

Hello RASA community,

Excited to have my first post here.

I have a problem regarding entity extraction and slot value setting in RASA nlu. I need to ask the user whether his subscription model is based on a “flat-rate” or “actual” costs and provide information based on his response. So as outlined in the docs, I created a categorical slot and an entity in the domain.yml as follows:

slots:
  expenseType:
    type: categorical
    values:
    - actual
    - flatrate

entities:
- expenseType

And in the NLU traning data, my examples for this case look like this:

## intent:provide_info
    - I have a travel costs [flat-rate](expenseType:flatrate) agreement 
    - I only pay [actual](expenseType:actual) travel costs
    - [actual](expenseType:actual) costs
    - [actual](expenseType:actual)
    - [flat-rate](expenseType:flatrate)
    - [flatrate](expenseType:flatrate)

Additionally, I added synonyms there:

## synonym:flatrate
- flat-rate
- flat rate

I assumed that since I specified two possible categories, the value of the slot will always be one of the categories, and the provided info would be sufficient to learn the synonyms. But it makes a difference whether the user writes flat-rate or flat rate or flatrate and depending on writing style this is actually set as the slot value. For example in the case of flat-rate I get this warning:

WARNING:rasa_core.slots:Categorical slot 'expenseType' is set to a value ('flat - rate') that is not specified in the domain. Value will be ignored and the slot will behave as if no value is set. Make sure to add all values a categorical slot should store to the domain.

Since I do a look-up based on these slot values in a custom action, these action also fails since the value is not found in the table.

What do you recommend in order to only have slot values as specified by the categories in the domain? (Is the domain.yml actually used during NLU traning?).

Thanks! Martin

2

Hello there, sry, just pushing this topic again since I did not find any solution yet. Would be great to have someone’s opinion on that.

Best, Martin

3

Hi @martin.m sorry about not getting back to you yet, it’s been a busy few days :smiley: That’s very strange behaviour, my first question: is ner_synonyms included in your pipeline for NLU?

4

Hi Akelad, no worries :slight_smile:

So, yes, this is my pipeline:

pipeline:
- name: "nlp_spacy"                   # loads the spacy language model
- name: "tokenizer_spacy"             # splits the sentence into tokens
- name: "ner_crf"                  
- name: "ner_spacy"                   # uses the pretrained spacy NER model
- name: "intent_featurizer_spacy"     # transform the sentence into a vector representation
- name: "intent_classifier_sklearn"   # uses the vector representation to classify using SVM
- name: "ner_synonyms"                # trains the synonyms

I wonder if this is really so strange but caused by the NLU and Dialog part being separated. Does the NLU part actually see the domain.yml while training? It seems it will just extract the entities as specified in the nlu.md training data, without taking care what categories an entity can actually have.

Best, Martin

1 Like
5

yeah NLU has no knowledge of what Core is doing. But the way you’ve specified your training data it should be doing it correctly. Though, I just noticed the error message says flat - rate rather than flat-rate, could you try adding flat - rate to your synonyms?

6

Ok, I did this and it now can convert the cases specified in synonyms to “flatrate”. The problems with “flat- rate” or “Flatrate” persist, but I guess then I have to be more creative in the synonyms section.

But regarding your remark “NLU has no knowledge of what Core is doing”, then I wonder what the purpose of the slot types is? I mean I can specify categories, or boundaries for some float variable, but when the slot is filled with the value that NLU extracts from user input, what influence does this slot specification have?

Best, Martin

7

you would just have to add those synonyms then :slight_smile:

And different slot types are there to be able to influence the prediction of Core. If it’s unfeaturized, it won’t influence anything. If it’s a text slot, you can specify two different stories on whether a slot is empty or not. If you have a categorical slot you can specify different paths dependent on the exact value of the slot. Please read our docs for more info: https://rasa.com/docs/core/api/slots_api/#categorical-slot

1 Like