Correct use of rasa slots

Rasa Open Source
1

Greetings!

I’ve been looking through the forums, and documentation, and can’t solve a problem I’m having with slots. I have a slot (and entity) called profile. It’s categorical, and has 3 values: super, standard and anonimous. I don’t have an initial value (so a /restart puts null in the slot.

According to the profile, rasa should show different answers, so I have something like this in my stories.md:

## support_view-all-ads-super
* support_view-all-ads
  - slot{"profile":"super"}
  - utter_support-all-ads-super

## support_view-all-adds-standard
* support_view-all-ads
  - slot{"profile":"standard"}
  - utter_support-all-ads-standard

## support_view_all_ads_anonimous
* support_view-all-ads
  - slot{"profile":"anonimous"}
  - utter_support-all-ads-anonimous

These work ok… I have them in separete “##” blocks otherwise I get multiple responses to the same intent.

I also have stories that don’t use a profile, that should respond with the same answer irrespective to profile:

## thanks
* support_thanks
  - utter_you-are-welcome

Even if I define the intent in my domain with { use_entities: None } or { use_entities: false }, I still get dropped back into fallback if I run /support_thanks. The debug shows that the intent is correcltly picked up, but if profile has a value, I get:

DEBUG   rasa.core.policies.memoization  - Current tracker state [{'prev_action_listen': 1.0, 'slot_profile_0': 1.0, 'intent_thanks': 1.0}]
DEBUG   rasa.core.policies.memoization  - There is no memorised next action
DEBUG   rasa.core.policies.mapping_policy  - There is no mapped action for the predicted intent, 'support_thanks'
DEBUG   rasa.core.policies.ensemble  - Predicted next action using policy_3_FallbackPolicy

The only case in which the “thanks” story works is if profile slot is null.

This seems to be related to the size of the stories file… It works for smaller files, but with a file size of around 32K I get the problem, I can have a stories.md and by adding a single simple story, it breaks the whole thing. Has anybody run accross this?

Also, I couldn’t find in the documentation that I needed to have an entity that matches the slot, but if I don’t, I get a warning that ‘entity_profile’ (value: ‘1.0’) was not found in the feature map. Is this correct?

2

How does your config.yml file look like?

How often do you use

* support_thanks
  - utter_you-are-welcome

in your stories? Does the action utter_you-are-welcome always follows the intent support_thanks? Or do you have stories that react with a different action after support_thanks?

3

Hi Tanya! Thanks for helping… here’s my config.yml… most stories only have on response (it’s a faq based system using Portuguese):

# Configuration for Rasa NLU.
# https://rasa.com/docs/rasa/nlu/components/
language: pt
pipeline:
- name: "EntitySynonymMapper"
- name: "WhitespaceTokenizer"
- name: "RegexFeaturizer"
- name: "CRFEntityExtractor"
- name: "CountVectorsFeaturizer"
  strip_accents: "unicode"
  analyzer: "char_wb"
  min_ngram: 1
  max_ngram: 4
  max_features: 1000
- name: "EmbeddingIntentClassifier"

# Configuration for Rasa Core.
policies:
  - name: MemoizationPolicy
    max_history: 1
  - name: KerasPolicy
    validation_split: 0.2
    batch_size: 32
    augmentation: 100
    epochs: 200
  - name: MappingPolicy
  - name: "FallbackPolicy"
    nlu_threshold: 0.8
    core_threshold: 0.8
    fallback_action_name: "utter_default"

And utter_you-are-welcome always follows support_thanks, and no other stories use it.

Cheers

4

Can you try adding AugmentedMemoizationPolicy to your policies? And you may also need to concatenate your stories, e.g.:

* support_view-all-ads
  - slot{"profile":"super"}
  - utter_support-all-ads-super
* support_thanks
  - utter_you-are-welcome
5

Thanks for your help again…

The thing is, I can’t be sure that support_thanks will follow on after a support_view-all-ads, I can have other intents in random order before I reach a “thanks”. This was an example;

Out of 235 intents, about 30% aren’t dependendant on the slot value, and I get to a point where all of them fail when the the stories.md reaches a certain size (around 33K). I wasn’t sure if I was specifying everything correctly in the domain, nlu and stories files.

I added the AugmentedMemoizationPolicy to the pipeline, but no matter where I placed it in the pipline, when I run rasa train I get the following error:

Exception: Cannot find class '{0}' from global namespace. Please check that there is no typo in the class

(This fails in rasa/nlu/components.py, line 38)

6

My Rasa is version 1.3.9

7

You need to add it to your policies not the pipeline. E.g.:

# Configuration for Rasa NLU.
# https://rasa.com/docs/rasa/nlu/components/
language: pt
pipeline:
- name: "EntitySynonymMapper"
- name: "WhitespaceTokenizer"
- name: "RegexFeaturizer"
- name: "CRFEntityExtractor"
- name: "CountVectorsFeaturizer"
  strip_accents: "unicode"
  analyzer: "char_wb"
  min_ngram: 1
  max_ngram: 4
  max_features: 1000
- name: "EmbeddingIntentClassifier"

# Configuration for Rasa Core.
policies:
  - name: AugmentedMemoizationPolicy
  - name: MemoizationPolicy
    max_history: 1
  - name: KerasPolicy
    validation_split: 0.2
    batch_size: 32
    augmentation: 100
    epochs: 200
  - name: MappingPolicy
  - name: "FallbackPolicy"
    nlu_threshold: 0.8
    core_threshold: 0.8
    fallback_action_name: "utter_default"
8

Silly me! Sorry about that.

I added it and trained the data. During testing, stories without “- slot” lines fell back on utter_default, if I have a value in the profile slot.

I am getting a warning that “Found policies [‘AugmentedMemoizationPolicy’, ‘MemoizationPolicy’] with the same priority 3 in PolicyEnsemble. When personalizing priorities, be sure to give all policies differente priorities. More information: …” during training and when I run rasa shell

9

I don’t want to get you off track since @Tanja is helping you debug this. BUT if there seems to be a size limit/problem, you can split out the stories.md into multiple smaller files. I actually do this for organization purposes not necessarily for size limits.

To do that, in your data directory create a core directory ( /data/core ) copy stories.md into that, and then split it out into smaller bits. The file names are arbitrary so pick names that make sense to you, maybe splitting stories by those profile names or something. Just make sure you keep the .md extension.

You can also split up your nlu.md file as well. Create /data/nlu and again copy nlu.md there and split it out naming them whatever you want, just with the .md extension.

If you want to get crazy, you can split up your domain.yml file - but Rasa doesn’t support doing that for some reason. Good news though, I wrote a script that will let you do this and merge them together into a single domain.yml file ( Rasa domain Assembler · GitHub )

Maybe that will help, maybe it won’t but I imagine the smaller files will make it easier to navigate and edit.

1 Like
10

Hi Jonathan… thanks for the suggestions. I tried splitting the files up, but it hasn’t made any difference so far.

11

Sorry, my bad, you should just use either AugmentedMemoizationPolicy or MemoizationPolicy . Please, remove the MemoizationPolicy and just use AugmentedMemoizationPolicy.

12

Update… thanks for all the help. I’ve been running lots of tests here…

I had been splitting the files and leaving them in the /data folder. Doing this, when the stories files hit around 33Kbytes total, the stories stop working consistently.

If I put them in a /data/core folder, as per Jonathan’s suggestion, the files worked consistently. Then I tried training our full set of stories (around 45Kbytes atm). Strangely, every three times I call the same intent, I get a fallback. It works twice, then the third time, I get a failure.

This was without changing the policy to AugmentedMemoizationPolicy. I’m trying that now and should have some results soon.

13

Keep me posted. We actually have a ton of NLU data that will be going into our files (not there yet).

I’m curious - do you have large NLU files too, or just large story files? Trying to prepare myself for future problems :slight_smile:

14

I was out of the office all day yesterday, and this morning I took a deep breath, wiped my models folder, added the AugmentedMemoizationPolicy and did a rasa train --force.

And… it looks like it’s working! I tried hammering Rasa with the same intent over and over again, and the problem I saw last week where it would fail every third repeat isn’t happening anymore. I had to do both what Tanja and Jonathan suggested. We’ll be testing a lot more over the next few days, but it’s responding perfectly so far.

At the moment, the domain file is around 117K; the NLU is around 54K and stories is running around 48K across three files. The NLU will grow a bit, because we have whole load of synonyms to integrate into the data.

Thanks for all the help!

2 Likes
15

That’s great news!

I’m hoping they cover policies in the Rasa Master Class You Tube series. I’d love to know why that one works better, other than it is “augmented” LOL

16

Hi @Tanja @jonathanpwheat @samscudder @JiteshGaikwad @amn41 I am new to rasa. Please help me with below issue. I have a slot “moved” which is of categorical type having values YES and NO. I dont have entity of same. My stories are

  1. *inform_address_update

    • utter_moving_confirmation

    *inform_moving_confirmation

    • slot{“moved”:“YES”}
    • utter_ask_yes

  2. *inform_address_update

    – utter_moving_confirmation

    *inform_moving_confirmation

    – slot{“moved”:“NO”}

    – utter_ask_no

My NLU file has following entity

##intent:inform_moving_confirmation

- YES

- NO

But “utter_ask_yes” or “utter_ask_no” are not getting executed. Please help. Also how to can I use the slot value in actions without declaring its entity?. If I only have slot and no entity how my nlu.md file should look like?. Please help

17

If you aren’t using entities, you have to have different intents for yes and no. You don’t even need to use a slot for this.:

In NLU:

## intent: inform_moving_yes
- yes
- yep
- sure
- ok
- right
- sounds good
- okey dokey

## intent: inform_moving_no
- no
- nope
- nah
- never
- cancel

The something like this in your stories.md:

## inform_address_confirm
* inform_address_update
  - utter_moving_confirmation
* inform_moving_yes
  - utter_ask_yes

## inform_address_deny
* inform_address_update
  - utter_moving_confirmation
* inform_moving_no
  - utter_ask_no

You may need to adjust your max_history in your config.yml.

18

Thanks @samscudder It’s working perfect. Can you please tell me how to can I use the slot value in custom actions without declaring its entity?. If I only have slot and no entity how my nlu.md file should look like? Do i need to map user responses to slot? If yes, how?. For example I want to ask user’s DOB and store it in slot “DOB”, without creating it’s entity. How do I do this?. I need to use that slot value in custom actions for validation. I don’t want to create entity of DOB. Please help @Tanja @jonathanpwheat @JiteshGaikwad @amn41

19

There are three ways to fill a slot… from entities extracted from the input text, through custom actions that set the slot, or through a form.

In a custom action, you need this:

from rasa_sdk.events import SlotSet

Then instead of return[] you need to return [SlotSet('slot_name', value)]. Return a value of None to clear a slot. If you need to set more than one, the object returned is an array, so just add to the end like this:

return [SlotSet('slot1', 123), SlotSet('slot2', 'aaaa'), SlotSet('slot3', None)]

If a slot is changed in an action, you should say in your stories. So if action action_clear_name clears the name slot, you need to have something like this in your stories whenever you call the action:

* how_can_i_help
  - utter_how_can_i_help
  - action_clear_name
  - slot{"name" : null }

I’m not using slots in forms without entities, so I’m not sure if you can fill a slot in a form without declaring the entity in the domain.yml. Have to test that and see.

In a form you can fill a slot from an entity (you say which intents to accept) or from “anything” the user types (using self.from_text(intent=intent_name)). This is defined in the slot_mappings method of your form class. A form is a loop that executes until you fill all the required slots or cancel the form. It will show utterances for any unfilled slots marked as required. The utterance asking for a value should be called utter_ask_ and the name of your slot.

20

thanks @samscudder