Listing entities and set slots in stories - best practises

Rasa Open Source
1

I have a few questions regarding “best practises” for listing the entities or slots set in stories. Since each question itself would have relatively small answers, I wasn’t sure whether to have separate topics for each of them. In any case, let me know I’ll split this.

1.Does listing entities in stories affect conversation flow significantly? or do only slots do that? (Assuming that entities aren’t autofilling slots here).
For example, if I’m building a bot to suggest apartments based on number of bedrooms, budget and location, and I want to use stories for this instead of forms,

story: provide apartment details - number of bedrooms and location
steps:
- intent: ask_for_apartment_suggestions
  entities:
    - bedrooms: 4
    - location: new york
- action: action_ask_for_budget

story: provide apartment details - number of bedrooms and budget
steps:
- intent: ask_for_apartment_suggestions
  entities:
    - bedrooms: 4
    - budget: 500000
- action: action_ask_for_location

are these two stories always going to work conditionally based on which entities are detected, or would I always have to use slots for this kind of behavior?

2. Does mentioning value of set slots or entities make a difference when compared to just listing the entity or slot names?
Is this:

story: provide apartment details - number of bedrooms and location
steps:
- intent: ask_for_apartment_suggestions
  entities:
    - bedrooms: 4
    - location: new york
- action: action_entities_mentioned

the same as this?

story: provide apartment details - number of bedrooms and location
steps:
- intent: ask_for_apartment_suggestions
  entities:
    - bedrooms
    - location
- action: action_no_entities

3. if no entities are listed, does it mean that no entities are expected to be extracted?
Is this:

story: provide apartment details - number of bedrooms and location
steps:
- intent: ask_for_apartment_suggestions
  entities:
    - bedrooms: 4
    - location: new york
- action: action_ask_for_budget

the same as this?

story: provide apartment details - number of bedrooms and location
steps:
- intent: ask_for_apartment_suggestions
- action: action_ask_for_budget

Is the bot likely to prefer one over the other depending on whether entities are extracted or not?

4. Is it a good practise to write slot_was_set for each slot separately or can you group them together?
Are these two equivalent or is there a preferred convention?

story: provide apartment details - number of bedrooms and location
steps:
- intent: ask_for_apartment_suggestions
- slot_was_set:
    - bedrooms: 4
    - location: new york
- action: action_ask_for_budget

and

story: provide apartment details - number of bedrooms and location
steps:
- intent: ask_for_apartment_suggestions
- slot_was_set:
    - bedrooms: 4
- slot_was_set:
    - location: new york
- action: action_ask_for_budget

5. If slots are set to autofill, do you need to mention both entities and slots that are set in stories?
Assuming that bedrooms and location are both entities as well as slots, is this:

story: provide apartment details - number of bedrooms and location
steps:
- intent: ask_for_apartment_suggestions
  entities:
    - bedrooms: 4
    - location: new york
- slot_was_set:
    - bedrooms: 4
    - location: new york
- action: action_ask_for_budget

the same as this (in terms of both performance and convention/readability)?

story: provide apartment details - number of bedrooms and location
steps:
- intent: ask_for_apartment_suggestions
- slot_was_set:
    - bedrooms: 4
    - location: new york
- action: action_ask_for_budget

Thanks in advance.

4 Likes
9

Can someone at least direct me to where these things are discussed? Couldn’t find anything directly relating to this in the documentation.

@ChrisRahme, really sorry for tagging you this way, but its been two months without any help. Could you help me with this if you’re available? If not, maybe tag someone who can help with these things? I feel like minor things are hindering my confidence and understanding in using rasa! Thanks in advance!

10

Hey @Polaris000! No worries at all :slight_smile: Sorry you haven’t had an answer for so long.

I’ve seen your post before, and that’s a really good question! But I haven’t answered at the time because I was not (and still am not) sure about the answer. The best/truest answer I can come up with is: Try them and see for yourself (I don’t mean it in a mean way!).

You can check every pair of stories and compare them by seeing the different levels of confidence. You can also use Tensorboard or Confusion Matrices and Reports.

Remember to set the random_seed parameter in the DIET, ResponseSelector, and TED components in your pipeline so you can control the ML randomness and accurately compare the two models. You can look at this example.

I’ll try to answer with what I think might be the case.

Before answering each individual one, I want to point out an important common point between them and what it means: All these are stories, not rules.

Stories are flexible and show the bot what it can do when presented with a similar situation, while rules are strict and show the bot what it should do when presented with this exact situation.

During training, stories are mixed up (intelligently) among themselves to create some new stories, while keeping the original stories as well of course, like DNA.

The core of all of these questions is whether or not slots/entities are present and what values they have.

I would usually check for these in a custom action, since that way I’m sure I’ll get what I want and leave less room for ML randomness.

If mentioning these in stories/rules is as reliable as doing it inside of custom actions, then it just comes down to personal preferences.

These are the two I’m the least sure of.

Everything affects conversation flow, but I doubt doing these affect it significantly. Maybe their effect is even negligeable.

This one I’m sure of.

Nope! Entities will be extracted as you have defined them in the NLU data (intent examples).

I personally prefer doing them separately since this is how Rasa X exports stories.

Nope! No need to mention neither slots nor entities.

I never use entities and slot_was_set in stories. The only ones I have are those that were generated by real conversations. You can see some examples here.

1 Like
11

Thanks for the prompt replies @ChrisRahme! Regarding your answers:

OK. Got it. I had tried this before asking the question, but couldn’t get a definitive answer. Hopefully, someone else may have a better idea.

Sorry, I didn’t mean it in the sense that would entities be extracted or not–I know that NLU would do that, regardless of the stories.
My doubt here was that if no entities are mentioned in the stories, but they happen to be extracted from the user’s message, would the bot still be likely to go with that story (since entities do affect conversation)? Coming to my example, if entities are extracted (or not), would the two stories (one where entities are mentioned and the other where they are not) below still be equally referenced by the bot before making a prediction?

story: provide apartment details - number of bedrooms and location
steps:
- intent: ask_for_apartment_suggestions
  entities:
    - bedrooms: 4
    - location: new york
- action: action_entities_expected

story: provide apartment details - number of bedrooms and location
steps:
- intent: ask_for_apartment_suggestions
- action: action_no_entities

Got it. Thanks

This answer is what I find the most conflict with. Since while training Rasa doesn’t consult the action server (whether slots are set or not), I thought it was recommended to mention slots that were set under the slot_was_set key, though I can’t seem to find where I read that at the moment.

And though I’m not certain, I’m pretty sure I’ve noticed the negative effect of not mentioning slots in stories.

But I haven’t been able to tell the same thing about entities, and hence my question. If slots are being auto-filled anyway, I know that only mentioning entities is fine (the documentation says that), but would only mentioning slots work?

Once again, a big thanks for the help! I totally get what you’re saying when you say “try them and see for yourself”. In fact, for almost everything I ask on these forums, I’ve already experimented with, but couldn’t get a definitive answer, since what usually happens is:

  • smaller examples don’t have a lot of variation, so small changes like mentioning the values of entities vs not mentioning them, usually don’t affect the bot too much
  • more complex examples have many stories, entities and actions and so for any variation I do notice, it’s hard to conclude if it’s because of what I’m experimenting with or just random complexity issues (poor data, fewer stories, etc).
  • I’m generally looking for best practises in my questions (like the question on grouping slots under slot_was_set), since I generally can’t find these things in other forum posts on the Rasa documentation.

If you find the time, I clarified a few things in these answers in this reply, so maybe you can go through those.

Thanks!

1 Like
12

I think it should have an effect. Try it with both stories and rules and come back with your findings if you can :slight_smile:

You’re probably right, but I suggest not writing those yourself (this is also what the docs recommend, but I can’t seem to find where). Instead, write stories without considering slots and entities (unless the following action should be different), and let Interactive sessions generate more complex ones.

I totally get where you’re coming from since I think the same :slight_smile: I’m sorry I couldn’t come up with definite answers but I’m happy I could point out some stuff.

And I’m as curious about these answers as you are! If you or someone has a solution, please post here - I’ll do the same!

1 Like