Be aware of your stories - stories not working correctly

Hi everyone,

We’ve been struggling with failing stories for a while now and we couldn’t get our heads around WHY they weren’t working. We were adamant that we set everything up correctly. Turns out, we didn’t. And it was all an NLU issue.

So let’s say we have the following story:

  • make_reservation
    • utter_here_is_your_reservation
  • can_you_park
    • utter_parking_opportunities

Then, in your NLU file you’ve got the following data:

intent: make_reservation

  • Can I make a reservation for 6:00 PM?
  • I want to reserve a table at 5:30PM etc…

That’ll be all right?

wrong.

Thing is, since the story doesn’t specify that the slot “time” is set in this use case, the story won’t work correctly. The pipeline will work as follows:

step 1: NLU recognizes intent and entity in the user message step 2: slot “time” is set to XX:XX step 3: rasa core Memoization steps in, doesn’t recognize this exact story step 4: Keras tries to identify which action to take and can either succeed (yay!) or fail (most likely)

the solution is to account for this in the story:

  • make_reservation{“time”:“6:00PM”}

    • utter_here_is_your_reservation
  • can_you_park

    • utter_parking_opportunities
  • make_reservation{“time”:“7:00PM”}

    • utter_here_is_your_reservation
  • can_you_park

    • utter_parking_opportunities

etc.

You can always make a custom slot that handles out of scope timeframes in this particular case of course.

@akelad @Dominik am I correct in typing this?

3 Likes

That’s correct, this is covered in our documentation as well: Story Data Format

Thanks for the response Akela!

Looking through the docs again, it still isn’t clear to me that the intent{entity} format (as described in the docs) actually fulfills this function.

The docs describe the following:

Messages sent by the user are shown as lines starting with `*` in the format `intent{"entity1": "value", "entity2": "value"}` . 

And nothing beyond this. Thus, it is stated that you fill your stories in this format, but it doesn’t become clear what this format actually does and how it affects your stories. Is this something to add to the Story Format page?

Yeah if you think it warrants some further explanation than we’d be happy to accept a PR! (sorry about the late response, I was travelling)

A what now? “I didn’t understand your question, please rephrase and my engineers will look into it” :smiley: :smiley: :smiley:

If you mean that I should write up a little explanation on how it works, then I’d be happy to deliver it to you. I’d just need info on the format, how you want me to write it down and stuff.

pull request :smiley: yeah exactly, i think it’d probably just be a sentence or two. The relevant file is here: https://github.com/RasaHQ/rasa_nlu/blob/master/docs/core/stories.rst#format

I’ll leave it up to you how you think is best to phrase it, we can always discuss adjustments in the pull request :slight_smile:

@akelad and @Remy: I have the same question regarding this:

inform{"location": "rome", "price": "cheap"}

In what situation should I include the ‘entity:value’ into the entity specification? What’s the difference between specifying it or not?

And the entity/value pair can be a random one? Because there could be many such entity/value pairs to speficify. Just supply one example helps the training?

Heya,

It depends on what type your slots are.

If both of those slots you mentioned are categorical it does depend on the value of your slot. This means that if you specify ‘Paris’ as a slot vs. ‘Rome’ as a slot, the bot will behave differently.

Consider this example: You want your bot to have different responses per city a user gives. Your user wants to book a trip to Paris and you want your bot to respond with ‘ah oui! Paris!’.

If you write a story like this:

## paris story
* inform
    - utter_ah_oui

Your bot will then respond with ‘ah oui’ regardless of what the user actually says. As long as ‘inform’ is recognized as the intent, the bot will respond with ‘ah oui’. This means that if the users responds with ‘Rome’, the bot will still say Ah oui! Paris!

Now, how do you solve this? Consider the following:

## paris story
* inform{"location":"Paris"}
    - utter_ah_oui

## rome story
* inform{"location":"Rome"}
    - utter_molto_bene

You see that you’re telling the bot to behave differently when the user specifies rome vs. paris.

Keep in mind: This only works when your slot for ‘location’ is categorical.

To summarize: specifying an entity and value in your intents means just telling the bot: only go to this story if the intent AND this entity value(s) is recognized.

1 Like

@Remy in addition, if you don’t care whether or not the entity is picked up for the dialogue flow, you can define the intent with the use_entities: false flag, i.e.

intents:
- make_reservation:
    use_entities: false

This will essentially make it so that your stories act like

make_reservation{“time”:“7:00PM”} OR make_reservation

any time that intent pops up. (although in this specific case, you probably do care that the time is picked up, as if it isn’t you probably want to ask the user for the time!)

@Remy, that’s the question: in such a case, typically there can be hundreds or thousands of city names.In such a case, in my story file, I need so many ‘inform’? Paris, Rome, Tokyo, Berlin, etc? Similarly, make_reservation{“time”:“7:00PM”}, what about reservations for other times? I also saw cases with ‘null’ value, like in the demo bot, form{“name”: null}.

Hey, sorry for my absence. I’ve been busy.

So, it’s true what you say, you’d need a lot of informs in that particular case. So: Make the ‘city’ slot a text slot. This makes it so your bot just recognizes there’s a city so it can respond. Then, you have 2 options:

  1. Draw from a lookup table in your NLU file so your bot can recognize the cities. This is the ‘simple’ solution that requires just 1 response.

  2. Make a bunch of different responses for cities and make a custom action that reads a response from a CSV file holding those responses.

How can we use multiple chatbot with single rasa I want to make weather bot and stack market bot two different bots no one should interrupt each other at a time.