Rasa Core not making predictions based on previous slot values

Rasa Open Source
1

Hello everyone, first time posting here. I’m working on a chatbot using Rasa Core and I’m having an issue: The model predicts actions correctly based on stories, but it doesn’t seem to take into account already filled slots when predicting.

I’m gonna give a small example, keep in mind this is a VERY small example, and in reality I have 500+ stories in my training data, all of them suffering from the same issue.

## info Story on computers
* info_intent{"computer": "computer_val"}
  - slot{"computer": "computer_val"}
  - give_info_on_computer
  
## info Story on phones
* info_intent{"phone": "phone_val"}
  - slot{"phone": "phone_val"}
  - give_info_on_phone

## generic info story
* info_intent
  - generic_info_answer

This is how the stories look, when using them separately they work fine, the thing is, if I use, say, the computer info story, it will give me the right action and save the slot for computer as whichever value I gave it. If after this I use the “info_intent”, it gives me the generic_info_answer even though there IS a slot filled with a computer value from a previous turn. My understanding is that this is supposed to work.

The reason I’m assuming this should work is because:

  1. The docstrings for policies in the code indicate it: (https://github.com/RasaHQ/rasa/blob/master/rasa/core/policies/memoization.py)

Since slots that are set some time in the past are preserved in all future feature vectors until they are set to None, this policy has a capability to recall the turns up to max_history from training stories during prediction even if additional slots were filled in the past for current dialogue.

  1. This blog post How to Design Rasa Training Stories | Rasa Blog | The Rasa Blog | Rasa :

if the email slot is not populated, the assistant will ask for this information, but if the slot is already filled, then the assistant should be good to execute the action

For this I’m using AugmentedMemoization, Keras and Fallback policies if that makes any difference. My max_history parameters are 2 for AugmentedMemo and 3 for Keras.

Additionally, I understand that this problem can be solved by adding a story for it with 2 steps, but like I said, in reality my model is a lot bigger and I would have to great a new story for every single combination of situations where this might happen. I have also confirmed that the tracker is properly detecting and saving the slots throughout the conversation.

Am I missing something here? Is Rasa Core not supposed to be able to make these predictions based on slots filled from previous turns?

Thanks!

2

hi jorge - yes what you say is correct. However in the first turn your model receives two pieces of information, that the slot computer has been set, but also that the entity computer has been set. The model is probably overfitting to the enttity, which you can switch off

However it’s always a good idea to add more training stories with full, real conversations rather than relying on these snippets

3

Hello Alan! Thank you for your response.

I tried making the intent ignore entities with “info_intent: use_entities: []”. It seems to work the same way, predicts each story separately fine but after filling the slot, it still doesn’t make use of it in future messages.

In this case I want to avoid creating stories that will probably fix this since like I said, in reality I got 500+ stories and tons of intents/entities, meaning I would have to do that for every single intent/entity. These are all intents that produce one message as output and that’s it, I’ve always assumed that having single snippets of each case + the augmentation factor would be enough, is this not the case?

Let me know if there’s anything else I should try to get this to work, preferably without adding new stories if possible. Surprisingly, I can’t seem to find a single mention of people having the same issue as me online. :thinking:

Thanks for your help!

4

if you have a large number of intents to which the bot should just respond with a single message, that sounds like a great case for Integrate response retrieval models in assistants built with Rasa