I currently have a Rasa bot that has a lot of categorical slots set automatically throughout conversation. For example, I have a topic categorical slot that captures the user’s currently selected topic, or more straightforward user_profile_update_needed boolean slot.
There’s a large number of these kinds of slots and I’m starting to run into issues where the bot loses accuracy in responses, I’m guessing in large part because it’s starting to encounter conversations that didn’t have those slots set. I’m currently using the AugmentedMemoizationPolicy to leverage the “forgetting” mechanism to help handle longer form conversations, but I feel as though the slots are starting to take a larger role as the reason for the bot losing accuracy over time chatting with the bot.
That being said, I’m still on rasa 1.10.7 and on the KerasPolicy, so it could simply be a matter of using a difference ML policy or increasing data augmentation.
Are these slots used for prediction purposes in stories, or just to store data that you use to formulate the proper response?
If you use them for prediction purposes, maybe you can look into using a custom action that resets the slots.
If you are not using them for prediction purposes, you should change all slots to unfeaturized slots, so they don’t clutter up the tracker and confuse the bot.
The slots are used for prediction purposes, so I would need to keep them featurized. I was thinking there’s a need for the custom action to reset the slots, so I feel a little reassured, though my concern is that adding in that custom action into the more than 1000 stories I have in my bot would lead to additional prediction issues. I was thinking I could run it as an external action on a schedule (something like every 15 or so minutes as that’s when the prediction issues start) in my custom channel and use a UserUtteranceReverted() to prevent that from being a problem.
Yeah, I would not add a new custom action just to reset slots, but do it as part of the flow. For example, if you have stories that predict utter_something based on a slot being set, change that into action_something, which does both the uttering and then resets that particular slot.
So, you would keep the 1000 stories exactly as they are right now, but you swap out the utter_... responses with action_... custom actions, and on-the-fly reset those slots.