How to efficiently store conversations of rasa bot

Rasa Open Source
1

Hi,

I have tried saving rasa conversations using custom MongoDB store. But with more number of users the document size increased drastically.

if i want to store every conversation with the user what is the best way to do.

2

Hi @ojas, it’s pretty common at this stage to introduce an Event Broker to your architecture. This will allow you to forward the conversation events to another DB that you can build for longer term storage, and have a plan to clean Mongo as your Tracker store every so often. This also helps with ensuring your analytics queries are not impacting production usage of the Tracker store.

3

Thanks for the reply @desmarchris !

I am currently using Custom Tracker Store with MongoDB. The problem i am facing is with the size of document stored in MongoDB. While using CustomTrackerStore with every event two mongo queries conversations.find_one_and_update() and conversations.update_one() gets executed. My avg. document size reaches ~35MB that results in delay in fetch and save options thus effecting overall response time.

I want to clear the events data of a user(particular sender_id) when chat session ends from my end. So that if user again starts chat from the beginning then previous event data must not become an overhead.

Just like /restart which clears slots data whenever we pass it. There must be something that clears event data when required thus reducing a lot of overhead.

Please suggest a better way to achieve this.

Thanks in advance!

1 Like
4

You could look at running a custom action at the end of each conversation to run delete the record using this, but I’m curious what your bot is doing that is creating that large of documents? There may be other solutions to look at than just deleting the document.

5

Thanks @desmarchris!

This large document is created when users interact multiple times with the bot. It’s like chatting again and again and that results in large event data which is being stored in mongodb.

I have tried deleting the event data but that is affecting my stories i.e. flows