How can i use max_event_history to avoid fetching large records from DB

We are running an app which uses Rasa on production from more than a year and we are using MariaDb for the DB store. Now for some users (sender_id) there are more than 4M+ records. Recently we have also seen the app taking more than 10GB+ of memory.

This could be happening because when the users with more than 4M+ records interact with our system we might be fetching all these 4M+ records into our app memory.

To solve this I’ve found that there is a field called max_event_history, but I’m not sure how to use this. Please guide me here as it is causing a huge problem in our production.

tracker_store = SQLTrackerStore(domain=domain, dialect="mysql", host=self.tracker_store_host,
                                port=self.tracker_store_port, db=self.tracker_store_db,
                                username=self.tracker_store_user, password=self.tracker_store_password)

This is how we are configuring tracker_store. Does adding

tracker_store.max_event_history = 100 after creating tracker_store

would work for me?

Rasa Version - 2.5.0

Hello,

Yes, adding tracker_store.max_event_history = 100 after creating the tracker_store object should limit the number of events stored in the tracker to 100, which should help reduce the memory usage of your app. max_event_history is a parameter in the Rasa tracker store configuration that determines the maximum number of events that will be stored in the tracker. By default, Rasa stores all events in the tracker, which can lead to high memory usage for users with a large number of interactions. By setting max_event_history to a lower value, you can limit the number of events that are stored in the tracker, which can help reduce memory usage. Keep in mind that setting max_event_history to a low value may result in some loss of data. If you set it too low, you may not be able to analyze the complete conversation history of users. You will need to balance the need for complete data with the need to reduce memory usage. In your case, since you have users with more than 4M+ records, setting max_event_history to a value such as 100 or 1000 should help reduce memory usage while still allowing you to analyze most of the conversation history. You can experiment with different values of max_event_history to find the optimal value for your use case.

Thanks for the reply, my another doubt is since that user already has 4M+ rows now, does settings max_event_history now will help me?.

Because in retrive function which RASA calls internally to fetch event history I don’t see max_event_history field getting used anywhere