I am using custom SQL event broker with postgres dialet for my bots. I have 50-60 bots running on different instances in production and every one is using the same db. And only for some messages, I’m getting an error of Error happened when trying to save conversation tracker to 'MongoTrackerStore'. Falling back to use the 'InMemoryTrackerStore'. Please investigate the following error: (psycopg2.OperationalError) SSL connection has been closed unexpectedly .
I have disabled the event broker for all and this error is gone suddenly. Want to know what is the reason behind this. If there is an issue related to event broker db, why is affecting TrackerStore DB ? I have attached the event broker code.
custom_event_broker.py (2.6 KB)
I haven’t looked at your custom event broker but the error message you posted isn’t directly related to the event broker. I’d be curious how often the tracker store error happens. If it’s intermittent, I would look into resource or networking issues.
I’m also concerned that you have 50-60 rasas instances using a single mongodb instance. These are 50-60 different models or the same model? I would suspect that this is a mongo resource issue and look more closely at mongo logs.
@stephens Thanks for the response. It doesn’t happen with any pattern. Some times it’s happening for external trigger. And sometimes it happens for normal triggers. And coming the models point… Yeah they are different models. Each server is connected to different number of backends from where the requests are coming to bot.
Also want to know one thing…Want to change tracker store to postgres and check how is it going. But before I put it in production, I want to get some alerts on slack or somewhere if some error happens in NLU server logs. How do I do it ?
Found the issue. We created rds proxy for postgres db and that proxy has idle connection timeout of 30 minutes. So if the connection is idle for 30 minutes, connection is getting closed.