Rasa Production container memory increase during load testing, but never decrease after load test

Rasa Open Source
1

Rasa version 1.10.20 when i up RasaX profile by docker-compose the memory (RAM) usage is around 5 GB but during our load testing the memory will increase up to 10-12 GB and never decrease (or decrease minor) after load test and next day during load test memory will increase from 10 GB to 13-14 GB and after load test will minor decrease

so i need to restart the Rasa every time, but i can’t restart Rasa at production

Note: i set replica = 8 in Rasa production service to increase the number of rasa production containers of quick response and when i check docker stats rasa production containers consuming a lot of memory

2

@Tobias_Wochinger can you please help me here

3

Have you tried whether the problem still persists with Rasa Open Source > 2? You test models via the API?

4

Thanks @Tobias_Wochinger for your time Yes i am testing with rest channel (API) - we’re working on our custom channel too

In Rasa 2.3

  • before load testing the memory consumption was ~ 4.6 GB
  • while ending load test the memory consumption was ~ 8.3 GB
  • and next day morning (after a night) the memory consumption was ~ 6.4 GB

yes in 2.3 the memory was decrease but not completely

so what’s your recommendation? should we migrate our all bots to 2.x my question is why rasa production containers memory increase so much? Rasa job is just answer a request

5

It would be amazing if you could create an issue here together with the exact HTTP request you’re using + versions. I don’t exactly know why this is happening but it looks like a bug to me.

6

@Tobias_Wochinger should i report issue related to 1.10.20 or 2.3.1 ?

7

2.3.1 would be great!