# How to reduce memory footprint in rasa open source? **URL:** \ **Category:** Rasa Open Source\ **Created:** [February 28, 2023, 11:53am UTC](https://forum.rasa.com/t/how-to-reduce-memory-footprint-in-rasa-open-source/57381 "2023-02-28T11:53:07Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![faraz](https://avatars.discourse-cdn.com/v4/letter/f/278dde/32.png) [@faraz](https://forum.rasa.com/u/faraz)\ **Post date:** [February 28, 2023, 11:53am UTC](https://forum.rasa.com/t/how-to-reduce-memory-footprint-in-rasa-open-source/57381/1 "2023-02-28T11:53:07Z")
Problem Statement: I want to split pipeline processes across different servers to reduce the load. Proposed Idea: ![rasa](https://europe1.discourse-cdn.com/flex013/uploads/rasa/original/3X/1/4/143fc4c2811d6a9b991eccb8bc95c32de353525f.jpeg) Even with the initially trained rasa model it eats up ~1.5gb ram, apparently this was happening because of how the model is implemented via tensorflow. 1. Does that mean there is no way to run multiple instances of rasa on the server without hogging up ~1.5gb ram per instance? 2. If I train rasa nlu only then can I run rasa core only(i couldn’t find a command yet to use core only) to reduce that ram usage? 3.Is there some other way where I can reduce the memory to say around 500mb per instance? Thanks ---
**Author:** ![anoopshrma](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/anoopshrma/32/13349_2.png) [@anoopshrma](https://forum.rasa.com/u/anoopshrma)\ **Post date:** [February 28, 2023, 1:34pm UTC](https://forum.rasa.com/t/how-to-reduce-memory-footprint-in-rasa-open-source/57381/2 "2023-02-28T13:34:53Z")
If you want to run multiple rasa models at once, You can try handling Rasa models via Python. For example create a Python server in a framework of your choice and load all the models. I think the only time memory size would increase is during the processing of the query. An example of how you can train and deploy Rasa models is shown here [Rasa NLU models in Python](https://pythonwarriors.com/how-to-create-rasa-nlu-model-and-use-it-in-python/) \*\* > Note that the blog is for handling NLU models but you will get an idea how you can load your complete model as well. \*\* ---
**Author:** ![faraz](https://avatars.discourse-cdn.com/v4/letter/f/278dde/32.png) [@faraz](https://forum.rasa.com/u/faraz)\ **Post date:** [March 1, 2023, 11:55am UTC](https://forum.rasa.com/t/how-to-reduce-memory-footprint-in-rasa-open-source/57381/3 "2023-03-01T11:55:01Z")
Thank you for the response I have looked at the given code and will try it out meanwhile I would like to know what is the standard hardware recommendation from rasa to serve multi client from a single server. (If recommended) **Use case: To serve multiple clients of cross domain on a single server.** How do players in the industry currently handle multiple clients? ---
**Author:** ![anoopshrma](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/anoopshrma/32/13349_2.png) [@anoopshrma](https://forum.rasa.com/u/anoopshrma)\ **Post date:** [March 2, 2023, 7:22am UTC](https://forum.rasa.com/t/how-to-reduce-memory-footprint-in-rasa-open-source/57381/4 "2023-03-02T07:22:43Z")
> [@faraz](#): > > I would like to know what is the standard hardware recommendation from rasa to serve multi client from a single server. (If recommended) Currently Rasa does not provide feature to handle multiple models at once, And best way that I found to go over this problem statement is creating a python application as Rasa library is available only in python and is very easy to implement ---
**Author:** ![Stepsvd](https://avatars.discourse-cdn.com/v4/letter/s/898d66/32.png) [@Stepsvd](https://forum.rasa.com/u/Stepsvd)\ **Post date:** [March 14, 2023, 8:22am UTC](https://forum.rasa.com/t/how-to-reduce-memory-footprint-in-rasa-open-source/57381/5 "2023-03-14T08:22:41Z")
Hi Anoop, is there a difference in how Rasa Open source vs Pro/Enterprise consume the memory footprint.? Our development team has been training a conversational bot and we have upgraded to 1 tb memory on AWS infrastructure that still fails.? it just seem very excessive and not sure if it has to do with the open source vs “other” versions? it has fairly complex story blocks with checkpoints and intents but unsure.? thanks ---
**Author:** ![anoopshrma](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/anoopshrma/32/13349_2.png) [@anoopshrma](https://forum.rasa.com/u/anoopshrma)\ **Post date:** [March 14, 2023, 8:48am UTC](https://forum.rasa.com/t/how-to-reduce-memory-footprint-in-rasa-open-source/57381/6 "2023-03-14T08:48:25Z")
Hi @Stepsvd , > [@Stepsvd](#): > > is there a difference in how Rasa Open source vs Pro/Enterprise consume the memory footprint.? For this, You’ll have to connect with Rasa Enterprise team as they will be able to tell you about this in a much better way. Also is it just one model that’s taking 1TB memory ? ---
**Author:** ![Stepsvd](https://avatars.discourse-cdn.com/v4/letter/s/898d66/32.png) [@Stepsvd](https://forum.rasa.com/u/Stepsvd)\ **Post date:** [March 14, 2023, 9:50am UTC](https://forum.rasa.com/t/how-to-reduce-memory-footprint-in-rasa-open-source/57381/7 "2023-03-14T09:50:48Z")
Thanks Anoop yes just 1 model, is there someone on ENT team perhaps you can refer.? ---
**Author:** ![siriusraja](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/siriusraja/32/22306_2.png) [@siriusraja](https://forum.rasa.com/u/siriusraja)\ **Post date:** [March 14, 2023, 1:41pm UTC](https://forum.rasa.com/t/how-to-reduce-memory-footprint-in-rasa-open-source/57381/8 "2023-03-14T13:41:53Z")
@Stepsvd Just curious to know the below data points. 1. what is the volume of training data - Number of intents , entities & training examples 2. what is the compressed size of model ? 3. Is it taking up 1 TB RAM when the service is started. we have been serving multiple clients via different models as different services on the same EC2. But the size is way smaller than yours. ---
**Author:** ![faraz](https://avatars.discourse-cdn.com/v4/letter/f/278dde/32.png) [@faraz](https://forum.rasa.com/u/faraz)\ **Post date:** [March 15, 2023, 4:54am UTC](https://forum.rasa.com/t/how-to-reduce-memory-footprint-in-rasa-open-source/57381/9 "2023-03-15T04:54:43Z")
Hi @siriusraja, are you serving multiple clients through single EC2 server? ---
**Author:** ![anoopshrma](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/anoopshrma/32/13349_2.png) [@anoopshrma](https://forum.rasa.com/u/anoopshrma)\ **Post date:** [March 15, 2023, 11:14am UTC](https://forum.rasa.com/t/how-to-reduce-memory-footprint-in-rasa-open-source/57381/10 "2023-03-15T11:14:43Z")
Exactly @siriusraja, I am curious on these points as well. As none of the models that i have created got this big or started consuming this much memory. I have one more question > [@siriusraja](#): > > we have been serving multiple clients via different models as different services on the same EC2 How are you serving multiple models ? - **Different Rasa servers**? but this case would create so many ports and endpoints and rasa services - **Or handling multiple models via python** : This is what i have done in my project [handling multiple rasa models](https://pythonwarriors.com/how-to-create-rasa-nlu-model-and-use-it-in-python/) ---
**Author:** ![siriusraja](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/siriusraja/32/22306_2.png) [@siriusraja](https://forum.rasa.com/u/siriusraja)\ **Post date:** [March 15, 2023, 5:54pm UTC](https://forum.rasa.com/t/how-to-reduce-memory-footprint-in-rasa-open-source/57381/11 "2023-03-15T17:54:54Z")
Hi @anoopshrma we have starting rasa services on different ports for each client which uses the respective model. Ofcourse, this is approach would end up opening multiple ports. But that manageable. If you want, you can put a nginx like proxy which can listen on single port and internally route to different client services. ---
**Author:** ![siriusraja](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/siriusraja/32/22306_2.png) [@siriusraja](https://forum.rasa.com/u/siriusraja)\ **Post date:** [March 15, 2023, 5:57pm UTC](https://forum.rasa.com/t/how-to-reduce-memory-footprint-in-rasa-open-source/57381/12 "2023-03-15T17:57:05Z")
@faraz Yes, we have multiple clients but the concurrent traffic is not high. Its spread across different hours of the day. So we were able to run multiple services servicing different clients. Each client model takes upto 1 - 1.5 GB of RAM. ---
**Author:** ![anoopshrma](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/anoopshrma/32/13349_2.png) [@anoopshrma](https://forum.rasa.com/u/anoopshrma)\ **Post date:** [March 15, 2023, 6:14pm UTC](https://forum.rasa.com/t/how-to-reduce-memory-footprint-in-rasa-open-source/57381/13 "2023-03-15T18:14:57Z")
Hi @siriusraja But will it be easy if you have to handle 100 clients (let say) at once. That would mean to manager 100 rasa services. I’m just curios, will it not be good if you could handle models at one place ? Just want your thoughts on this ---
**Author:** ![siriusraja](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/siriusraja/32/22306_2.png) [@siriusraja](https://forum.rasa.com/u/siriusraja)\ **Post date:** [March 16, 2023, 9:16am UTC](https://forum.rasa.com/t/how-to-reduce-memory-footprint-in-rasa-open-source/57381/14 "2023-03-16T09:16:38Z")
@anoopshrma Its a trade off between administrative overhead of managing multiple services & flexibility. When i run as different services, I can start / stop the services for each client when the business hours start in their time zone. This would reduce the memory overhead in te server. Do you see any benefits in terms of Memory / CPU savings in the single service approach ? ---
**Author:** ![khangnd](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/khangnd/32/24836_2.png) [@khangnd](https://forum.rasa.com/u/khangnd)\ **Post date:** [November 15, 2024, 2:34am UTC](https://forum.rasa.com/t/how-to-reduce-memory-footprint-in-rasa-open-source/57381/15 "2024-11-15T02:34:11Z")
Hello sir @siriusraja , we are also deploying our service like in your case. However, since every customer has been spam creating new bot with their account, we have been struggling to keep the resource consumption under control. Is there a way to load the pre-trained language model as an internal server to parse features to other components, instead of loading it for each and every bot instance?