I’ve got a Rasa bot using 1.10.2, source here: GitHub - praekeltfoundation/healthcheckbot , that I’m having some difficulties scaling. We’re using PostgreSQL for our tracker store, and Redis for our lock store, and a custom connector ( GitHub - praekeltfoundation/turn-rasa-connector: A Rasa Connector for https://www.turn.io/ ), although we get the same performance using the Rest connector, so I don’t think that’s where the performance issue is. While doing some benchmarking, with our bot we are only able to process 10msgs/s per instance, with that instance at 100% CPU usage. The actions server, postgresql, and redis usage remains low during the benchmarking. Is this the kind of performance to expect, or are there ways that we can improve the performance of the bot? It seems very slow, and it will be very expensive to scale up this bot to production levels if we’re only getting 10msgs/s per cpu.

Doing some profiling, it seems like ~45% of the CPU time is being spent in the tracker store, that drops to around 40% using a local sqlite file. ~35% of the CPU time is being spent inside tensorflow, and the remainder inside misc python things, like the featurizer. [image]

Thanks for sharing your profiling results. Super interesting :pray: In which setting are you running these tests? Is it a single machine / K8s setup and what are the specs of the machines? To me it makes sense that inference is computational intense, but the tracker store definitely occupies too much time in there.

The profiling is run on my local machine, single instance, which is an i5 2014 MacBook Pro. For benchmarking, that is done in a k8s setup, using Azure D8s v3 VMs, Dv3 and Dsv3-series - Azure Virtual Machines | Microsoft Docs But I am getting similar enough performance results on my local machine, compared to running on the VMs, that I think it’s a good enough comparison. What I’m wondering is, is this the kind of per-core performance we should be expecting, or are there ways to increase the efficiency of the bot by tweaking some things, and if so, what are the things that we should look at tweaking?

@rudi I think you’ve been in touch with Mady from our team to set up a call about this before - are you still up for discussing this in a call?

@akelad The call wasn’t specifically for this, but happy to add this to the list of things. I’ll contact Mady to set up a time for a call.

Hi!, I came here also looking for Performance benchmarks and other kind of metrics. Our rasa instance is also experiencing some latency. How are you measuring the CPU consumption and other metrics per rasa component/function?

@MStanwood In terms of setting up the benchmark, here is some information about how I did that: turn-rasa-connector/benchmark at develop · praekeltfoundation/turn-rasa-connector · GitHub . It will differ depending on the connector you use. You can probably simplify it by using the REST connector, if you don’t want to benchmark the performance of your connector. In terms of profiling, I used python’s cProfile. So I ran my bot using something like: python -m cProfile -o out.profile /path/to/venv/lib/python3.6/site-packages/rasa/__main__.py run And then I used SnakeViz to plot the resulting profile, and investigate the results. snakeviz out.profile

Thanks… will definitely try the benchmark Our rasa doesn’t feel super slow… but for some utterances it does So we really need to investigate

It’s strange that it’s only for some utterances, I would expect the NLU, etc, to have consistent processing times. If it’s only happening for certain users, it could be the tracker store. Most tracker stores store and retrieve the entire history to process any message. The reason we chose the SQL tracker store is that it only retrieves history for the current session. Dialogue tracker gets too big If it’s happening for certain actions, especially for custom actions, then I would look what those are doing, and maybe benchmark and profile those actions to see where you can make improvements.

That’s definitely the next step… Taking a look to custom actions and get some metrics.

Performance of a Production bot

Rasa Open Source

mohangunta (Mohan) September 18, 2020, 6:29am 14

I am also facing this issue. I found that the issue is with post call made to action server.

Topic		Replies	Views
Long response time Rasa Open Source	6	1141	February 26, 2024
Rasa core performance issues Rasa Open Source	4	1354	June 1, 2020
Bot respond very slow/performance issue/lag? Rasa Open Source	3	2009	March 2, 2020
My bot is running slow very quickly Rasa Open Source	4	2617	March 2, 2020
Benchmark Rasa Performance Rasa Open Source	5	4292	July 17, 2019

Performance of a Production bot

Related topics