Benchmark Rasa Performance

[lahsuk]

Hi @twittmin,

I don’t think the history size effects performance (it only effects prediction).

The numbers of active users it can support depends on their conversation length. So, for very short conversations it can support 20+ users (depends on other factors too including custom actions) and for long conversations, it will struggle to support even single user.

And no, bombarding it with 10,000 sentences is not similar to multiple users talking to the bot simultaneously. It is just a single user conversing with the bot.

And, looks like other people have faced this issue too:

Rasa performance for increasingly large trackers

We are currently building a bot using rasa_core==0.14.4 . During testing we notice considerable slowdown, to the point that it is no longer usable, when having longer conversation. Typically, this happens after around 20 rounds, in our bot this corresponds to 80 events. After profiling our bots we came to the conclusion that the constant (de)serializing and recreating of the tracker objects is the current bottleneck. Looking at the code, it seems that the processing becomes slower as more event…