Rasa performance

Rasa Open Source
1

Hi Folks, I’ve been doing some strees tests on a server running an rasa instance to subject rasa to extreme situations to find out its boundaries and behavior.

To do this I used the online tool RESTFUL STRESS2 where it is possible to perform stress tests through the HTTP protocol and check which values ​​are correct according to the number of requirements applied. So I used this tool to make HTTP requests equivalent to this:

curl -XPOST http://localhost:5005/webhooks/rest/webhook \
            -H 'Content-Type: application/json' \
            -d '{"sender":"default","message":"hello"}'

estrese100
estrese1001200×568 23.2 KB
estresse10
estresse101205×567 22.8 KB

48
48961×514 43.2 KB
44
44984×523 35.2 KB

10ketresse
10ketresse1160×564 37.4 KB
perfomance10k
perfomance10k1183×519 20 KB

As you can see the server stopped responding when reached about 1.5k requests. And the number of concurrent theads was limited to 7.

I am using a digitalocean VM with 1GB ram, 25GB disk and Linux 18.04 LTS x64. I am using rasa with docker and the docker-compose config is the following:

09

Also, when the server stopped responding I went to see the container logs and got this:

43
43766×425 59.1 KB

I’ve been searching for a while for content about rasa performance and found this issue where someone states that rasa can only handle about 40 concurrent user requests. But still I am not sure about it because I ran a JMeter test with those 40 theads and it worked just fine.

So, I want to know: Is rasa scalable ? How much concurrent and over time request it can handle?

Any new information is welcome!

1 Like
2

Hi @quixote15, Thanks a lot for running and reporting this test!

The number of concurrent requests a Rasa instance can handle will depend on your machine specs and on other parts of your setup, for example which database you’re using. Are you connecting to a postgreSQL server, or just SQLite? From your screenshot it looks like you haven’t defined a database in your docker-compose and are therefore using SQLite, but it would be great if you could confirm that.

Your machine specs with 1 GB of RAM is certainly at the very low end of the requirements. We usually recommend 8 GB of RAM with 4 GB minimum.

3

Firstly, thanks for your reply!! About the database I am using. Your guess was 100% right, its SQLite.

Also, I’m aware that my machine specs are very low but What if I used the recommend 8 GB of RAM with 4 GB minimum? What would the expected outcome be?

4

@quixote15, thanks for confirming that. To answer your question, I would expect a response rate of at least 60-80 messages a second if you increased your memory to 8 GB. I have measured around 100 per second on a machine with 16 GB RAM (mac os X)