We’re using rasa (former Rasa core) with default sync rest endpoint (set by specifying rest key in credentials.yml). we’ve also made custom pipeline intent component (intent classifier). Our intent classifier uses large pretrained multilingual embedding wrapped into micro-service. To perform a request, rasa makes a POST request to embedding micro-service and blocks the main thread (main event loop). So we can only have one concurrent flow. The increasing number of concurrent connections leads to increasing message processing time. As we can see here rasa/components.py at 55caf93c75adf6e23273d16c0d21dd5acca58a39 · RasaHQ/rasa · GitHub parsing of a message declared is a synchronous function. So we can’t redeclare it as an async one in Rasa standard NLU interpreter. Can we handle messages in an async manner in Rasa Enterprise? Or we have to make our own NLU interpreter, Agent and so on?
Hi @KirillBarabanov, welcome to the forum!
As you say, the generic Component's process() method is currently only synchronous, so at the moment there’s no way to run async code out of the component.
In the meantime, you could try the following: The rest endpoint is asynchronous and should not be blocking, so one option might be to move your async call into the REST endpoint itself, and attach the result to the UserMessage's metadata. Your component can then in its process() method retrieve that result and set the intent.
Alternatively, if you run Rasa with a RedisLockStore, you can replicate your rasa microservice and handle more concurrent users.
Let me know if either of these work for you!