Rasa model loading twice (equal to number of Sanic workers)

Rasa Open Source
1

Hi Rasa Team,

I am trying to use RASA with 2 Sanic workers and model is getting loaded twice. But in Rasa version 1.10.10 it used to load once. Can someone assist on this latest Rasa version please ?

root@mahesh-Latitude-5591:/home/ubuntu/rasa-demo# rasa run --debug

2020-12-28 10:22:40 DEBUG rasa.cli.utils - Parameter ‘endpoints’ not set. Using default location ‘endpoints.yml’ instead. 2020-12-28 10:22:42 DEBUG rasa.model - Extracted model to ‘/tmp/tmp2mpge2ge’. No chat connector configured, falling back to the REST input channel. To connect your bot to another channel, read the docs here: Connecting to Messaging and Voice Channels 2020-12-28 10:22:42 DEBUG sanic.root - CORS: Configuring CORS with resources: {’/’: {‘origins’: [’’], ‘methods’: ‘DELETE, GET, HEAD, OPTIONS, PATCH, POST, PUT’, ‘allow_headers’: [’.’], ‘expose_headers’: ‘filename’, ‘supports_credentials’: True, ‘max_age’: None, ‘send_wildcard’: False, ‘automatic_options’: True, ‘vary_header’: True, ‘resources’: {’/*’: {‘origins’: ‘’}}, ‘intercept_exceptions’: True, ‘always_send’: True}} 2020-12-28 10:22:42 DEBUG rasa.core.utils - Available web server routes: /webhooks/rest GET custom_webhook_RestInput.health /webhooks/rest/webhook POST custom_webhook_RestInput.receive / GET hello 2020-12-28 10:22:42 INFO root - Starting Rasa server on http://localhost:5005 2020-12-28 10:22:42 DEBUG rasa.core.utils - Using 2 Sanic workers. 2020-12-28 10:22:44 DEBUG rasa.model - Extracted model to ‘/tmp/tmpzvqabi6l’. 2020-12-28 10:22:46 INFO root - Enabling coroutine debugging. Loop id 97299880. 2020-12-28 10:22:46 INFO root - Enabling coroutine debugging. Loop id 97299880. 2020-12-28 10:22:48 DEBUG rasa.model - Extracted model to ‘/tmp/tmp6dk95qzr’. 2020-12-28 10:22:48 DEBUG rasa.model - Extracted model to ‘/tmp/tmpp4fo1fxg’. 2020-12-28 10:22:48 DEBUG rasa.utils.tensorflow.models - Loading the model … 2020-12-28 10:22:48 DEBUG rasa.utils.tensorflow.models - Loading the model …

RASA Version:

root@mahesh-Latitude-5591:/home/ubuntu/rasa-demo# rasa --version Rasa Version : 2.0.8 Rasa SDK Version : 2.2.0 Rasa X Version : None Python Version : 3.6.9 Operating System : Linux-5.4.0-58-generic-x86_64-with-Ubuntu-18.04-bionic Python Path : /usr/bin/python3

Thanks, Mahesh

1 Like
2

Hi Mahesh, this is expected as each Sanic worker runs its own process. I’m not sure that this was different in 1.10.10, in fact I think we only added the option to specify the number of Sanic workers in 1.4.0

3

Thanks Felicia. I have encountered an another issue that when we use 2 SANIC workers, process is hanging/stuck at below function in our custom connector file. We have redis for lockstore and trackerstore as well. For the first request it is stuck, but for next subsequent requests failing with “Message handling timed out for”. (Followed the link - Custom Connectors for custom connector)

                await on_new_message(
                    UserMessage(
                        asr_text,
                        collector,
                        sender_id,
                        input_channel=input_channel,
                        metadata=metadata,
                    )
                )

Will you be able to answer in this thread ? Or, should i raise another question instead of this thread.

4

You’re welcome! I can try to answer here, but nothing immediately comes to mind. I guess it’s possible you’re getting some sort of race condition. What channel is this a connector for?

5

Hi Felicia,

This is the rest channel and custom connector. Followed the steps mentioned as in below link.

I am unable to debug because when i use SANIC_WORKERS=2 in env variable, i cannot go inside on_new_message(). We are trying to hit only one request so i do not think there is a race condition.

Also please be informed that we are using Agent class to initiate the request. And, here when i set SANIC_WORKERS=1 works fine but when SANIC_WORKERS=2, it hangs on on_new_message

agent = Agent.load(parameters["core"], interpreter = nlu_interpreter, lock_store = lock_store, action_endpoint=act, tracker_store=track_store)
agent.handle_channels([chan], 5002)

Regards, Mahesh

6

@maheshjss08 Where exactly does it hang? Any more logs you should share with us? Maybe it’s some connection problem with the lock store? I remember we fixed a bug in Rasa Open Source 1.10.11 (rasa/CHANGELOG.mdx at master · RasaHQ/rasa · GitHub) which made some connection errors visible which weren’t visible before.

7

SANIC_WORKERS_Issue
SANIC_WORKERS_Issue1920×1080 351 KB

Hi Tobias, If you look at the image, we have a custom_connector in which it gets stuck at “await on_new_message()” line 619 in the image. If we use sanic_workers=1, it works fine but hangs there when we set sanic_workers=2.

It stuck at 09:51 and i waited until 09:56 for more than 4 mins but no luck.

Even in rasa 1.10.11 I faced the same issue ==>

rasa_1_10_11
rasa_1_10_111920×1080 380 KB

8

Did anyone able to figure out what is the cause for this ? Or, do we have any suggestion on how to spin off rasa agent in multi process with loading model only once.

9

@maheshjss08 I see you’re using Rasa Open Source from Python directly - did you try if it works via rasa run together with your custom connector?

10

Hi Tobias,

Yes, i exported SANIC_WORKERS=2 as an environment variable and ran below command.

rasa run --endpoints configs/endpoints.yml -m models/ --enable-api --connector main.custom_connector.TestConnector

It hung exactly at the on_new_message() function :frowning:

I have a questions that is tensorflow or keras stopping use of multi workers (SANIC_WORKERS=2)/multi threading or multiprocessing because of on_new_message() function is in async in nature ?

Another hint: When i stopped via keyboard interruption, error shows it stops because of Rasa’s ‘popen_fork’ in python…

2021-01-28 16:03:24,166 — custom_connector — INFO - 6845 - 140138355988288 — receive:610 — Before calling on_new_message !!

^CError in atexit._run_exitfuncs: Traceback (most recent call last): File “/usr/lib/python3.6/multiprocessing/popen_fork.py”, line 28, in poll pid, sts = os.waitpid(self.pid, flag) KeyboardInterrupt

Could the above error help ?

Regards, Mahesh