Failure when

Run the instance using rasa run and call the training method of rasa through the HTTP API. From the logs, it can be seen that the model has been trained and stored in the/tmp/tmp * * directory, but through callback_ URL retrieval fails. Query the/tmp directory and find that the directory where the Generative model was just generated does not exist.

Will rasa automatically delete temporary model files and their directories?

The log is as follows:


2023-07-31 09:26:04,976 [DEBUG] Start to created model package for path ‘/tmp/tmpo_10b8sb/nlu-20230731-091555-furious-caulk.tar.gz’.
2023-07-31 09:26:18,808 [DEBUG] Model package created in path ‘/tmp/tmpo_10b8sb/nlu-20230731-091555-furious-caulk.tar.gz’.
2023-07-31 09:26:19,077 [DEBUG] Starting new HTTPS connection (1): api.segment.io:443
2023-07-31 09:26:19,778 [DEBUG] https://api.segment.io:443 “POST /v1/track HTTP/1.1” 200 21
2023-07-31 09:26:19,810 [DEBUG] Starting new HTTPS connection (1): api.segment.io:443
2023-07-31 09:26:20,446 [DEBUG] https://api.segment.io:443 “POST /v1/track HTTP/1.1” 200 21
2023-07-31 09:26:20,732 [DEBUG] Asynchronous processing of request was successful. Sending result to callback URL.
2023-07-31 09:31:21,266 [ERROR] Task exception was never retrieved
future: <Task finished name=‘Task-7’ coro=<async_if_callback_url..decorated_function. .wrapped() done, defined at /root/miniconda3/envs/rasa/lib/python3.10/site-packages/rasa/server.py:554> exception=TimeoutError() created at /root/miniconda3/envs/rasa/lib/python3.10/site-packages/sanic/app.py:1565> source_traceback: Object created at (most recent call last):
File “/root/miniconda3/envs/rasa/lib/python3.10/runpy.py”, line 196, in _run_module_as_main return _run_code(code, main_globals, None,
File “/root/miniconda3/envs/rasa/lib/python3.10/runpy.py”, line 86, in _run_code exec(code, run_globals)
File “/root/miniconda3/envs/rasa/lib/python3.10/site-packages/rasa/main.py”, line 145, in
main() File “/root/miniconda3/envs/rasa/lib/python3.10/site-packages/rasa/main.py”, line 127, in main cmdline_arguments.func(cmdline_arguments)
File “/root/miniconda3/envs/rasa/lib/python3.10/site-packages/rasa/cli/run.py”, line 93, in run rasa.run(**vars(args))
File “/root/miniconda3/envs/rasa/lib/python3.10/site-packages/rasa/api.py”, line 56, in run rasa.core.run.serve_application(
File “/root/miniconda3/envs/rasa/lib/python3.10/site-packages/rasa/core/run.py”, line 229, in serve_application
app.run( File “/root/miniconda3/envs/rasa/lib/python3.10/site-packages/sanic/app.py”, line 1204, in run serve_single(server_settings) File “/root/miniconda3/envs/rasa/lib/python3.10/site-packages/sanic/server/runners.py”, line 206, in serve_single
serve(**server_settings) File “/root/miniconda3/envs/rasa/lib/python3.10/site-packages/sanic/server/runners.py”, line 155, in serve
loop.run_forever()
File “connection_task”, line 15, in connection_task from sanic.http import Http, Stage
File “/root/miniconda3/envs/rasa/lib/python3.10/site-packages/sanic/http.py”, line 146, in http1 await self.protocol.request_handler(self.request) File “handle_request”, line 83, in handle_request )
File “/root/miniconda3/envs/rasa/lib/python3.10/site-packages/rasa/server.py”, line 264, in decorated return await result if isawaitable(result) else result
File “/root/miniconda3/envs/rasa/lib/python3.10/site-packages/rasa/server.py”, line 583, in decorated_function
request.app.add_task(wrapped()) File “/root/miniconda3/envs/rasa/lib/python3.10/site-packages/sanic/app.py”, line 1609, in add_task return self._loop_add_task(
File “/root/miniconda3/envs/rasa/lib/python3.10/site-packages/sanic/app.py”, line 1565, in _loop_add_task
task = loop.create_task(prepped, name=name) Traceback (most recent call last):
File “/root/miniconda3/envs/rasa/lib/python3.10/site-packages/rasa/server.py”, line 580, in wrapped await session.post(callback_url, raise_for_status=True, **payload)
File “/root/miniconda3/envs/rasa/lib/python3.10/site-packages/aiohttp/client.py”, line 560, in _request await resp.start(conn)
File “/root/miniconda3/envs/rasa/lib/python3.10/site-packages/aiohttp/client_reqrep.py”, line 894, in start
with self._timer:
File “/root/miniconda3/envs/rasa/lib/python3.10/site-packages/aiohttp/helpers.py”, line 721, in exit
raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError

May I ask if you have encountered a similar situation? Thank you for your solution.

Issue temporarily resolved

I used the /model/train interface to train the model, and sent the trained model to another business server through callback_url. The postback model always times out due to bandwidth issues between the two devices. The reason is that the default timeout period of aiohttp is 5 minutes, and rasa does not expose this timeout parameter.
So my solution is to modify line 579 of rasa/server.py,
async with aiohttp.ClientSession() as session:
changed to
async with aiohttp.ClientSession(timeout=aiohttp.ClientTimeout(total=3000)) as session:

Temporarily set the timeout to 50 minutes, the problem is now resolved.

It is hoped that the --request-timeout parameter setting of the rasa run command line can affect the timeout value of this callback.