RasaX stops responding when using socket.io

I have RasaX deployed on a server and I am trying to use the botfornts rasa-webchat with socketio. I also tried using the rest channel with a different chat widget and everything works as expected. But with socketio, the server stops responding as soon as I send a message. Many times even the chat widget doesn’t display and I get timeout errors on Chrome.

Hi Divya,

Can you provide the Rasa log message from the time of the connection attempts. Also, try a curl from the command line to confirm that the port is listening. Replace the host and port in this curl.

curl --location --request GET 'http://<host:5005>/socket.io/?EIO=3&transport=polling&t=NHB_3lo'

@stephens Here are the logs. I deployed rasax using docker as described on the rasa documentation.

  • nginx
nginx 13:27:36.90 
nginx 13:27:36.90 Welcome to the Bitnami nginx container
nginx 13:27:36.90 Subscribe to project updates by watching https://github.com/bitnami/bitnami-docker-nginx
nginx 13:27:36.91 Submit issues and feature requests at https://github.com/bitnami/bitnami-docker-nginx/issues
nginx 13:27:36.91 

SSL encryption is not used since no certificates were provided.
>> exec docker CMD
/opt/bitnami/scripts/nginx/run.sh
nginx 13:27:36.97 INFO  ==> ** Starting NGINX **
2020/09/15 13:27:37 [notice] 1#0: using the "epoll" event method
2020/09/15 13:27:37 [notice] 1#0: nginx/1.17.10
2020/09/15 13:27:37 [notice] 1#0: built by gcc 8.3.0 (Debian 8.3.0-6) 
2020/09/15 13:27:37 [notice] 1#0: OS: Linux 4.15.0-117-generic
2020/09/15 13:27:37 [notice] 1#0: getrlimit(RLIMIT_NOFILE): 1048576:1048576
2020/09/15 13:27:37 [notice] 1#0: start worker processes
2020/09/15 13:27:37 [notice] 1#0: start worker process 16
2020/09/15 13:27:37 [notice] 1#0: start worker process 17
2020/09/15 13:27:37 [notice] 1#0: start worker process 18
2020/09/15 13:27:37 [notice] 1#0: start worker process 19
178.73.215.171 - - [15/Sep/2020:14:04:59 +0000] "GET / HTTP/1.0" 200 3217 "-" "-"
27.34.22.20 - - [15/Sep/2020:14:22:28 +0000] "GET /socket.io/?EIO=3&transport=polling&t=NHB_3lo HTTP/1.1" 200 119 "-" "curl/7.65.3"
2020/09/15 14:22:28 [info] 19#0: *3 client 25.34.22.20 closed keepalive connection
27.34.22.20 - - [15/Sep/2020:14:22:48 +0000] "GET /socket.io/?EIO=3&transport=polling&t=NHB_3lo HTTP/1.1" 200 119 "-" "curl/7.65.3"
2020/09/15 14:22:49 [info] 19#0: *5 client 25.34.22.20 closed keepalive connection
27.34.22.20 - - [15/Sep/2020:14:23:45 +0000] "GET /socket.io/?EIO=3&transport=polling&t=NII9cp7 HTTP/1.1" 200 119 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.102 Safari/537.36"
27.34.22.20 - - [15/Sep/2020:14:23:45 +0000] "POST /socket.io/?EIO=3&transport=polling&t=NII9cpk&sid=f5c2dee86b4144f7ab7cdadea0af96d5 HTTP/1.1" 200 2 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.102 Safari/537.36"
27.34.22.20 - - [15/Sep/2020:14:23:45 +0000] "GET /socket.io/?EIO=3&transport=polling&t=NII9cpm&sid=f5c2dee86b4144f7ab7cdadea0af96d5 HTTP/1.1" 200 4 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.102 Safari/537.36"
2020/09/15 14:23:45 [info] 19#0: *9 recv() failed (104: Connection reset by peer) while proxying upgraded connection, client: 25.34.22.20, server: , request: "GET /socket.io/?EIO=3&transport=websocket&sid=f5c2dee86b4144f7ab7cdadea0af96d5 HTTP/1.1", upstream: "http://172.19.0.8:5005/socket.io/?EIO=3&transport=websocket&sid=f5c2dee86b4144f7ab7cdadea0af96d5", host: "272.45.245.70"
27.34.22.20 - - [15/Sep/2020:14:23:45 +0000] "GET /socket.io/?EIO=3&transport=websocket&sid=f5c2dee86b4144f7ab7cdadea0af96d5 HTTP/1.1" 101 8 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.102 Safari/537.36"
2020/09/15 14:23:45 [info] 19#0: *12 client 25.34.22.20 closed keepalive connection (104: Connection reset by peer)
2020/09/15 14:23:45 [info] 19#0: *7 client 25.34.22.20 closed keepalive connection (104: Connection reset by peer)
  • RasaX
INFO:alembic.runtime.migration:Context impl PostgresqlImpl.
INFO:alembic.runtime.migration:Will assume transactional DDL.
INFO:rasax.community.services.event_service:Starting event service (standalone: False).
INFO:rasax.community.services.integrated_version_control.git_service:Cloning git repository from URL 'git@gitlab.com:diyoai/projects/chatbot/ministry-bot/moest-bot.git'.
/usr/local/lib/python3.7/site-packages/rasax/community/services/test_service.py:43: UserWarning: Unable to get tests from /app/git/1/tests/conversation_tests.md:
File '/app/git/1/tests/conversation_tests.md' does not exist. Please, make sure you have end-to-end tests added to your assistant. See https://rasa.com/docs/rasa-x/user-guide/test-assistant for more information.
  UserWarning,
Starting Rasa X server... 🚀
INFO:rasax.community.services.event_consumers.pika_consumer:Start consuming queue 'rasa_production_events' on pika host 'rabbit'.
  • rasa-production and rasa-worker
2020-09-15 13:27:52.000334: E tensorflow/stream_executor/cuda/cuda_driver.cc:351] failed call to cuInit: UNKNOWN ERROR (303)
  • Action server
2020-09-15 13:27:36 INFO     rasa_sdk.endpoint  - Starting action endpoint server...
2020-09-15 13:27:36 INFO     rasa_sdk.executor  - Registered function for 'action_query_knowledge_base'.
2020-09-15 13:27:36 INFO     rasa_sdk.endpoint  - Action endpoint is up and running on http://localhost:5055

This is what I got from curl when I ran it the first time

curl --location --request GET 'http://<host:5005>/socket.io/?EIO=3&transport=polling&t=NHB_3lo'

�0{"sid":"ad2ad8f8f5c54dc4ae9b3e30f6454435","upgrades":["websocket"],"pingTimeout":60000,"pingInterval":25000}�40% 

Most of the times I get a timeout error after sending the first message to the server. Sometimes I cannot even send a message using socket.io

Thanks for the logs. Rasa X and the action server aren’t involved in this issue. Is there any indication in the rasa-production log that the socket channel is being hit?

The nginx connection reset looks like an issue. Are there any firewalls that could be restricting socket connections?