Rasa X does not connect to Production/Worker hosts

kappa90 · February 14, 2020, 2:34pm

Trying to install Rasa-X, first I installed manually v.0.25.1 because the quick-install was not running, then I managed to upgrade to v.0.25.2 through the install.sh

I managed to install everything and get my data in from a Bitbucket repo through the Integrated Version Control feature.

It seems though that I can’t train models nor run the bot, and that rasa-X does not connect to rasa-production and rasa-worker, and viceversa.

Installed Rasa X 0.25.2, and Rasa NLU/Core 1.7.2, with a Rasa X demo as Action server.

This is what happens on the Rasa-X logs:

INFO:rasax.community.services.event_service:Waiting until database migrations have been executed...
INFO:alembic.runtime.migration:Context impl PostgresqlImpl.
INFO:alembic.runtime.migration:Will assume transactional DDL.
INFO:rasax.community.services.event_service:Check for database migrations completed.
ERROR:pika.adapters.utils.io_services_utils:Socket failed to connect: <socket.socket fd=18, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('172.20.0.6', 36010)>; error=111 (Connection refused)
ERROR:pika.adapters.utils.connection_workflow:TCP Connection attempt failed: ConnectionRefusedError(111, 'Connection refused'); dest=(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('172.20.0.4', 5672))
ERROR:pika.adapters.utils.connection_workflow:AMQPConnector - reporting failure: AMQPConnectorSocketConnectError: ConnectionRefusedError(111, 'Connection refused')
INFO:alembic.runtime.migration:Context impl PostgresqlImpl.
INFO:alembic.runtime.migration:Will assume transactional DDL.
INFO:rasax.community.services.git_service:Cloning git repository from URL 'git@bitbucket.org:kappa90/larachat.git'.
Starting Rasa X server... 🚀
ERROR:pika.adapters.utils.io_services_utils:Socket failed to connect: <socket.socket fd=18, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('172.20.0.6', 36038)>; error=111 (Connection refused)
ERROR:pika.adapters.utils.connection_workflow:TCP Connection attempt failed: ConnectionRefusedError(111, 'Connection refused'); dest=(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('172.20.0.4', 5672))
ERROR:pika.adapters.utils.connection_workflow:AMQPConnector - reporting failure: AMQPConnectorSocketConnectError: ConnectionRefusedError(111, 'Connection refused')
ERROR:pika.adapters.utils.io_services_utils:Socket failed to connect: <socket.socket fd=18, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('172.20.0.6', 36044)>; error=111 (Connection refused)
ERROR:pika.adapters.utils.connection_workflow:TCP Connection attempt failed: ConnectionRefusedError(111, 'Connection refused'); dest=(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('172.20.0.4', 5672))
ERROR:pika.adapters.utils.connection_workflow:AMQPConnector - reporting failure: AMQPConnectorSocketConnectError: ConnectionRefusedError(111, 'Connection refused')
ERROR:pika.adapters.utils.io_services_utils:Socket failed to connect: <socket.socket fd=18, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('172.20.0.6', 36050)>; error=111 (Connection refused)
ERROR:pika.adapters.utils.connection_workflow:TCP Connection attempt failed: ConnectionRefusedError(111, 'Connection refused'); dest=(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('172.20.0.4', 5672))
ERROR:pika.adapters.utils.connection_workflow:AMQPConnector - reporting failure: AMQPConnectorSocketConnectError: ConnectionRefusedError(111, 'Connection refused')
INFO:rasax.community.services.event_consumers.pika_consumer:Start consuming queue 'rasa_production_events' on pika host 'rabbit'.
ERROR:rasax.community.api.blueprints.models:Cannot connect to host rasa-worker:5005 ssl:None [Connection refused]

This is what happens on the rasa-production and rasa-worker:

Starting Rasa X in production mode... 🚀
Could not fetch runtime config from server at 'http://rasa-x:5002/api/config?token=TnEu+dtouXM2QAhK3XMxTw=='. Exiting.

koszta5 · February 14, 2020, 5:23pm

Same for me! when will this get fixed?! Its really troubling this has been there apparently for a longer time and was on & off with the fix. Proofs that this is a problem…:

github.com/RasaHQ/rasa

Pika timeout fix

RasaHQ:1.4.x ← RasaHQ:pika-timeout-fix

opened 05:55PM - 28 Oct 19 UTC

wochinge

+141 -32

**Proposed changes**: - fix https://github.com/RasaHQ/rasa-x/issues/1399 - use… `SelectConnection` instead of `BlockingConnection` in order to be able to process heart beat events from the event broker in the background **Status (please check what you already did)**: - [x] made PR ready for code review - [ ] added some tests for the functionality - [ ] updated the documentation - [x] updated the changelog - [x] reformat files using `black` (please check [Readme](https://github.com/RasaHQ/rasa_nlu#code-style) for instructions)

kappa90 · February 14, 2020, 6:48pm

I managed to install everything (did the exact same steps as before), but with rasa-x v.0.25.0, anything above that doesn’t work!

Tobias_Wochinger · February 19, 2020, 10:27am

Can you please share your environments.yml? You used the docker-compose file from the docs I assume?

RielKIM · February 26, 2020, 1:38am

It’s same for me. Is it fixed?

yiouyou · March 2, 2020, 9:53am

Same here, any idea how to fix the problem?

agustingp · March 4, 2020, 4:02pm

Same here!

akelad · March 6, 2020, 1:42pm

Could you share some logs/Rasa X versions/configuration etc?

argideritzalpea · March 15, 2020, 7:53pm

Yes, I have this same problem. I have this currently with Rasa X version “latest”. Here is my: .env:

RASA_X_VERSION=latest
RASA_VERSION=1.8.0
RASA_X_DEMO_VERSION=0.26.0

environments.yml:

  production:
    url: ${RASA_PRODUCTION_HOST}
    token: ${RASA_TOKEN}
  worker:
    url: ${RASA_WORKER_HOST}
    token: ${RASA_TOKEN}

docker-compose logs:

docker-compose up
rasa_redis_1 is up-to-date
rasa_db_1 is up-to-date
rasa_rabbit_1 is up-to-date
rasa_duckling_1 is up-to-date
rasa_rasa-x_1 is up-to-date
rasa_rasa-production_1 is up-to-date
rasa_rasa-worker_1 is up-to-date
rasa_app_1 is up-to-date
rasa_nginx_1 is up-to-date
Attaching to rasa_redis_1, rasa_db_1, rasa_rabbit_1, rasa_duckling_1, rasa_rasa-x_1, rasa_rasa-production_1, rasa_rasa-worker_1, rasa_app_1, rasa_nginx_1
app_1              | 2020-03-15 19:32:02 INFO     rasa_sdk.endpoint  - Starting action endpoint server...
app_1              | 2020-03-15 19:32:02 INFO     rasa_sdk.executor  - Registered function for 'action_inform_peopleCount'.
app_1              | 2020-03-15 19:32:02 INFO     rasa_sdk.executor  - Registered function for 'action_inform_vehicleCount'.
app_1              | 2020-03-15 19:32:02 INFO     rasa_sdk.executor  - Registered function for 'action_inform_partyVehics'.
app_1              | 2020-03-15 19:32:02 INFO     rasa_sdk.executor  - Registered function for 'utter_goodbye'.
app_1              | 2020-03-15 19:32:02 INFO     rasa_sdk.executor  - Registered function for 'action_inform_countTime'.
app_1              | 2020-03-15 19:32:02 INFO     rasa_sdk.executor  - Registered function for 'action_inform_source'.
app_1              | 2020-03-15 19:32:02 INFO     rasa_sdk.executor  - Registered function for 'action_inform_returnStatus'.
app_1              | 2020-03-15 19:32:02 INFO     rasa_sdk.executor  - Registered function for 'action_inform_trailVisits'.
app_1              | 2020-03-15 19:32:02 INFO     rasa_sdk.executor  - Registered function for 'action_inform_yearBorn'.
app_1              | 2020-03-15 19:32:02 INFO     rasa_sdk.executor  - Registered function for 'action_inform_howLong'.
app_1              | 2020-03-15 19:32:02 INFO     rasa_sdk.executor  - Registered function for 'action_inform_zipCode'.
app_1              | 2020-03-15 19:32:02 INFO     rasa_sdk.executor  - Registered function for 'action_inform_seenBefore'.
app_1              | 2020-03-15 19:32:02 INFO     rasa_sdk.executor  - Registered function for 'action_inform_seenPastDay'.
app_1              | 2020-03-15 19:32:02 INFO     rasa_sdk.executor  - Registered function for 'action_inform_ambigNumber'.
app_1              | 2020-03-15 19:32:02 INFO     rasa_sdk.executor  - Registered function for 'action_inform_ambigVehic'.
app_1              | 2020-03-15 19:32:02 INFO     rasa_sdk.executor  - Registered function for 'action_setDefaultSlotValues'.
app_1              | 2020-03-15 19:32:02 INFO     rasa_sdk.executor  - Registered function for 'action_default_fallback'.
app_1              | 2020-03-15 19:32:02 INFO     rasa_sdk.executor  - Registered function for 'feedback_form'.
app_1              | 2020-03-15 19:32:02 INFO     rasa_sdk.executor  - Registered function for 'count_episode'.
app_1              | 2020-03-15 19:32:02 INFO     rasa_sdk.executor  - Registered function for 'personal_form'.
db_1               | 
db_1               | Welcome to the Bitnami postgresql container
db_1               | Subscribe to project updates by watching https://github.com/bitnami/bitnami-docker-postgresql
db_1               | Submit issues and feature requests at https://github.com/bitnami/bitnami-docker-postgresql/issues
db_1               | Send us your feedback at containers@bitnami.com
db_1               | 
db_1               | INFO  ==> ** Starting PostgreSQL setup **
db_1               | INFO  ==> Validating settings in POSTGRESQL_* env vars..
db_1               | INFO  ==> Initializing PostgreSQL database...
db_1               | INFO  ==> postgresql.conf file not detected. Generating it...
db_1               | INFO  ==> pg_hba.conf file not detected. Generating it...
db_1               | INFO  ==> Deploying PostgreSQL with persisted data...
db_1               | INFO  ==> Cleaning stale postmaster.pid file
db_1               | INFO  ==> Configuring replication parameters
db_1               | INFO  ==> Configuring fsync
db_1               | INFO  ==> Loading custom scripts...
db_1               | INFO  ==> Enabling remote connections
db_1               | INFO  ==> Stopping PostgreSQL...
db_1               | INFO  ==> ** PostgreSQL setup finished! **
db_1               | 
db_1               | INFO  ==> ** Starting PostgreSQL **
db_1               | 2020-03-15 19:31:55.398 GMT [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
db_1               | 2020-03-15 19:31:55.398 GMT [1] LOG:  listening on IPv6 address "::", port 5432
db_1               | 2020-03-15 19:31:55.403 GMT [1] LOG:  listening on Unix socket "/tmp/.s.PGSQL.5432"
db_1               | 2020-03-15 19:31:55.674 GMT [151] LOG:  database system was interrupted; last known up at 2020-03-15 19:20:19 GMT
db_1               | 2020-03-15 19:31:56.523 GMT [151] LOG:  database system was not properly shut down; automatic recovery in progress
db_1               | 2020-03-15 19:31:56.538 GMT [151] LOG:  redo starts at 0/19EA938
db_1               | 2020-03-15 19:31:56.538 GMT [151] LOG:  invalid record length at 0/19EA970: wanted 24, got 0
db_1               | 2020-03-15 19:31:56.538 GMT [151] LOG:  redo done at 0/19EA938
db_1               | 2020-03-15 19:31:56.767 GMT [1] LOG:  database system is ready to accept connections
db_1               | 2020-03-15 19:32:29.023 GMT [162] WARNING:  there is no transaction in progress
db_1               | 2020-03-15 19:32:29.024 GMT [163] WARNING:  there is no transaction in progress
duckling_1         | no port specified, defaulting to port 8000
duckling_1         | Listening on http://0.0.0.0:8000
nginx_1            | SSL encryption is not used since no certificates were provided.
nginx_1            |   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
nginx_1            |                                  Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (7) Failed to connect to app port 80: Connection refused
nginx_1            | 7
nginx_1            | >> exec docker CMD
nginx_1            | nginx
nginx_1            | 2020/03/15 19:31:58 [notice] 1#0: using the "epoll" event method
nginx_1            | 2020/03/15 19:31:58 [notice] 1#0: nginx/1.14.2
nginx_1            | 2020/03/15 19:31:58 [notice] 1#0: built by gcc 6.3.0 20170516 (Debian 6.3.0-18+deb9u1) 
nginx_1            | 2020/03/15 19:31:58 [notice] 1#0: OS: Linux 4.9.0-12-amd64
nginx_1            | 2020/03/15 19:31:58 [notice] 1#0: getrlimit(RLIMIT_NOFILE): 1048576:1048576
nginx_1            | 2020/03/15 19:31:58 [notice] 1#0: start worker processes
nginx_1            | 2020/03/15 19:31:58 [notice] 1#0: start worker process 12
rabbit_1           |  19:31:52.02 
rabbit_1           |  19:31:52.04 Welcome to the Bitnami rabbitmq container
rabbit_1           |  19:31:52.04 Subscribe to project updates by watching https://github.com/bitnami/bitnami-docker-rabbitmq
rabbit_1           |  19:31:52.05 Submit issues and feature requests at https://github.com/bitnami/bitnami-docker-rabbitmq/issues
rabbit_1           |  19:31:52.05 Send us your feedback at containers@bitnami.com
rabbit_1           |  19:31:52.05 
rabbit_1           |  19:31:52.05 INFO  ==> ** Starting RabbitMQ setup **
rabbit_1           |  19:31:52.12 INFO  ==> Validating settings in RABBITMQ_* env vars..
rabbit_1           |  19:31:52.27 INFO  ==> Initializing RabbitMQ...
rabbit_1           |  19:31:52.34 INFO  ==> Generating random cookie
rabbit_1           |  19:31:52.44 INFO  ==> Starting RabbitMQ in background...
rabbit_1           | Waiting for erlang distribution on node 'rabbit@localhost' while OS process '43' is running
rabbit_1           | Error: operation wait on node rabbit@localhost timed out. Timeout value used: 5000
rabbit_1           | Waiting for erlang distribution on node 'rabbit@localhost' while OS process '43' is running
rabbit_1           | Waiting for applications 'rabbit_and_plugins' to start on node 'rabbit@localhost'
rasa-production_1  | 2020-03-15 19:32:16.803739: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer.so.6'; dlerror: libnvinfer.so.6: cannot open shared object file: No such file or directory
rasa-production_1  | 2020-03-15 19:32:16.805715: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer_plugin.so.6'; dlerror: libnvinfer_plugin.so.6: cannot open shared object file: No such file or directory
rasa-production_1  | 2020-03-15 19:32:16.806075: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:30] Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
rasa-production_1  | 2020-03-15 19:32:28 ERROR    pika.adapters.utils.io_services_utils  - Socket failed to connect: <socket.socket fd=21, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('172.20.0.7', 46658)>; error=111 (Connection refused)
rasa-production_1  | 2020-03-15 19:32:28 ERROR    pika.adapters.utils.connection_workflow  - TCP Connection attempt failed: ConnectionRefusedError(111, 'Connection refused'); dest=(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('172.20.0.4', 5672))
rasa-production_1  | 2020-03-15 19:32:28 ERROR    pika.adapters.utils.connection_workflow  - AMQPConnector - reporting failure: AMQPConnectorSocketConnectError: ConnectionRefusedError(111, 'Connection refused')
rasa-worker_1      | 2020-03-15 19:32:16.804313: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer.so.6'; dlerror: libnvinfer.so.6: cannot open shared object file: No such file or directory
rasa-worker_1      | 2020-03-15 19:32:16.805031: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer_plugin.so.6'; dlerror: libnvinfer_plugin.so.6: cannot open shared object file: No such file or directory
rasa-worker_1      | 2020-03-15 19:32:16.805891: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:30] Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
rasa-worker_1      | 2020-03-15 19:32:28 ERROR    pika.adapters.utils.io_services_utils  - Socket failed to connect: <socket.socket fd=21, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('172.20.0.8', 40460)>; error=111 (Connection refused)
rasa-worker_1      | 2020-03-15 19:32:28 ERROR    pika.adapters.utils.connection_workflow  - TCP Connection attempt failed: ConnectionRefusedError(111, 'Connection refused'); dest=(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('172.20.0.4', 5672))
rasa-worker_1      | 2020-03-15 19:32:28 ERROR    pika.adapters.utils.connection_workflow  - AMQPConnector - reporting failure: AMQPConnectorSocketConnectError: ConnectionRefusedError(111, 'Connection refused')
rasa-x_1           | INFO:rasax.community.services.event_service:Waiting until database migrations have been executed...
rasa-x_1           | INFO:alembic.runtime.migration:Context impl PostgresqlImpl.
rasa-x_1           | INFO:alembic.runtime.migration:Context impl PostgresqlImpl.
rasa-x_1           | INFO:alembic.runtime.migration:Will assume transactional DDL.
rasa-x_1           | INFO:alembic.runtime.migration:Will assume transactional DDL.
rasa-x_1           | INFO:rasax.community.services.event_service:Check for database migrations completed.
rasa-x_1           | ERROR:pika.adapters.utils.io_services_utils:Socket failed to connect: <socket.socket fd=18, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('172.20.0.6', 52716)>; error=111 (Connection refused)
rasa-x_1           | ERROR:pika.adapters.utils.connection_workflow:TCP Connection attempt failed: ConnectionRefusedError(111, 'Connection refused'); dest=(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('172.20.0.4', 5672))
rasa-x_1           | ERROR:pika.adapters.utils.connection_workflow:AMQPConnector - reporting failure: AMQPConnectorSocketConnectError: ConnectionRefusedError(111, 'Connection refused')
rasa-x_1           | Run time of job "GitService.run_background_synchronization (trigger: cron[minute='*'], next run at: 2020-03-15 19:32:18 UTC)" was missed by 0:00:01.299251
rasa-x_1           | Starting Rasa X server... 🚀
rasa-x_1           | ERROR:pika.adapters.utils.io_services_utils:Socket failed to connect: <socket.socket fd=18, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('172.20.0.6', 52734)>; error=111 (Connection refused)
rasa-x_1           | ERROR:pika.adapters.utils.connection_workflow:TCP Connection attempt failed: ConnectionRefusedError(111, 'Connection refused'); dest=(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('172.20.0.4', 5672))
rasa-x_1           | ERROR:pika.adapters.utils.connection_workflow:AMQPConnector - reporting failure: AMQPConnectorSocketConnectError: ConnectionRefusedError(111, 'Connection refused')
rasa-x_1           | ERROR:pika.adapters.utils.io_services_utils:Socket failed to connect: <socket.socket fd=18, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('172.20.0.6', 52746)>; error=111 (Connection refused)
rasa-x_1           | ERROR:pika.adapters.utils.connection_workflow:TCP Connection attempt failed: ConnectionRefusedError(111, 'Connection refused'); dest=(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('172.20.0.4', 5672))
rasa-x_1           | ERROR:pika.adapters.utils.connection_workflow:AMQPConnector - reporting failure: AMQPConnectorSocketConnectError: ConnectionRefusedError(111, 'Connection refused')
redis_1            |  19:31:51.48 
redis_1            |  19:31:51.49 Welcome to the Bitnami redis container
redis_1            |  19:31:51.49 Subscribe to project updates by watching https://github.com/bitnami/bitnami-docker-redis
redis_1            |  19:31:51.50 Submit issues and feature requests at https://github.com/bitnami/bitnami-docker-redis/issues
redis_1            |  19:31:51.50 Send us your feedback at containers@bitnami.com
redis_1            |  19:31:51.50 
redis_1            |  19:31:51.51 INFO  ==> ** Starting Redis setup **
redis_1            |  19:31:51.59 INFO  ==> Initializing Redis...
redis_1            |  19:31:51.74 INFO  ==> ** Redis setup finished! **
redis_1            | 
redis_1            |  19:31:51.77 INFO  ==> ** Starting Redis **
redis_1            | 1:C 15 Mar 2020 19:31:51.956 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis_1            | 1:C 15 Mar 2020 19:31:51.956 # Redis version=5.0.5, bits=64, commit=00000000, modified=0, pid=1, just started
redis_1            | 1:C 15 Mar 2020 19:31:51.956 # Configuration loaded
redis_1            |                 _._                                                  
redis_1            |            _.-``__ ''-._                                             
redis_1            |       _.-``    `.  `_.  ''-._           Redis 5.0.5 (00000000/0) 64 bit
redis_1            |   .-`` .-```.  ```\/    _.,_ ''-._                                   
redis_1            |  (    '      ,       .-`  | `,    )     Running in standalone mode
redis_1            |  |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
redis_1            |  |    `-._   `._    /     _.-'    |     PID: 1
redis_1            |   `-._    `-._  `-./  _.-'    _.-'                                   
redis_1            |  |`-._`-._    `-.__.-'    _.-'_.-'|                                  
redis_1            |  |    `-._`-._        _.-'_.-'    |           http://redis.io        
redis_1            |   `-._    `-._`-.__.-'_.-'    _.-'                                   
redis_1            |  |`-._`-._    `-.__.-'    _.-'_.-'|                                  
redis_1            |  |    `-._`-._        _.-'_.-'    |                                  
redis_1            |   `-._    `-._`-.__.-'_.-'    _.-'                                   
redis_1            |       `-._    `-.__.-'    _.-'                                       
redis_1            |           `-._        _.-'                                           
redis_1            |               `-.__.-'                                               
redis_1            | 
redis_1            | 1:M 15 Mar 2020 19:31:51.961 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
redis_1            | 1:M 15 Mar 2020 19:31:51.961 # Server initialized
redis_1            | 1:M 15 Mar 2020 19:31:51.961 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
redis_1            | 1:M 15 Mar 2020 19:31:51.961 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
redis_1            | 1:M 15 Mar 2020 19:31:51.961 * Ready to accept connections
rasa-x_1           | ERROR:pika.adapters.utils.io_services_utils:Socket failed to connect: <socket.socket fd=18, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('172.20.0.6', 52768)>; error=111 (Connection refused)
rasa-x_1           | ERROR:pika.adapters.utils.connection_workflow:TCP Connection attempt failed: ConnectionRefusedError(111, 'Connection refused'); dest=(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('172.20.0.4', 5672))
rasa-x_1           | ERROR:pika.adapters.utils.connection_workflow:AMQPConnector - reporting failure: AMQPConnectorSocketConnectError: ConnectionRefusedError(111, 'Connection refused')
rasa-production_1  | 2020-03-15 19:32:33 ERROR    pika.adapters.utils.io_services_utils  - Socket failed to connect: <socket.socket fd=26, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('172.20.0.7', 46678)>; error=111 (Connection refused)
rasa-production_1  | 2020-03-15 19:32:33 ERROR    pika.adapters.utils.connection_workflow  - TCP Connection attempt failed: ConnectionRefusedError(111, 'Connection refused'); dest=(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('172.20.0.4', 5672))
rasa-production_1  | 2020-03-15 19:32:33 ERROR    pika.adapters.utils.connection_workflow  - AMQPConnector - reporting failure: AMQPConnectorSocketConnectError: ConnectionRefusedError(111, 'Connection refused')
rasa-worker_1      | 2020-03-15 19:32:33 ERROR    pika.adapters.utils.io_services_utils  - Socket failed to connect: <socket.socket fd=25, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('172.20.0.8', 40480)>; error=111 (Connection refused)
rasa-worker_1      | 2020-03-15 19:32:33 ERROR    pika.adapters.utils.connection_workflow  - TCP Connection attempt failed: ConnectionRefusedError(111, 'Connection refused'); dest=(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('172.20.0.4', 5672))
rasa-worker_1      | 2020-03-15 19:32:33 ERROR    pika.adapters.utils.connection_workflow  - AMQPConnector - reporting failure: AMQPConnectorSocketConnectError: ConnectionRefusedError(111, 'Connection refused')
rabbit_1           | Applications 'rabbit_and_plugins' are running on node 'rabbit@localhost'
rasa-x_1           | ERROR:pika.connection:Connection closed while authenticating indicating a probable authentication error
rasa-x_1           | ERROR:pika.adapters.utils.connection_workflow:AMQPConnector - reporting failure: AMQPConnectorAMQPHandshakeError: ProbableAuthenticationError: Client was disconnected at a connection stage indicating a probable authentication error: ("ConnectionClosedByBroker: (403) 'ACCESS_REFUSED - Login was refused using authentication mechanism PLAIN. For details see the broker logfile.'",)
rasa-x_1           | INFO:rasax.community.services.event_consumers.pika_consumer:Start consuming queue 'rasa_production_events' on pika host 'rabbit'.
rabbit_1           |  19:32:42.62 INFO  ==> Stopping RabbitMQ...
rasa-x_1           | ERROR:pika.adapters.blocking_connection:Unexpected connection close detected: ConnectionClosedByBroker: (320) "CONNECTION_FORCED - broker forced connection closure with reason 'shutdown'"
rasa-x_1           | ERROR:rasax.community.services.event_service:Caught an exception while consuming events. Will retry in 5 s.
rasa-x_1           | Traceback (most recent call last):
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/rasax/community/services/event_service.py", line 1400, in continuously_consume
rasa-x_1           |     consumer.consume()
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/rasax/community/services/event_consumers/pika_consumer.py", line 123, in consume
rasa-x_1           |     self.channel.start_consuming()
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/pika/adapters/blocking_connection.py", line 1866, in start_consuming
rasa-x_1           |     self._process_data_events(time_limit=None)
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/pika/adapters/blocking_connection.py", line 2027, in _process_data_events
rasa-x_1           |     self.connection.process_data_events(time_limit=time_limit)
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/pika/adapters/blocking_connection.py", line 825, in process_data_events
rasa-x_1           |     self._flush_output(common_terminator)
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/pika/adapters/blocking_connection.py", line 522, in _flush_output
rasa-x_1           |     raise self._closed_result.value.error
rasa-x_1           | pika.exceptions.ConnectionClosedByBroker: (320, "CONNECTION_FORCED - broker forced connection closure with reason 'shutdown'")
rasa-x_1           | ERROR:pika.adapters.utils.io_services_utils:Socket failed to connect: <socket.socket fd=18, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('172.20.0.6', 52802)>; error=111 (Connection refused)
rasa-x_1           | ERROR:pika.adapters.utils.connection_workflow:TCP Connection attempt failed: ConnectionRefusedError(111, 'Connection refused'); dest=(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('172.20.0.4', 5672))
rasa-x_1           | ERROR:pika.adapters.utils.connection_workflow:AMQPConnector - reporting failure: AMQPConnectorSocketConnectError: ConnectionRefusedError(111, 'Connection refused')
rasa-x_1           | ERROR:pika.adapters.utils.io_services_utils:Socket failed to connect: <socket.socket fd=18, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('172.20.0.6', 52804)>; error=111 (Connection refused)
rasa-x_1           | ERROR:pika.adapters.utils.connection_workflow:TCP Connection attempt failed: ConnectionRefusedError(111, 'Connection refused'); dest=(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('172.20.0.4', 5672))
rasa-x_1           | ERROR:pika.adapters.utils.connection_workflow:AMQPConnector - reporting failure: AMQPConnectorSocketConnectError: ConnectionRefusedError(111, 'Connection refused')
rabbit_1           |  19:32:56.53 INFO  ==> ** RabbitMQ setup finished! **
rabbit_1           | 
rabbit_1           |  19:32:56.91 INFO  ==> ** Starting RabbitMQ **
rasa-production_1  | 2020-03-15 19:32:58.191556: E tensorflow/stream_executor/cuda/cuda_driver.cc:351] failed call to cuInit: UNKNOWN ERROR (303)
rasa-worker_1      | 2020-03-15 19:32:58.196410: E tensorflow/stream_executor/cuda/cuda_driver.cc:351] failed call to cuInit: UNKNOWN ERROR (303)
rasa-x_1           | INFO:rasax.community.services.integrated_version_control.git_service:Cloning git repository from URL 'git@bitbucket.org:outdoorrd/crowdy.git'.
rasa-x_1           | ERROR:pika.adapters.utils.io_services_utils:Socket failed to connect: <socket.socket fd=18, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('172.20.0.6', 52812)>; error=111 (Connection refused)
rasa-x_1           | ERROR:pika.adapters.utils.connection_workflow:TCP Connection attempt failed: ConnectionRefusedError(111, 'Connection refused'); dest=(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('172.20.0.4', 5672))
rasa-x_1           | ERROR:pika.adapters.utils.connection_workflow:AMQPConnector - reporting failure: AMQPConnectorSocketConnectError: ConnectionRefusedError(111, 'Connection refused')
rasa-x_1           | ERROR:pika.adapters.utils.io_services_utils:Socket failed to connect: <socket.socket fd=18, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('172.20.0.6', 52816)>; error=111 (Connection refused)
rasa-x_1           | ERROR:pika.adapters.utils.connection_workflow:TCP Connection attempt failed: ConnectionRefusedError(111, 'Connection refused'); dest=(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('172.20.0.4', 5672))
rasa-x_1           | ERROR:pika.adapters.utils.connection_workflow:AMQPConnector - reporting failure: AMQPConnectorSocketConnectError: ConnectionRefusedError(111, 'Connection refused')
rasa-x_1           | Job "GitService.run_background_synchronization (trigger: cron[minute='*'], next run at: 2020-03-15 19:34:00 UTC)" raised an exception
rasa-x_1           | Traceback (most recent call last):
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/apscheduler/executors/base.py", line 125, in run_job
rasa-x_1           |     retval = job.func(*job.args, **job.kwargs)
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/rasax/community/services/integrated_version_control/git_service.py", line 826, in run_background_synchronization
rasa-x_1           |     git_service.synchronize_project(force_data_injection)
rasa-x_1           |   File "uvloop/loop.pyx", line 1456, in uvloop.loop.Loop.run_until_complete
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/rasax/community/services/integrated_version_control/git_service.py", line 675, in synchronize_project
rasa-x_1           |     await self._inject_data()
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/rasax/community/services/integrated_version_control/git_service.py", line 710, in _inject_data
rasa-x_1           |     str(self.repository_path()), str(data_path), self.session, SYSTEM_USER
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/rasax/community/initialise.py", line 293, in inject_files_from_disk
rasa-x_1           |     username,
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/rasax/community/initialise.py", line 129, in inject_domain
rasa-x_1           |     rasa_cli_utils.error_and_exit(
rasa-x_1           | AttributeError: module 'rasa.cli.utils' has no attribute 'error_and_exit'
rasa-x_1           | ERROR:pika.adapters.utils.io_services_utils:Socket failed to connect: <socket.socket fd=18, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('172.20.0.6', 52818)>; error=111 (Connection refused)
rasa-x_1           | ERROR:pika.adapters.utils.connection_workflow:TCP Connection attempt failed: ConnectionRefusedError(111, 'Connection refused'); dest=(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('172.20.0.4', 5672))
rasa-x_1           | ERROR:pika.adapters.utils.connection_workflow:AMQPConnector - reporting failure: AMQPConnectorSocketConnectError: ConnectionRefusedError(111, 'Connection refused')
rasa-x_1           | ERROR:pika.adapters.utils.io_services_utils:Socket failed to connect: <socket.socket fd=18, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('172.20.0.6', 52820)>; error=111 (Connection refused)
rasa-x_1           | ERROR:pika.adapters.utils.connection_workflow:TCP Connection attempt failed: ConnectionRefusedError(111, 'Connection refused'); dest=(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('172.20.0.4', 5672))
rasa-x_1           | ERROR:pika.adapters.utils.connection_workflow:AMQPConnector - reporting failure: AMQPConnectorSocketConnectError: ConnectionRefusedError(111, 'Connection refused')
rasa-x_1           | ERROR:pika.adapters.utils.io_services_utils:Socket failed to connect: <socket.socket fd=18, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('172.20.0.6', 52822)>; error=111 (Connection refused)
rasa-x_1           | ERROR:pika.adapters.utils.connection_workflow:TCP Connection attempt failed: ConnectionRefusedError(111, 'Connection refused'); dest=(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('172.20.0.4', 5672))
rasa-x_1           | ERROR:pika.adapters.utils.connection_workflow:AMQPConnector - reporting failure: AMQPConnectorSocketConnectError: ConnectionRefusedError(111, 'Connection refused')
nginx_1            | 178.73.215.171 - - [15/Mar/2020:19:33:23 +0000] "GET / HTTP/1.0" 200 3173 "-" "-"
rasa-x_1           | ERROR:pika.adapters.utils.io_services_utils:Socket failed to connect: <socket.socket fd=18, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('172.20.0.6', 52826)>; error=111 (Connection refused)
rasa-x_1           | ERROR:pika.adapters.utils.connection_workflow:TCP Connection attempt failed: ConnectionRefusedError(111, 'Connection refused'); dest=(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('172.20.0.4', 5672))
rasa-x_1           | ERROR:pika.adapters.utils.connection_workflow:AMQPConnector - reporting failure: AMQPConnectorSocketConnectError: ConnectionRefusedError(111, 'Connection refused')
rasa-x_1           | ERROR:pika.adapters.utils.io_services_utils:Socket failed to connect: <socket.socket fd=18, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('172.20.0.6', 52830)>; error=111 (Connection refused)
rasa-x_1           | ERROR:pika.adapters.utils.connection_workflow:TCP Connection attempt failed: ConnectionRefusedError(111, 'Connection refused'); dest=(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('172.20.0.4', 5672))
rasa-x_1           | ERROR:pika.adapters.utils.connection_workflow:AMQPConnector - reporting failure: AMQPConnectorSocketConnectError: ConnectionRefusedError(111, 'Connection refused')
rasa-x_1           | ERROR:pika.adapters.utils.io_services_utils:Socket failed to connect: <socket.socket fd=18, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('172.20.0.6', 52832)>; error=111 (Connection refused)
rasa-x_1           | ERROR:pika.adapters.utils.connection_workflow:TCP Connection attempt failed: ConnectionRefusedError(111, 'Connection refused'); dest=(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('172.20.0.4', 5672))
rasa-x_1           | ERROR:pika.adapters.utils.connection_workflow:AMQPConnector - reporting failure: AMQPConnectorSocketConnectError: ConnectionRefusedError(111, 'Connection refused')
rasa-x_1           | ERROR:pika.adapters.utils.io_services_utils:Socket failed to connect: <socket.socket fd=18, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('172.20.0.6', 52834)>; error=111 (Connection refused)
rasa-x_1           | ERROR:pika.adapters.utils.connection_workflow:TCP Connection attempt failed: ConnectionRefusedError(111, 'Connection refused'); dest=(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('172.20.0.4', 5672))
rasa-x_1           | ERROR:pika.adapters.utils.connection_workflow:AMQPConnector - reporting failure: AMQPConnectorSocketConnectError: ConnectionRefusedError(111, 'Connection refused')
nginx_1            | 64.113.20.166 - - [15/Mar/2020:19:33:45 +0000] "GET / HTTP/1.1" 200 1572 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36"
rabbit_1           | 2020-03-15 19:33:45.430 [info] <0.8.0> Feature flags: list of feature flags found:
rabbit_1           | 2020-03-15 19:33:45.430 [info] <0.8.0> Feature flags: feature flag states written to disk: yes
rabbit_1           | 2020-03-15 19:33:45.950 [info] <0.270.0> 
rabbit_1           |  Starting RabbitMQ 3.7.17 on Erlang 22.0
rabbit_1           |  Copyright (C) 2007-2019 Pivotal Software, Inc.
rabbit_1           |  Licensed under the MPL.  See https://www.rabbitmq.com/
rabbit_1           | 
rabbit_1           |   ##  ##
rabbit_1           |   ##  ##      RabbitMQ 3.7.17. Copyright (C) 2007-2019 Pivotal Software, Inc.
rabbit_1           |   ##########  Licensed under the MPL.  See https://www.rabbitmq.com/
rabbit_1           |   ######  ##
rabbit_1           |   ##########  Logs: <stdout>
rabbit_1           | 
rabbit_1           |               Starting broker...
rabbit_1           | 2020-03-15 19:33:45.962 [info] <0.270.0> 
rabbit_1           |  node           : rabbit@localhost
rabbit_1           |  home dir       : /opt/bitnami/rabbitmq/.rabbitmq
rabbit_1           |  config file(s) : /opt/bitnami/rabbitmq/etc/rabbitmq/rabbitmq.config
rabbit_1           |  cookie hash    : T0mERZ0InjuL/ot0zSDcYQ==
rabbit_1           |  log(s)         : <stdout>
rabbit_1           |  database dir   : /bitnami/rabbitmq/mnesia/rabbit@localhost
rabbit_1           | 2020-03-15 19:33:46.024 [info] <0.270.0> Running boot step pre_boot defined by app rabbit
rabbit_1           | 2020-03-15 19:33:46.024 [info] <0.270.0> Running boot step rabbit_core_metrics defined by app rabbit
rabbit_1           | 2020-03-15 19:33:46.027 [info] <0.270.0> Running boot step rabbit_alarm defined by app rabbit
rabbit_1           | 2020-03-15 19:33:46.162 [info] <0.280.0> Memory high watermark set to 1481 MiB (1553593139 bytes) of 3704 MiB (3883982848 bytes) total
rasa-x_1           | ERROR:pika.adapters.utils.io_services_utils:Socket failed to connect: <socket.socket fd=18, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('172.20.0.6', 52842)>; error=111 (Connection refused)
rasa-x_1           | ERROR:pika.adapters.utils.connection_workflow:TCP Connection attempt failed: ConnectionRefusedError(111, 'Connection refused'); dest=(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('172.20.0.4', 5672))
rasa-x_1           | ERROR:pika.adapters.utils.connection_workflow:AMQPConnector - reporting failure: AMQPConnectorSocketConnectError: ConnectionRefusedError(111, 'Connection refused')
nginx_1            | 64.113.20.166 - - [15/Mar/2020:19:33:46 +0000] "GET /api/projects/default/git_repositories HTTP/1.1" 401 57 "http://35.212.244.136/conversations" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36"
rabbit_1           | 2020-03-15 19:33:46.355 [info] <0.291.0> Enabling free disk space monitoring
rabbit_1           | 2020-03-15 19:33:46.367 [info] <0.291.0> Disk free limit set to 388MB
rabbit_1           | 2020-03-15 19:33:46.453 [info] <0.270.0> Running boot step code_server_cache defined by app rabbit
rabbit_1           | 2020-03-15 19:33:46.466 [info] <0.270.0> Running boot step file_handle_cache defined by app rabbit
rabbit_1           | 2020-03-15 19:33:46.474 [info] <0.327.0> Limiting to approx 1048476 file handles (943626 sockets)
rabbit_1           | 2020-03-15 19:33:46.474 [info] <0.330.0> FHC read buffering:  OFF
rabbit_1           | 2020-03-15 19:33:46.481 [info] <0.330.0> FHC write buffering: ON
rabbit_1           | 2020-03-15 19:33:46.533 [info] <0.270.0> Running boot step worker_pool defined by app rabbit
rabbit_1           | 2020-03-15 19:33:46.538 [info] <0.273.0> Will use 1 processes for default worker pool
rabbit_1           | 2020-03-15 19:33:46.545 [info] <0.273.0> Starting worker pool 'worker_pool' with 1 processes in it
nginx_1            | 64.113.20.166 - - [15/Mar/2020:19:33:46 +0000] "GET /api/conversations/tags HTTP/1.1" 401 57 "http://35.212.244.136/conversations" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36"
nginx_1            | 64.113.20.166 - - [15/Mar/2020:19:33:46 +0000] "GET /api/conversationEntities HTTP/1.1" 401 57 "http://35.212.244.136/conversations" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36"
rabbit_1           | 2020-03-15 19:33:46.567 [info] <0.270.0> Running boot step database defined by app rabbit
rabbit_1           | 2020-03-15 19:33:46.702 [info] <0.270.0> Waiting for Mnesia tables for 30000 ms, 9 retries left
rabbit_1           | 2020-03-15 19:33:46.860 [info] <0.270.0> Waiting for Mnesia tables for 30000 ms, 9 retries left
rabbit_1           | 2020-03-15 19:33:46.862 [info] <0.270.0> Peer discovery backend rabbit_peer_discovery_classic_config does not support registration, skipping registration.
rabbit_1           | 2020-03-15 19:33:46.863 [info] <0.270.0> Running boot step database_sync defined by app rabbit
rabbit_1           | 2020-03-15 19:33:46.863 [info] <0.270.0> Running boot step feature_flags defined by app rabbit
rabbit_1           | 2020-03-15 19:33:46.866 [info] <0.270.0> Running boot step codec_correctness_check defined by app rabbit
rabbit_1           | 2020-03-15 19:33:46.867 [info] <0.270.0> Running boot step external_infrastructure defined by app rabbit
rabbit_1           | 2020-03-15 19:33:46.867 [info] <0.270.0> Running boot step rabbit_registry defined by app rabbit
rabbit_1           | 2020-03-15 19:33:46.869 [info] <0.270.0> Running boot step rabbit_auth_mechanism_cr_demo defined by app rabbit
rabbit_1           | 2020-03-15 19:33:46.869 [info] <0.270.0> Running boot step rabbit_queue_location_random defined by app rabbit
rabbit_1           | 2020-03-15 19:33:46.870 [info] <0.270.0> Running boot step rabbit_event defined by app rabbit
rabbit_1           | 2020-03-15 19:33:46.878 [info] <0.270.0> Running boot step rabbit_auth_mechanism_amqplain defined by app rabbit
rabbit_1           | 2020-03-15 19:33:46.880 [info] <0.270.0> Running boot step rabbit_auth_mechanism_plain defined by app rabbit
rabbit_1           | 2020-03-15 19:33:46.886 [info] <0.270.0> Running boot step rabbit_exchange_type_direct defined by app rabbit
rabbit_1           | 2020-03-15 19:33:46.887 [info] <0.270.0> Running boot step rabbit_exchange_type_fanout defined by app rabbit
rabbit_1           | 2020-03-15 19:33:46.889 [info] <0.270.0> Running boot step rabbit_exchange_type_headers defined by app rabbit
rabbit_1           | 2020-03-15 19:33:46.892 [info] <0.270.0> Running boot step rabbit_exchange_type_topic defined by app rabbit
rabbit_1           | 2020-03-15 19:33:46.893 [info] <0.270.0> Running boot step rabbit_mirror_queue_mode_all defined by app rabbit
rabbit_1           | 2020-03-15 19:33:46.894 [info] <0.270.0> Running boot step rabbit_mirror_queue_mode_exactly defined by app rabbit
rabbit_1           | 2020-03-15 19:33:46.895 [info] <0.270.0> Running boot step rabbit_mirror_queue_mode_nodes defined by app rabbit
rabbit_1           | 2020-03-15 19:33:46.895 [info] <0.270.0> Running boot step rabbit_priority_queue defined by app rabbit
rabbit_1           | 2020-03-15 19:33:46.895 [info] <0.270.0> Priority queues enabled, real BQ is rabbit_variable_queue
rabbit_1           | 2020-03-15 19:33:46.898 [info] <0.270.0> Running boot step rabbit_queue_location_client_local defined by app rabbit
rabbit_1           | 2020-03-15 19:33:46.902 [info] <0.270.0> Running boot step rabbit_queue_location_min_masters defined by app rabbit
rabbit_1           | 2020-03-15 19:33:46.903 [info] <0.270.0> Running boot step kernel_ready defined by app rabbit
rabbit_1           | 2020-03-15 19:33:46.903 [info] <0.270.0> Running boot step rabbit_sysmon_minder defined by app rabbit
rabbit_1           | 2020-03-15 19:33:46.907 [info] <0.270.0> Running boot step rabbit_epmd_monitor defined by app rabbit
rabbit_1           | 2020-03-15 19:33:46.926 [info] <0.270.0> Running boot step guid_generator defined by app rabbit
rabbit_1           | 2020-03-15 19:33:46.945 [info] <0.270.0> Running boot step rabbit_node_monitor defined by app rabbit
nginx_1            | 64.113.20.166 - - [15/Mar/2020:19:33:46 +0000] "GET /api/conversations/slotNames HTTP/1.1" 401 57 "http://35.212.244.136/conversations" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36"
rabbit_1           | 2020-03-15 19:33:46.956 [info] <0.370.0> Starting rabbit_node_monitor
rabbit_1           | 2020-03-15 19:33:46.958 [info] <0.270.0> Running boot step delegate_sup defined by app rabbit
rabbit_1           | 2020-03-15 19:33:46.960 [info] <0.270.0> Running boot step rabbit_memory_monitor defined by app rabbit
rabbit_1           | 2020-03-15 19:33:46.963 [info] <0.270.0> Running boot step core_initialized defined by app rabbit
rabbit_1           | 2020-03-15 19:33:46.963 [info] <0.270.0> Running boot step upgrade_queues defined by app rabbit
nginx_1            | 64.113.20.166 - - [15/Mar/2020:19:33:46 +0000] "GET /api/conversationIntents HTTP/1.1" 401 57 "http://35.212.244.136/conversations" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36"
nginx_1            | 64.113.20.166 - - [15/Mar/2020:19:33:47 +0000] "GET /api/conversationPolicies HTTP/1.1" 401 57 "http://35.212.244.136/conversations" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36"
nginx_1            | 64.113.20.166 - - [15/Mar/2020:19:33:47 +0000] "GET /api/conversationActions HTTP/1.1" 401 57 "http://35.212.244.136/conversations" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36"
nginx_1            | 64.113.20.166 - - [15/Mar/2020:19:33:47 +0000] "GET /api/user HTTP/1.1" 401 57 "http://35.212.244.136/conversations" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36"
nginx_1            | 64.113.20.166 - - [15/Mar/2020:19:33:47 +0000] "GET /api/projects/default/intents?fields[example_hashes]=false&fields[suggestions]=false HTTP/1.1" 401 57 "http://35.212.244.136/conversations" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36"
rabbit_1           | 2020-03-15 19:33:47.090 [info] <0.270.0> Running boot step rabbit_connection_tracking defined by app rabbit
rabbit_1           | 2020-03-15 19:33:47.092 [info] <0.270.0> Running boot step rabbit_connection_tracking_handler defined by app rabbit
rabbit_1           | 2020-03-15 19:33:47.093 [info] <0.270.0> Running boot step rabbit_exchange_parameters defined by app rabbit
rabbit_1           | 2020-03-15 19:33:47.094 [info] <0.270.0> Running boot step rabbit_mirror_queue_misc defined by app rabbit
rabbit_1           | 2020-03-15 19:33:47.098 [info] <0.270.0> Running boot step rabbit_policies defined by app rabbit
rabbit_1           | 2020-03-15 19:33:47.100 [info] <0.270.0> Running boot step rabbit_policy defined by app rabbit
rabbit_1           | 2020-03-15 19:33:47.101 [info] <0.270.0> Running boot step rabbit_queue_location_validator defined by app rabbit
rabbit_1           | 2020-03-15 19:33:47.101 [info] <0.270.0> Running boot step rabbit_vhost_limit defined by app rabbit
rabbit_1           | 2020-03-15 19:33:47.102 [info] <0.270.0> Running boot step rabbit_mgmt_reset_handler defined by app rabbitmq_management
rabbit_1           | 2020-03-15 19:33:47.102 [info] <0.270.0> Running boot step rabbit_mgmt_db_handler defined by app rabbitmq_management_agent
rabbit_1           | 2020-03-15 19:33:47.102 [info] <0.270.0> Management plugin: using rates mode 'basic'
rabbit_1           | 2020-03-15 19:33:47.104 [info] <0.270.0> Running boot step recovery defined by app rabbit
rabbit_1           | 2020-03-15 19:33:47.115 [info] <0.404.0> Making sure data directory '/bitnami/rabbitmq/mnesia/rabbit@localhost/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L' for vhost '/' exists
rabbit_1           | 2020-03-15 19:33:47.181 [info] <0.404.0> Starting message stores for vhost '/'
rabbit_1           | 2020-03-15 19:33:47.184 [info] <0.408.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_transient": using rabbit_msg_store_ets_index to provide index
rabbit_1           | 2020-03-15 19:33:47.203 [info] <0.404.0> Started message store of type transient for vhost '/'
rabbit_1           | 2020-03-15 19:33:47.208 [info] <0.411.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": using rabbit_msg_store_ets_index to provide index
rabbit_1           | 2020-03-15 19:33:47.227 [info] <0.404.0> Started message store of type persistent for vhost '/'
rabbit_1           | 2020-03-15 19:33:47.256 [info] <0.270.0> Running boot step load_definitions defined by app rabbitmq_management
rabbit_1           | 2020-03-15 19:33:47.256 [info] <0.270.0> Running boot step empty_db_check defined by app rabbit
rabbit_1           | 2020-03-15 19:33:47.270 [info] <0.270.0> Running boot step rabbit_looking_glass defined by app rabbit
rabbit_1           | 2020-03-15 19:33:47.271 [info] <0.270.0> Running boot step rabbit_core_metrics_gc defined by app rabbit
rabbit_1           | 2020-03-15 19:33:47.273 [info] <0.270.0> Running boot step background_gc defined by app rabbit
rabbit_1           | 2020-03-15 19:33:47.274 [info] <0.270.0> Running boot step connection_tracking defined by app rabbit
rabbit_1           | 2020-03-15 19:33:47.279 [info] <0.270.0> Setting up a table for connection tracking on this node: tracked_connection_on_node_rabbit@localhost
rabbit_1           | 2020-03-15 19:33:47.286 [info] <0.270.0> Setting up a table for per-vhost connection counting on this node: tracked_connection_per_vhost_on_node_rabbit@localhost
rabbit_1           | 2020-03-15 19:33:47.288 [info] <0.270.0> Running boot step routing_ready defined by app rabbit
rabbit_1           | 2020-03-15 19:33:47.288 [info] <0.270.0> Running boot step pre_flight defined by app rabbit
rabbit_1           | 2020-03-15 19:33:47.289 [info] <0.270.0> Running boot step notify_cluster defined by app rabbit
rabbit_1           | 2020-03-15 19:33:47.290 [info] <0.270.0> Running boot step networking defined by app rabbit
rabbit_1           | 2020-03-15 19:33:47.301 [warning] <0.438.0> Setting Ranch options together with socket options is deprecated. Please use the new map syntax that allows specifying socket options separately from other options.
rabbit_1           | 2020-03-15 19:33:47.310 [info] <0.452.0> started TCP listener on [::]:5672
rabbit_1           | 2020-03-15 19:33:47.312 [info] <0.270.0> Running boot step direct_client defined by app rabbit
rabbit_1           | 2020-03-15 19:33:47.450 [info] <0.502.0> Management plugin: HTTP (non-TLS) listener started on port 15672
rabbit_1           | 2020-03-15 19:33:47.456 [info] <0.608.0> Statistics database started.
rabbit_1           | 2020-03-15 19:33:47.458 [info] <0.607.0> Starting worker pool 'management_worker_pool' with 3 processes in it
nginx_1            | 64.113.20.166 - - [15/Mar/2020:19:33:47 +0000] "GET /api/conversations/inputChannels HTTP/1.1" 401 57 "http://35.212.244.136/conversations" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36"
nginx_1            | 64.113.20.166 - - [15/Mar/2020:19:33:47 +0000] "GET /api/projects/default/entities HTTP/1.1" 401 57 "http://35.212.244.136/conversations" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36"
nginx_1            | 64.113.20.166 - - [15/Mar/2020:19:33:47 +0000] "GET /api/conversations?limit=20&offset=0&intent=&entity=&action=&policies=&slots=&input_channels=&tags_any=&with_tags=true&interactive=true&exclude_self=true HTTP/1.1" 401 57 "http://35.212.244.136/conversations" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36"
nginx_1            | 64.113.20.166 - - [15/Mar/2020:19:33:47 +0000] "GET /api/features HTTP/1.1" 200 2 "http://35.212.244.136/conversations" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36"
nginx_1            | 64.113.20.166 - - [15/Mar/2020:19:33:47 +0000] "POST /api/telemetry HTTP/1.1" 401 57 "http://35.212.244.136/conversations" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36"
rabbit_1           |  completed with 3 plugins.
rabbit_1           | 2020-03-15 19:33:49.291 [info] <0.8.0> Server startup complete; 3 plugins started.
rabbit_1           |  * rabbitmq_management
rabbit_1           |  * rabbitmq_management_agent
rabbit_1           |  * rabbitmq_web_dispatch
nginx_1            | 64.113.20.166 - - [15/Mar/2020:19:33:50 +0000] "POST /api/auth HTTP/1.1" 200 1772 "http://35.212.244.136/login" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36"
nginx_1            | 64.113.20.166 - - [15/Mar/2020:19:33:50 +0000] "GET /api/projects/default/git_repositories HTTP/1.1" 200 169 "http://35.212.244.136/conversations" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36"
nginx_1            | 64.113.20.166 - - [15/Mar/2020:19:33:51 +0000] "GET /api/conversations/slotNames HTTP/1.1" 200 2 "http://35.212.244.136/conversations" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36"
nginx_1            | 64.113.20.166 - - [15/Mar/2020:19:33:51 +0000] "GET /api/user HTTP/1.1" 200 913 "http://35.212.244.136/conversations" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36"
nginx_1            | 64.113.20.166 - - [15/Mar/2020:19:33:51 +0000] "GET /api/conversations/tags HTTP/1.1" 200 2 "http://35.212.244.136/conversations" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36"
rabbit_1           | 2020-03-15 19:33:51.275 [info] <0.618.0> accepting AMQP connection <0.618.0> (172.20.0.6:52892 -> 172.20.0.4:5672)
rabbit_1           | 2020-03-15 19:33:51.316 [info] <0.618.0> connection <0.618.0> (172.20.0.6:52892 -> 172.20.0.4:5672): user 'user' authenticated and granted access to vhost '/'```

diegoluisr · March 16, 2020, 9:41am

Same issue here, when trying to connect to a remote broker aka RabbitMQ in a a different instance. All the params in the endpoints.yml are correct but by default always try to connect to localhost with its Private IP. 172.x.x.x

akelad · March 16, 2020, 12:35pm

@argideritzalpea, could you pin your RASA_X_VERSION to a specific version? e.g. 0.26.1. It’s possible you’ve got an incompatible one running.

@diegoluisr could you create a separate post for your issue, i don’t think it’s related to the one on this post

argideritzalpea · March 22, 2020, 4:55am

Can’t run 26.1 on my install due to this issue:

I am running Rasa X with the following:

RASA_X_VERSION=latest
RASA_VERSION=1.8.0
RASA_X_DEMO_VERSION=0.26.0

Also getting this error:

rasa-x_1           | Job "GitService.run_background_synchronization (trigger: cron[minute='*'], next run at: 2020-03-22 05:04:00 UTC)" raised an exception
rasa-x_1           | Traceback (most recent call last):
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/apscheduler/executors/base.py", line 125, in run_job
rasa-x_1           |     retval = job.func(*job.args, **job.kwargs)
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/rasax/community/services/integrated_version_control/git_service.py", line 851, in run_background_synchronization
rasa-x_1           |     git_service.synchronize_project(force_data_injection)
rasa-x_1           |   File "uvloop/loop.pyx", line 1456, in uvloop.loop.Loop.run_until_complete
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/rasax/community/services/integrated_version_control/git_service.py", line 695, in synchronize_project
rasa-x_1           |     await self._inject_data()
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/rasax/community/services/integrated_version_control/git_service.py", line 725, in _inject_data
rasa-x_1           |     str(self.repository_path()), str(data_path), self.session, SYSTEM_USER
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/rasax/community/initialise.py", line 293, in inject_files_from_disk
rasa-x_1           |     username,
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/rasax/community/initialise.py", line 129, in inject_domain
rasa-x_1           |     rasa_cli_utils.error_and_exit(
rasa-x_1           | AttributeError: module 'rasa.cli.utils' has no attribute 'error_and_exit'

akelad · March 23, 2020, 3:20pm

@argideritzalpea ok the error you’re encountering is an issue with the way we call a function, we’ll fix that. But this indicates that it can’t find your domain file - what does the structure of your git repo look like?

As for your other error, we’ll get back to you on that post

argideritzalpea · March 23, 2020, 5:10pm

@akelad

RielKIM · March 25, 2020, 12:43am

Hi, guys.
I faced same issue when I did manual installation. (Refer Link)
Because I don’t have sudo permission on my dev server. So I struggled during a few days…

Then I tried to quick install by ansible on google vm server. (Refer Link)
Change the server that I have sudo permission.
After $ sudo bash ./install.sh …
Project files were generated then I copy that files to my dev server.

Then I fixed this issue.

VonNeuke · July 7, 2020, 4:00pm

Hello, guy. Could you please show me what files were generated when you install by ansible? I faced same issue when I did manual installation, andI have sudo permission on my server.

emorycardozo · August 7, 2020, 8:31pm

I have same problem.

My .env:

enviroments.yml:

rasa:
  production:
     url: ${RASA_PRODUCTION_HOST}
     token: ${RASA_TOKEN}
 worker:
     url: ${RASA_WORKER_HOST}
     token: ${RASA_TOKEN}

log rasa-x:

    Attaching to rasa_rasa-x_1
    rasa-x_1           | INFO:alembic.runtime.migration:Context impl PostgresqlImpl.
    rasa-x_1           | INFO:alembic.runtime.migration:Will assume transactional DDL.
    rasa-x_1           | INFO:rasax.community.services.event_service:Starting event service (standalone: False).
    rasa-x_1           | INFO:rasax.community.services.integrated_version_control.git_service:Cloning 
    git repository from URL 'git@github.com:emorycardozo/bot-rasa-teste.git'.
    rasa-x_1           | /usr/local/lib/python3.7/site- 
    packages/rasax/community/services/test_service.py:43: UserWarning: Unable to get tests from 
    /app/git/1/tests/conversation_tests.md:
    rasa-x_1           | File '/app/git/1/tests/conversation_tests.md' does not exist. Please, make sure 
    you have end-to-end tests added to your assistant. See https://rasa.com/docs/rasa-x/user- 
    guide/test-assistant for more information.
    rasa-x_1           |   UserWarning,
    rasa-x_1           | wow
    rasa-x_1           | wow
    rasa-x_1           | Starting Rasa X server... 🚀
    rasa-x_1           | INFO:rasax.community.services.event_consumers.pika_consumer:Start 
    consuming queue 'rasa_production_events' on pika host 'rabbit'.
    rasa-x_1           | ERROR:pika.adapters.blocking_connection:Unexpected connection close 
    detected: ConnectionClosedByBroker: (320) "CONNECTION_FORCED - broker forced 
    connection closure with reason 'shutdown'"
    rasa-x_1           | ERROR:rasax.community.services.event_service:Caught an exception while 
    consuming events. Will retry in 5 s.
    rasa-x_1           | Traceback (most recent call last):
    rasa-x_1           |   File "/usr/local/lib/python3.7/site- 
    packages/rasax/community/services/event_service.py", line 1667, in continuously_consume
    rasa-x_1           |     consumer.consume()
    rasa-x_1           |   File "/usr/local/lib/python3.7/site- 
    packages/rasax/community/services/event_consumers/pika_consumer.py", line 180, in consume
    rasa-x_1           |     self.channel.start_consuming()
    rasa-x_1           |   File "/usr/local/lib/python3.7/site- 
    packages/pika/adapters/blocking_connection.py", line 1866, in start_consuming
    rasa-x_1           |     self._process_data_events(time_limit=None)
    rasa-x_1           |   File "/usr/local/lib/python3.7/site- 
    packages/pika/adapters/blocking_connection.py", line 2027, in _process_data_events
    rasa-x_1           |     self.connection.process_data_events(time_limit=time_limit)
    rasa-x_1           |   File "/usr/local/lib/python3.7/site- 
   packages/pika/adapters/blocking_connection.py", line 825, in process_data_events
   rasa-x_1           |     self._flush_output(common_terminator)
   rasa-x_1           |   File "/usr/local/lib/python3.7/site- 
   packages/pika/adapters/blocking_connection.py", line 522, in _flush_output
   rasa-x_1           |     raise self._closed_result.value.error
   rasa-x_1           | pika.exceptions.ConnectionClosedByBroker: (320, "CONNECTION_FORCED - 
   broker forced connection closure with reason 'shutdown'")
   rasa-x_1           | INFO:rasax.community.services.event_consumers.pika_consumer:Start 
   consuming queue 'rasa_production_events' on pika host 'rabbit'.

Iraklis · September 22, 2020, 12:14pm

Was anyone able to solve this problem? I’ve looked evereywhere and I still haven’t found a coherent solution to this. I’ve tried installing rasa x in multiple machines and with various versions, but I was unable to make it work. Every configuration was the default, so nothing weird or out of the ordinary, It just seems that installing with docker just doens’t want to work with my machines.

Ian · October 10, 2020, 6:45pm

Hey,

Im facing the same issue. I get the message Your model training has failed form pressing the train model on my rasa x server. Integration with my github repo is successful and I can view domain and all other files. rasa worker logs are as shown below

rasa-worker_1      | /opt/venv/lib/python3.7/site-packages/rasa/core/brokers/pika.py:294: FutureWarning: Your Pika event broker config contains the deprecated `queu
e` key. Please use the `queues` key instead.
rasa-worker_1      |   self.queues = self._get_queues_from_args(queues, kwargs)
rasa-worker_1      | 2020-10-10 17:46:00 ERROR    pika.adapters.utils.io_services_utils  - Socket failed to connect: <socket.socket fd=21, family=AddressFamily.AF_I
NET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.20.0.7', 43652)>; error=111 (Connection refused)
rasa-worker_1      | 2020-10-10 17:46:00 ERROR    pika.adapters.utils.connection_workflow  - TCP Connection attempt failed: ConnectionRefusedError(111, 'Connection 
refused'); dest=(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('172.20.0.3', 5672))
rasa-worker_1      | 2020-10-10 17:46:00 ERROR    pika.adapters.utils.connection_workflow  - AMQPConnector - reporting failure: AMQPConnectorSocketConnectError: Con
nectionRefusedError(111, 'Connection refused')
rasa-worker_1      | 2020-10-10 17:46:05 ERROR    pika.adapters.utils.io_services_utils  - Socket failed to connect: <socket.socket fd=25, family=AddressFamily.AF_I
NET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.20.0.7', 43668)>; error=111 (Connection refused)
rasa-worker_1      | 2020-10-10 17:46:05 ERROR    pika.adapters.utils.connection_workflow  - TCP Connection attempt failed: ConnectionRefusedError(111, 'Connection 
refused'); dest=(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('172.20.0.3', 5672))
rasa-worker_1      | 2020-10-10 17:46:05 ERROR    pika.adapters.utils.connection_workflow  - AMQPConnector - reporting failure: AMQPConnectorSocketConnectError: Con
nectionRefusedError(111, 'Connection refused')

My .env

PASSWORD_SALT=nw_tHX_SCxFMicX
RASA_X_VERSION=0.32.2
RASA_VERSION=1.10.10
RASA_X_DEMO_VERSION=0.32.0
REDIS_PASSWORD=88XWb1aIWfEZD_K
RABBITMQ_PASSWORD=cvbIKmIEubNBnjo
RASA_X_TOKEN=DosKKUOBUGEl1i2
JWT_SECRET=80RgsY2Sf3swzYIrcvJGMpgGy
RASA_TOKEN=HiEKhL24081qtmD
DB_PASSWORD=cVh0BMpM5cEm7eS

my environment.yml

rasa:
  production:
    url: ${RASA_PRODUCTION_HOST}
    token: ${RASA_TOKEN}
  worker:
    url: ${RASA_WORKER_HOST}
    token: ${RASA_TOKEN}

amardeep · November 5, 2020, 6:49am

I am facing similar error where rasa x server cannot reach rasa worker and rasa worker cannot fetch config from rasa x.

Installation type : Docker Compose manual installation
Host : Windows 10 - 1809
Docker : 19.03.13

RASA_X_VERSION=0.32.2
RASA_VERSION=1.10.16
RASA_X_DEMO_VERSION=0.32.0

Errors in Rasa worker logs

Starting Rasa X in production mode…

Could not fetch runtime config from server at ‘http://rasa-x:5002/api/config?token=EWqzovCplp7L+9zqYmgBBg==’. Exiting.

2020-11-04 14:07:55 DEBUG rasa.cli.x - Failed to get a proper response from remote server. Status Code: 401. Response: ‘{“reasons”:[“Authorization header not present.”],“exception”:“Unauthorized”}’

2020-11-04 14:08:00 DEBUG rasa.cli.x - Failed to get a proper response from remote server. Status Code: 401. Response: ‘{“reasons”:[“Authorization header not present.”],“exception”:“Unauthorized”}’

Till now I have tried RASA_X_VERSION=0.32.2 with RASA ranging 1.10.10 to 1.10.16 . This is kind of disappointing that we are not able to even start using the framework. I even tried version 0.33 with rasa 2.0.3 but there I faced permission error while fetching config .

Topic		Replies	Views
Rasa X setup rabbit error [Deprecated] Rasa X Community Edition	1	919	March 23, 2020
0.26.1 rasa x error in python alembic revisions [Deprecated] Rasa X Community Edition	17	2838	February 20, 2021
Errors in rasa-production [Deprecated] Rasa X Community Edition	4	1586	January 23, 2020
RASA X - Quick Install Method (Fresh installation) model not training [Deprecated] Rasa X Community Edition	9	713	June 17, 2020
RabbitMQ connection refused [Deprecated] Rasa X Community Edition	3	1592	September 8, 2020

Rasa X does not connect to Production/Worker hosts

Related topics