Connecting to External Postgres Database

Hi, I am deploying through helm charts on kubernetes. I wanted to connect to an external Postgres database and have created it on GCloud for use by Rasa-X, also created the tracker and worker-tracker database. I’ve created an external secret and added the required configurations for the external database as mentioned in the docs. Please find my values.yml as under-

rasax:
    tag: "0.40.1"
    initialUser:
        password: "hidden"
    passwordSalt: "something"
    token: "something"
    jwtSecret: "hidden"
rasa:
    token: "something"
    tag: "2.6.2-full"
    useLoginDatabase: false
    versions:
        rasaProduction:
            trackerDatabase: "tracker"
        rasaWorker:
            trackerDatabase: "worker_tracker"
rasaSecret: "chatbot-rasa-external"
debugMode: true
rabbitmq:
    rabbitmq:
        password: "hidden"
postgresql:
    install: false
    existingHost: "hidden"
    existingSecretKey: "databasePassword"
global:
    postgresql:
        postgresqlUsername: "me"
        existingSecret: "chatbot-rasa-external"
        postgresqlDatabase: "rasa"
        servicePort: 5432
    redis:
        password: "hidden"

But I am getting the following error from the event-service logs. I am not able to proceed further. I have tried deleting the older deployment and the PVCs and then tried redeploying. But it does not work. Also tried deleting old deployment, PVCs and all tables created in external Postgres DB and tried deployed fresh from the start. Still no luck :pensive:. Please help.

@stephens Can you please help with this?

@akelad

Can you duplicate this the default postgres image setup?

Yes, the default way to deploy postgres in dockerised environment using its subcharts works fine. But I wanted to use an external database, is it not an option anymore?

External should work fine so it must be something in the config parameters or db access.

From inside the rasa-x container, can you successfully curl the Postgres endpoint?

Thanks for replying. The Rasa-X container is able to curl the Postgres endpoint. Also, when I tried to deploy with the external Postgres database, the tables were also created in the Database.

However, the deployment fails with the issue that I mentioned above, that is.

kubectl logs chatbot-rasa-x-9747cbb6b-fvvvj -n chatbot-rasa
DEBUG:rasax.community.server:Starting API service.
DEBUG:git.cmd:Popen(['git', 'version'], cwd=/app, universal_newlines=False, shell=None, istream=None)
DEBUG:git.cmd:Popen(['git', 'version'], cwd=/app, universal_newlines=False, shell=None, istream=None)
DEBUG:rasax.community.utils.common:Generating JWT RSA key pair.
DEBUG:rasax.community.utils.common:Saved RSA private key to temporary file '/tmp/tmpzhvraxij'.
DEBUG:sanic_jwt.configuration:validating provided secret
DEBUG:sanic_jwt.configuration:validating keys (if needed)
DEBUG:sanic_jwt.configuration:loading secret and/or keys (if needed)
Starting Rasa X server... 🚀
[2021-06-17 10:16:00 +0000] [9] [INFO] Goin' Fast @ http://0.0.0.0:5002
DEBUG:rasax.community.server:Environment variable 'RUN_DATABASE_MIGRATION_AS_SEPARATE_SERVICE' set to 'True', meaning Rasa X expects the database migration to run as a separate service.
DEBUG:rasax.community.database.utils:DB revision(s) do not match migration scripts revision(s):
DEBUG:rasax.community.database.utils:DB revision: ['6fce9679db61']
DEBUG:rasax.community.database.utils:Migration scripts revision: ['5ff5fa10d3c0']
DEBUG:rasax.community.database.utils:DB revision(s) do not match migration scripts revision(s):
DEBUG:rasax.community.database.utils:DB revision: ['6fce9679db61']
DEBUG:rasax.community.database.utils:Migration scripts revision: ['5ff5fa10d3c0']
DEBUG:rasax.community.database.utils:Database revision does not match migrations' latest, trying again in 4 seconds.
DEBUG:rasax.community.database.utils:Database revision does not match migrations' latest, trying again in 4 seconds.
DEBUG:rasax.community.database.utils:DB revision(s) do not match migration scripts revision(s):
DEBUG:rasax.community.database.utils:DB revision: ['6fce9679db61']
DEBUG:rasax.community.database.utils:Migration scripts revision: ['5ff5fa10d3c0']
DEBUG:rasax.community.database.utils:Database revision does not match migrations' latest, trying again in 4 seconds.
DEBUG:rasax.community.database.utils:DB revision(s) do not match migration scripts revision(s):
DEBUG:rasax.community.database.utils:DB revision: ['6fce9679db61']
DEBUG:rasax.community.database.utils:Migration scripts revision: ['5ff5fa10d3c0']
DEBUG:rasax.community.database.utils:Database revision does not match migrations' latest, trying again in 4 seconds.
DEBUG:rasax.community.database.utils:DB revision(s) do not match migration scripts revision(s):
DEBUG:rasax.community.database.utils:DB revision: ['6fce9679db61']
DEBUG:rasax.community.database.utils:Migration scripts revision: ['5ff5fa10d3c0']
DEBUG:rasax.community.database.utils:Database revision does not match migrations' latest, trying again in 4 seconds.
ERROR:rasax.community.caches.redis_cache:Connection error while initializing RedisCache, error: Error 111 connecting to chatbot-redis-master:6379. Connection refused.
ERROR:rasax.community.caches.redis_cache:Connection error while initializing RedisCache, error: Error 111 connecting to chatbot-redis-master:6379. Connection refused.
DEBUG:rasax.community.caches.cached_query:Rasa X will use <class 'rasax.community.caches.redis_cache.RedisCache'> for caching queries.
ERROR:rasax.community.caches.redis_cache:Connection error while initializing RedisCache, error: Error 111 connecting to chatbot-redis-master:6379. Connection refused.
DEBUG:rasax.community.caches.cached_query:Rasa X will use <class 'rasax.community.caches.redis_cache.RedisCache'> for caching queries.
DEBUG:rasax.community.caches.cached_query:Rasa X will use <class 'rasax.community.caches.redis_cache.RedisCache'> for caching queries.
ERROR:rasax.community.caches.redis_cache:Connection error while initializing RedisCache, error: Error 111 connecting to chatbot-redis-master:6379. Connection refused.
DEBUG:rasax.community.caches.cached_query:Rasa X will use <class 'rasax.community.caches.redis_cache.RedisCache'> for caching queries.
[2021-06-17 10:16:03 +0000] [21] [INFO] Starting worker [21]
[2021-06-17 10:16:03 +0000] [23] [INFO] Starting worker [23]
[2021-06-17 10:16:03 +0000] [20] [INFO] Starting worker [20]
[2021-06-17 10:16:03 +0000] [25] [INFO] Starting worker [25]
DEBUG:rasax.community.database.utils:DB revision(s) do not match migration scripts revision(s):
DEBUG:rasax.community.database.utils:DB revision: ['6fce9679db61']
DEBUG:rasax.community.database.utils:Migration scripts revision: ['5ff5fa10d3c0']
DEBUG:rasax.community.database.utils:Database revision does not match migrations' latest, trying again in 4 seconds.
WARNING:rasax.community.database.utils:Unable to get database revision heads.
DEBUG:rasax.community.database.utils:DB revision(s) do not match migration scripts revision(s):
DEBUG:rasax.community.database.utils:DB revision: None
DEBUG:rasax.community.database.utils:Migration scripts revision: ['5ff5fa10d3c0']
DEBUG:rasax.community.database.utils:Database revision does not match migrations' latest, trying again in 4 seconds.
DEBUG:rasax.community.database.utils:DB revision(s) do not match migration scripts revision(s):
DEBUG:rasax.community.database.utils:DB revision: ['6fce9679db61']
DEBUG:rasax.community.database.utils:Migration scripts revision: ['5ff5fa10d3c0']
DEBUG:rasax.community.database.utils:Database revision does not match migrations' latest, trying again in 4 seconds.
WARNING:rasax.community.database.utils:Unable to get database revision heads.
DEBUG:rasax.community.database.utils:DB revision(s) do not match migration scripts revision(s):
DEBUG:rasax.community.database.utils:DB revision: None
DEBUG:rasax.community.database.utils:Migration scripts revision: ['5ff5fa10d3c0']
DEBUG:rasax.community.database.utils:Database revision does not match migrations' latest, trying again in 4 seconds.
WARNING:rasax.community.database.utils:Unable to get database revision heads.
DEBUG:rasax.community.database.utils:DB revision(s) do not match migration scripts revision(s):
DEBUG:rasax.community.database.utils:DB revision: None
DEBUG:rasax.community.database.utils:Migration scripts revision: ['5ff5fa10d3c0']
DEBUG:rasax.community.database.utils:Database revision does not match migrations' latest, trying again in 4 seconds.
DEBUG:rasax.community.database.utils:DB revision(s) do not match migration scripts revision(s):
DEBUG:rasax.community.database.utils:DB revision: ['6fce9679db61']
DEBUG:rasax.community.database.utils:Migration scripts revision: ['5ff5fa10d3c0']
DEBUG:rasax.community.database.utils:Database revision does not match migrations' latest, trying again in 4 seconds.
WARNING:rasax.community.database.utils:Unable to get database revision heads.
DEBUG:rasax.community.database.utils:DB revision(s) do not match migration scripts revision(s):
DEBUG:rasax.community.database.utils:DB revision: None
DEBUG:rasax.community.database.utils:Migration scripts revision: ['5ff5fa10d3c0']
WARNING:rasax.community.database.utils:Unable to get database revision heads.
DEBUG:rasax.community.database.utils:DB revision(s) do not match migration scripts revision(s):
DEBUG:rasax.community.database.utils:DB revision: None
DEBUG:rasax.community.database.utils:Migration scripts revision: ['5ff5fa10d3c0']
DEBUG:rasax.community.database.utils:Database revision does not match migrations' latest, trying again in 4 seconds.
DEBUG:rasax.community.database.utils:Database revision does not match migrations' latest, trying again in 4 seconds.
WARNING:rasax.community.database.utils:Unable to get database revision heads.
DEBUG:rasax.community.database.utils:DB revision(s) do not match migration scripts revision(s):
DEBUG:rasax.community.database.utils:DB revision: None
DEBUG:rasax.community.database.utils:Migration scripts revision: ['5ff5fa10d3c0']
DEBUG:rasax.community.database.utils:Database revision does not match migrations' latest, trying again in 4 seconds.
DEBUG:rasax.community.database.utils:DB revision(s) do not match migration scripts revision(s):
DEBUG:rasax.community.database.utils:DB revision: ['6fce9679db61']
DEBUG:rasax.community.database.utils:Migration scripts revision: ['5ff5fa10d3c0']
DEBUG:rasax.community.database.utils:Database revision does not match migrations' latest, trying again in 4 seconds.
[2021-06-17 10:16:09 +0000] - (sanic.access)[INFO][10.12.9.1:51654]: GET http://10.12.9.164:5002/  302 0
INFO:sanic.access:

Thanks for confirming all of that. I would stop rasa, drop the tables and re-try. Make sure you have the latest Rasa versions before re-starting.

Thanks a lot, @stephens. It worked for me this time. I deleted the helm deployment and all objects in the external postgresql DB. Then I upgraded to the latest versions and deployed.

# rasax specific settings
rasax:
    # Rasa X configuration you did in previous steps
    # ...
    tag: "0.41.1"
# rasa: Settings common for all Rasa containers
rasa:
    # Rasa Open Source configuration you did in previous steps
    # ...
    # tag refers to the Rasa image tag
    tag: "2.7.1-full"

I am also facing the same issue - I deleted all the helm deployment and dropped the rasa db and recreated the DB but no luck-Please help here… rasa-x

DEBUG:rasax.community.database.utils:DB revision(s) do not match migration scripts revision(s):

rasa-x

DEBUG:rasax.community.database.utils:DB revision: [‘45546b047abe’]

rasa-x

DEBUG:rasax.community.database.utils:Migration scripts revision: [‘652500998f3e’]

rasa-x

DEBUG:rasax.community.database.utils:Database revision does not match migrations’ latest, trying again in 4 seconds.

Hi @andyz, Which DB are you using? Does your tables get created even if the deployment fails? In my case, they got created. Also, there were indexes and sequences there! I had to delete the indexes and sequences too after deleting the tables. Only after that, the deployment worked for me. Also, make sure, you are using the rasax/rasa versions that I mentioned above.

I am using the same above versions tag: “0.41.1” 2.7.1-full I dropped the Postgres DB in-fact and tried to install it creates all tables but the rasa_x_user does not have an entry of the user and then when I copy the migration version into the alembic_version table as 97280f5b6803 it stops giving the error but since the user is not created other errors start popping out

Process ForkProcess-2: rasa-x Traceback (most recent call last): rasa-x File “/usr/lib/python3.8/multiprocessing/process.py”, line 315, in _bootstrap rasa-x self.run() rasa-x

File “/usr/lib/python3.8/multiprocessing/process.py”, line 108, in run rasa-x self._target(*self._args, **self._kwargs) rasa-x

File “/usr/local/lib/python3.8/dist-packages/rasax/community/utils/common.py”, line 848, in _run_in_process_target rasa-x fn(*args) rasa-x

File “/usr/local/lib/python3.8/dist-packages/rasax/community/server.py”, line 90, in _initialize_server_mode rasa-x password, is_password_generated = initialise.create_community_user(session) rasa-x

File “/usr/local/lib/python3.8/dist-packages/rasax/community/initialise.py”, line 229, in create_community_user rasa-x role_service.init_roles(project_id=constants.COMMUNITY_PROJECT_NAME) rasa-x

File “/usr/local/lib/python3.8/dist-packages/rasax/community/services/role_service.py”, line 418, in init_roles rasa-x self.save_permissions_for_role(role, permissions, project_id) rasa-x

File “/usr/local/lib/python3.8/dist-packages/rasax/community/services/role_service.py”, line 530, in save_permissions_for_role rasa-x (project_id,) = ( rasa-x TypeError: cannot unpack non-iterable NoneType object

Hi @andyz, we just released Rasa X version 0.42.0 recently. Are you able to update your Rasa X version and give this a try? Let me know if this resolves the issue, or if additional troubleshooting is needed.