Not able to start rasa x in production mode

Hello everyone. I have a very simple Docker script for my chatbot (I am relatively new to Docker). So far everything has been working great and I deployed it successfully to a server. However, I need the integrated version control feature of rasa x so my customer can train her model on production. For that, I need to run rasa x in production mode (rasa x --production). This doesn’t work however. rasa runs fine on port 5005 but rasax is not available at all (nothing running on port 5002). This guy had the same issue: https://forum.rasa.com/t/rasa-x-production-command-is-not-loading-ui/26920

Here are my Docker and Compose files:

docker-compose

services:
  rasa:
    container_name: rasax
    build: .
    networks: [ 'rasa-network' ]
    ports:
      - "5005:5005"
      - "5002:5002"
    working_dir: /root
    volumes:
      - ./:/root:cached
    environment:
      - RASA_X_PASSWORD=${RASA_X_PASSWORD}
      - RASA_TOKEN=${RASA_TOKEN}
      - PASSWORD_SALT=${PASSWORD_SALT}
      - JWT_SECRET=${JWT_SECRET}
    #FOR PRODUCTION, ADD THIS TAG: --production
    command: rasa x --production --no-prompt --enable-api --endpoints endpoints.yml --port 5005 --cors "*" --debug --log-file LOG_FILE
  action_server:
    image: rasa/rasa-sdk:latest
    networks: [ 'rasa-network' ]
    ports:
      - "5055:5055"
    volumes:
      - "./actions:/app/actions"

networks: { rasa-network: { } }

Dockerfile

# FROM registry.gitlab.com/youengineering/intern/craftcms/craft-shared/build:latest


FROM python:3.8

# install dependencies
WORKDIR /root

RUN python -m pip install --upgrade pip==20.2

COPY requirements.txt .
RUN pip install -r requirements.txt

# install nlp model spacy
RUN pip install rasa-x --extra-index-url https://pypi.rasa.com/simple
RUN pip install --default-timeout=1800 "https://github.com/explosion/spacy-models/releases/download/de_dep_news_trf-3.0.0/de_dep_news_trf-3.0.0-py3-none-any.whl"

# expose port for rasa server
EXPOSE 5005

# expose port for rasa X server
EXPOSE 5002

# expose port for ssh
EXPOSE 22
EXPOSE 443
EXPOSE 80

The only thing I spotted in the logs is the warning

Agent could not be loaded with the provided configuration. Load default agent without any model.

What am I missing? Is it looking for a model with a production tag on a production server?

Here are the full logs:

docker compose up
[+] Running 2/2
 ⠿ Container ip5-auxilio-rasa_action_server_1  Started                                                                                                                                                                             2.2s
 ⠿ Container rasax                             Started                                                                                                                                                                             2.1s
Attaching to action_server_1, rasax
action_server_1  | 2021-08-17 19:36:20 INFO     rasa_sdk.endpoint  - Starting action endpoint server...
action_server_1  | 2021-08-17 19:36:20 INFO     rasa_sdk.executor  - Registered function for 'action_show_time'.
action_server_1  | 2021-08-17 19:36:20 INFO     rasa_sdk.endpoint  - Action endpoint is up and running on http://0.0.0.0:5055
rasax            | Starting Rasa X in production mode... 🚀
rasax            | 2021-08-17 19:36:26 DEBUG    matplotlib  - (private) matplotlib data path: /usr/local/lib/python3.8/site-packages/matplotlib/mpl-data
rasax            | 2021-08-17 19:36:26 DEBUG    matplotlib  - matplotlib data path: /usr/local/lib/python3.8/site-packages/matplotlib/mpl-data
rasax            | 2021-08-17 19:36:27 DEBUG    matplotlib  - CONFIGDIR=/root/.config/matplotlib
rasax            | 2021-08-17 19:36:27 DEBUG    matplotlib  - matplotlib version 3.3.4
rasax            | 2021-08-17 19:36:27 DEBUG    matplotlib  - interactive is False
rasax            | 2021-08-17 19:36:27 DEBUG    matplotlib  - platform is linux
rasax            | 2021-08-17 19:36:27 DEBUG    rasa.cli.utils  - Parameter 'credentials' not set. Using default location 'credentials.yml' instead.
rasax            | 2021-08-17 19:36:27 DEBUG    rasa.core.utils  - Available web server routes: 
rasax            | /conversations/<conversation_id:path>/messages     POST                           add_message
rasax            | /conversations/<conversation_id:path>/tracker/events POST                           append_events
rasax            | /webhooks/rasa                                     GET                            custom_webhook_RasaChatInput.health
rasax            | /webhooks/rasa/webhook                             POST                           custom_webhook_RasaChatInput.receive
rasax            | /webhooks/rest                                     GET                            custom_webhook_RestInput.health
rasax            | /webhooks/rest/webhook                             POST                           custom_webhook_RestInput.receive
rasax            | /model/test/intents                                POST                           evaluate_intents
rasax            | /model/test/stories                                POST                           evaluate_stories
rasax            | /conversations/<conversation_id:path>/execute      POST                           execute_action
rasax            | /domain                                            GET                            get_domain
rasax            | /socket.io                                         POST                           handle_request
rasax            | /                                                  GET                            hello
rasax            | /model                                             PUT                            load_model
rasax            | /model/parse                                       POST                           parse
rasax            | /conversations/<conversation_id:path>/predict      POST                           predict
rasax            | /conversations/<conversation_id:path>/tracker/events PUT                            replace_events
rasax            | /conversations/<conversation_id:path>/story        GET                            retrieve_story
rasax            | /conversations/<conversation_id:path>/tracker      GET                            retrieve_tracker
rasax            | /webhooks/socketio                                 GET                            socketio_webhook.health
rasax            | /status                                            GET                            status
rasax            | /model/predict                                     POST                           tracker_predict
rasax            | /model/train                                       POST                           train
rasax            | /conversations/<conversation_id:path>/trigger_intent POST                           trigger_intent
rasax            | /model                                             DELETE                         unload_model
rasax            | /version                                           GET                            version
rasax            | 2021-08-17 19:36:27 INFO     root  - Starting Rasa server on http://localhost:5005
rasax            | 2021-08-17 19:36:27 DEBUG    rasa.core.utils  - Using the default number of Sanic workers (1).
rasax            | 2021-08-17 19:36:27 DEBUG    urllib3.connectionpool  - Starting new HTTPS connection (1): api.segment.io:443
rasax            | 2021-08-17 19:36:28 DEBUG    urllib3.connectionpool  - https://api.segment.io:443 "POST /v1/track HTTP/1.1" 200 21
rasax            | 2021-08-17 19:36:28 INFO     root  - Enabling coroutine debugging. Loop id 94535023215920.
rasax            | 2021-08-17 19:36:28 DEBUG    root  - Could not load interpreter from 'None'.
rasax            | 2021-08-17 19:36:28 DEBUG    rasa.core.tracker_store  - Connected to InMemoryTrackerStore.
rasax            | 2021-08-17 19:36:28 DEBUG    rasa.core.lock_store  - Connected to lock store 'InMemoryLockStore'.
rasax            | /usr/local/lib/python3.8/site-packages/rasa/shared/utils/io.py:97: UserWarning: No valid configuration given to load agent.
rasax            | /usr/local/lib/python3.8/site-packages/rasa/shared/utils/io.py:97: UserWarning: Agent could not be loaded with the provided configuration. Load default agent without any model.
rasax            | 2021-08-17 19:36:28 DEBUG    rasa.core.nlg.generator  - Instantiated NLG to 'TemplatedNaturalLanguageGenerator'.
rasax            | 2021-08-17 19:36:28 INFO     root  - Rasa server is up and running.

@niki can you use fixed image of rasa and rasa-sdk from docker hub rather then latest?

hey @nik202 I set the rasa-sdk version to 2.8.0 now. For rasa/rasax I am not using their docker image but rather installing it in my python docker container via pip command. I fixed the version like this now: RUN pip install rasa-x==0.42.0 --extra-index-url https://pypi.rasa.com/simple

any ideas what I need to set up for production mode to work?

@niki can you use 0.39.3? well I not use Rasa-X much, but I use docker so trying to help you as much as I can.

pip install rasa-x==0.39.3 --extra-index-url https://pypi.rasa.com/simple @niki or if rasa and rasa-sdk 2.8.1?

@niki some requirement you need to add such as

SQLAlchemy==1.3.22

sanic-jwt==1.6.0

hm seems it doesn’t want to run with those versions:

Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
[+] Running 3/3
 ⠿ Network ip5-auxilio-rasa_rasa-network       Created                                                                                                                                                                             0.4s
 ⠿ Container rasax                             Started                                                                                                                                                                             1.7s
 ⠿ Container ip5-auxilio-rasa_action_server_1  Started                                                                                                                                                                             1.8s
Attaching to action_server_1, rasax
action_server_1  | 2021-08-17 21:33:05 INFO     rasa_sdk.endpoint  - Starting action endpoint server...
action_server_1  | 2021-08-17 21:33:05 INFO     rasa_sdk.executor  - Registered function for 'action_show_time'.
action_server_1  | 2021-08-17 21:33:05 INFO     rasa_sdk.endpoint  - Action endpoint is up and running on http://0.0.0.0:5055
rasax            | 2021-08-17 21:33:06 DEBUG    rasax.community.config  - Environment variable `RASA_X_HOST ` is not set, defaulting to 'http://localhost:5002'. 
rasax            | 2021-08-17 21:33:06 DEBUG    git.cmd  - Popen(['git', 'version'], cwd=/root, universal_newlines=False, shell=None, istream=None)
rasax            | 2021-08-17 21:33:07 DEBUG    urllib3.connectionpool  - Starting new HTTPS connection (1): api.segment.io:443
rasax            | 2021-08-17 21:33:08 DEBUG    urllib3.connectionpool  - https://api.segment.io:443 "POST /v1/track HTTP/1.1" 200 21
rasax            | By adding the '--no_prompt' parameter you agreed to the Rasa X license agreement (https://storage.cloud.google.com/rasa-x-releases/rasa_x_ce_license_agreement.pdf)
rasax            | 2021-08-17 21:33:08 DEBUG    rasax.community.utils.common  - Generating JWT RSA key pair.
rasax            | 2021-08-17 21:33:08 DEBUG    rasax.community.telemetry  - Started consuming telemetry events.
rasax            | 2021-08-17 21:33:08 DEBUG    rasax.community.utils.common  - Saved RSA private key to temporary file '/tmp/tmpq6uysnpd'.
rasax            | 2021-08-17 21:33:08 DEBUG    sanic_jwt.configuration  - validating provided secret
rasax            | 2021-08-17 21:33:08 DEBUG    sanic_jwt.configuration  - validating keys (if needed)
rasax            | 2021-08-17 21:33:08 DEBUG    sanic_jwt.configuration  - loading secret and/or keys (if needed)
rasax            | 2021-08-17 21:33:08 DEBUG    rasax.community.sql_migrations  - Start running schema migrations.
rasax            | 2021-08-17 21:33:08 DEBUG    rasax.community.database.utils  - Turned SQLite foreign key enforcement on.
rasax            | 2021-08-17 21:33:08 DEBUG    rasax.community.database.utils  - Turned on SQLite WAL mode.
rasax            | 2021-08-17 21:33:08 DEBUG    rasax.community.database.utils  - Turned SQLite foreign key enforcement off.
rasax            | 2021-08-17 21:33:08 DEBUG    rasax.community.database.utils  - Turned on SQLite WAL mode.
rasax            | Starting Rasa X in local mode... 🚀
rasax            | Traceback (most recent call last):
rasax            |   File "/usr/local/lib/python3.8/site-packages/alembic/script/base.py", line 162, in _catch_revision_errors
rasax            |     yield
rasax            |   File "/usr/local/lib/python3.8/site-packages/alembic/script/base.py", line 364, in _upgrade_revs
rasax            |     revs = list(revs)
rasax            |   File "/usr/local/lib/python3.8/site-packages/alembic/script/revision.py", line 765, in _iterate_revisions
rasax            |     requested_lowers = self.get_revisions(lower)
rasax            |   File "/usr/local/lib/python3.8/site-packages/alembic/script/revision.py", line 319, in get_revisions
rasax            |     return sum([self.get_revisions(id_elem) for id_elem in id_], ())
rasax            |   File "/usr/local/lib/python3.8/site-packages/alembic/script/revision.py", line 319, in <listcomp>
rasax            |     return sum([self.get_revisions(id_elem) for id_elem in id_], ())
rasax            |   File "/usr/local/lib/python3.8/site-packages/alembic/script/revision.py", line 322, in get_revisions
rasax            |     return tuple(
rasax            |   File "/usr/local/lib/python3.8/site-packages/alembic/script/revision.py", line 323, in <genexpr>
rasax            |     self._revision_for_ident(rev_id, branch_label)
rasax            |   File "/usr/local/lib/python3.8/site-packages/alembic/script/revision.py", line 386, in _revision_for_ident
rasax            |     raise ResolutionError(
rasax            | alembic.script.revision.ResolutionError: No such revision or branch '652500998f3e'
rasax            | 
rasax            | The above exception was the direct cause of the following exception:
rasax            | 
rasax            | Traceback (most recent call last):
rasax            |   File "/usr/local/lib/python3.8/site-packages/rasa/cli/x.py", line 495, in run_locally
rasax            |     local.main(
rasax            |   File "/usr/local/lib/python3.8/site-packages/rasax/community/local.py", line 234, in main
rasax            |     sql_migrations.run_migrations(session)
rasax            |   File "/usr/local/lib/python3.8/site-packages/rasax/community/sql_migrations.py", line 25, in run_migrations
rasax            |     _run_schema_migrations(session)
rasax            |   File "/usr/local/lib/python3.8/site-packages/rasax/community/sql_migrations.py", line 41, in _run_schema_migrations
rasax            |     _run_alembic_migration(alembic_config)
rasax            |   File "/usr/local/lib/python3.8/site-packages/rasax/community/sql_migrations.py", line 65, in _run_alembic_migration
rasax            |     command.upgrade(alembic_config, target_revision)
rasax            |   File "/usr/local/lib/python3.8/site-packages/alembic/command.py", line 298, in upgrade
rasax            |     script.run_env()
rasax            |   File "/usr/local/lib/python3.8/site-packages/alembic/script/base.py", line 489, in run_env
rasax            |     util.load_python_file(self.dir, "env.py")
rasax            |   File "/usr/local/lib/python3.8/site-packages/alembic/util/pyfiles.py", line 98, in load_python_file
rasax            |     module = load_module_py(module_id, path)
rasax            |   File "/usr/local/lib/python3.8/site-packages/alembic/util/compat.py", line 184, in load_module_py
rasax            |     spec.loader.exec_module(module)
rasax            |   File "<frozen importlib._bootstrap_external>", line 843, in exec_module
rasax            |   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
rasax            |   File "/usr/local/lib/python3.8/site-packages/rasax/community/database/schema_migrations/alembic/env.py", line 96, in <module>
rasax            |     run_migrations_online()
rasax            |   File "/usr/local/lib/python3.8/site-packages/rasax/community/database/schema_migrations/alembic/env.py", line 86, in run_migrations_online
rasax            |     context.run_migrations()
rasax            |   File "<string>", line 8, in run_migrations
rasax            |   File "/usr/local/lib/python3.8/site-packages/alembic/runtime/environment.py", line 846, in run_migrations
rasax            |     self.get_context().run_migrations(**kw)
rasax            |   File "/usr/local/lib/python3.8/site-packages/alembic/runtime/migration.py", line 511, in run_migrations
rasax            |     for step in self._migrations_fn(heads, self):
rasax            |   File "/usr/local/lib/python3.8/site-packages/alembic/command.py", line 287, in upgrade
rasax            |     return script._upgrade_revs(revision, rev)
rasax            |   File "/usr/local/lib/python3.8/site-packages/alembic/script/base.py", line 365, in _upgrade_revs
rasax            |     return [
rasax            |   File "/usr/local/lib/python3.8/contextlib.py", line 131, in __exit__
rasax            |     self.gen.throw(type, value, traceback)
rasax            |   File "/usr/local/lib/python3.8/site-packages/alembic/script/base.py", line 194, in _catch_revision_errors
rasax            |     compat.raise_from_cause(util.CommandError(resolution))
rasax            |   File "/usr/local/lib/python3.8/site-packages/alembic/util/compat.py", line 308, in raise_from_cause
rasax            |     reraise(type(exception), exception, tb=exc_tb, cause=exc_value)
rasax            |   File "/usr/local/lib/python3.8/site-packages/alembic/util/compat.py", line 301, in reraise
rasax            |     raise value.with_traceback(tb)
rasax            |   File "/usr/local/lib/python3.8/site-packages/alembic/script/base.py", line 162, in _catch_revision_errors
rasax            |     yield
rasax            |   File "/usr/local/lib/python3.8/site-packages/alembic/script/base.py", line 364, in _upgrade_revs
rasax            |     revs = list(revs)
rasax            |   File "/usr/local/lib/python3.8/site-packages/alembic/script/revision.py", line 765, in _iterate_revisions
rasax            |     requested_lowers = self.get_revisions(lower)
rasax            |   File "/usr/local/lib/python3.8/site-packages/alembic/script/revision.py", line 319, in get_revisions
rasax            |     return sum([self.get_revisions(id_elem) for id_elem in id_], ())
rasax            |   File "/usr/local/lib/python3.8/site-packages/alembic/script/revision.py", line 319, in <listcomp>
rasax            |     return sum([self.get_revisions(id_elem) for id_elem in id_], ())
rasax            |   File "/usr/local/lib/python3.8/site-packages/alembic/script/revision.py", line 322, in get_revisions
rasax            |     return tuple(
rasax            |   File "/usr/local/lib/python3.8/site-packages/alembic/script/revision.py", line 323, in <genexpr>
rasax            |     self._revision_for_ident(rev_id, branch_label)
rasax            |   File "/usr/local/lib/python3.8/site-packages/alembic/script/revision.py", line 386, in _revision_for_ident
rasax            |     raise ResolutionError(
rasax            | alembic.util.exc.CommandError: Can't locate revision identified by '652500998f3e'
rasax            | 
rasax            | Sorry, something went wrong (see error above). Make sure to start Rasa X with valid data and valid domain and config files. Please, also check any warnings that popped up.
rasax            | If you need help fixing the issue visit our forum: https://forum.rasa.com/.
^CGracefully stopping... (press Ctrl+C again to force)

@niki delete the .db database file from the project i.e events and rasa and then try? because alembic error is associate with .db only.

Deleted the .db files and it runs fine with your suggested version. :grinning: Unfortunately it still only starts rasa and ignores rasa x completely

Well, still the initial issue. That I am not able to run rasa x --production

@niki rasa open source not required .db the solution I suggested is basically for rasa-x only. Then what is running, I not get you? can you share some error again?

Sorry but what exactly is your question? I described my problem in detail in the first post including my script and logs.

@niki Nothing, as you mention delete .db runs that’s why I asked because rasa-x required .db not rasa open source. I am aware of your first post and script and logs and your problem too.

@niki your issue is related to rasa-x version and compatible with rasa-sdk. Good Luck!

@ChrisRahme pinging for the help, if you can, thanks.

No, sorry, I don’t know a solution :sweat_smile:

@ChrisRahme :frowning: have you seen the first post in detail?

@niki Hey! Please see this thread Problem connecting mysql with rasa x using docker-compose in custom actions - #18 by M_R_LY may be this will help you or discuss with the thread owner.

1 Like