RasaX and Github not in sync. Responses disappear from RasaX when changes added

  • Rasa X 0.27.3.
  • Rasa 1.9.2.
  • Linux (ubuntu 18.04).

I added few new responses under existing ones that have buttons so the buttons would be quick replies when using facebook.

  1. I add the responses. Train. Sync to Github.
  2. Now everything seems ok. (buttons are quick replies when using facebook)
  3. Suddenly new responses have disappeared from Rasa X responses

Rasa X still says it’s up to date. Rasa X doesn’t show the new responses but i can see those in Github.

If i now train new model and test it, facebook does not give any buttons for the responses. Some of the responses don’t show up on Facebook at all, but I can see the right response in rasa x conversations.

New responses that i added were in the following format:

text: "text here"
channel: facebook
quick_replies:
  - content_type: text
    title: text here

Update: I tried to add the responses again. Then i added some training data to make rasa x understand that there has been changes. Responses disappear from rasa x immediately after i add changes to git. The pr rasa x sends says nothing about the responses.

All responses are still in the Github domain file like they should be(there are two responses for those i added earlier but rasa x only has one response for everything).

Other channels still work fine. Facebook gives utter_greet text but no buttons. If i make it to say something else with buttons it doesn’t say anything but i can see the right intent and answer in rasax conversations. After that it gives utter_default no matter what i say. This proves that the responses i created are there but i just can not see them in rasax(and they don’t work).

If you’re not adding your changes directly to the master branch (or whichever target branch you chose), then changes will dissappear from rasa x as soon as you send them to a git branch; since Rasa X is synced to master, it’s “resyncing” itself to master (which doesn’t yet contain the changes). Now if you merge those changes from the Rasa X branch to master, you should see them again.

That said, I have also experienced the “disappearing responses” behavior for channel-specific responses. Do you have any channel-specific responses for your other channels, and do those show up in Rasa X?

Hi @mloubser and thanks for your response.

I always add new branch and then merge to master. Here is what i see for example for utter_greet on master in domain file.

utter_greet:
  - text: Hi !
    buttons:
    - content_type: text
      title: text here
      payload: /intenthere
  - text: Hi !
    channel: facebook
    quick_replies:
    - content_type: text
      title: texthere
      payload: /intenthere

The channel-specific response for facebook can not be seen on Rasa X. “disappearing responses” happens only for channel-specific responses.

Update: I tried to add utter_geet for socketio. pr added that but now deleted all facebook responses. The socketio response do not show up in Rasa X.

Do you know if this 0.27.6 makes any difference?

#2544: Fixed a problem when Integrated Version Control indicated a clean status although there were changed files. Rasa Enterprise Changelog

Update 2: 0.27.6 didn’t make any difference.

This seems to be a bug, so I’m creating an issue for it internally. Thanks for bringing it to our attention!

I experience the similar issue where changes in the Github master - such as my Domain file are updated but do not pull through to Rasa X. There is no indication that there is a problem with syncing as X indicates that it is in-sync. Does the model have to be trained in my IDE with the model file updated to Master as well for the sync to happen? I guess to troubleshoot I should check the Rasa X logs in my GCS instance - what are the kubectl instructions for this? It might be worth mentioning that I just ran the Rasa X upgrade and when I check the log for this I get

INFO:alembic.runtime.migration:Context impl PostgresqlImpl.
INFO:alembic.runtime.migration:Will assume transactional DDL.
INFO:alembic.runtime.migration:Running upgrade 8893bba2a522 -> b67a67032c7f, This migration comes from RBAC backport for `0.24.x`.
It is empty because all the permissions that are required for
role-based access control fixes were already added before.
INFO:alembic.runtime.migration:Running upgrade ac3fba1c2b86, b67a67032c7f -> 9e7a97234e85, Merge the migrations from the 0.24.x pa
tch release with
Rasa X databases which didn't run this backported migration.
INFO:alembic.runtime.migration:Running upgrade 9e7a97234e85 -> 8a8562256a8e, Add `hash` column and an index for it to `response` t
able.
/usr/local/lib/python3.7/site-packages/sqlalchemy/dialects/postgresql/base.py:3367: SAWarning: Predicate of partial index conversa
tion_event_flag_index ignored during reflection
  % idx_name
Starting Rasa X server... 🚀`
1 Like

@mloubser this continues the issue I raised in the certification training. I would be really grateful for your assistance here please, as this is currently an impediment for our dev team.

You can check the logs for the pod to see what’s going on there - could be an incorrectly formatted domain file, I’ve seen that before.

kubectl --namespace rasa get pods

to get the name of the rasa x pod, then check the logs with

kubectl --namespace rasa get logs <pod name>

No, you don’t have to train a model for changes to be reflected in git, but of course you will need to train a model with the changes to see those changes in your live assistant

Running kubectl logs on RasaX, I get the following errors (note that I recently ran the one-line deploy to upgrade RasaX as advised in the application):

INFO:alembic.runtime.migration:Context impl PostgresqlImpl.
INFO:alembic.runtime.migration:Will assume transactional DDL.
INFO:alembic.runtime.migration:Running upgrade 8893bba2a522 -> b67a67032c7f, This migration comes from RBAC backport for `0.24.x`.
It is empty because all the permissions that are required for
role-based access control fixes were already added before.
INFO:alembic.runtime.migration:Running upgrade ac3fba1c2b86, b67a67032c7f -> 9e7a97234e85, Merge the migrations from the 0.24.x patch release with
Rasa X databases which didn't run this backported migration.
INFO:alembic.runtime.migration:Running upgrade 9e7a97234e85 -> 8a8562256a8e, Add `hash` column and an index for it to `response` table.
/usr/local/lib/python3.7/site-packages/sqlalchemy/dialects/postgresql/base.py:3367: SAWarning: Predicate of partial index conversation_event_flag_index ignored during reflection
  % idx_name
Starting Rasa X server... 🚀
Execution of job "GitService.run_background_synchronization (trigger: cron[minute='*'], next run at: 2020-05-27 19:09:00 UTC)" skipped: maximum number of running instances reached (1)
ERROR:rasax.community.api.blueprints.models:
Run time of job "GitService.run_background_synchronization (trigger: cron[minute='*'], next run at: 2020-05-27 19:11:00 UTC)" was missed by 0:00:01.086085
INFO:sanic_cors.core:The request's Access-Control-Request-Method header does not match allowed methods. CORS headers will not be applied.

When I try run kubectl --namespace rasa get logs <pod name> I get back the response error: the server doesn't have a resource type "logs" I did try various alternative combinations but none worked for me, except kubectl logs which returned the errors pasted in my previous reply.

To try resolve this, I have manually uploaded NLU and Stories data, as well as pasted and saved the Domain settings directly into X (after confirming no issues with Yamllint and the file saved with no problems in X).

However, when I try train the model, I still get Nothing has changed -It seems no changes were made to your model, so we skipped training a new one!

Not sure if the latest update supposed to fix this but I have not encountered the problem anymore.

In this format/order, responses are not getting deleted by Rasa X.

channel: facebook
quick_replies:
  - content_type: text
    payload: /intenthere
    title: Title here
text: Text here

I’ll mark this as resolved as the original issue was about responses getting deleted.

1 Like