Rasa deployment - quick install/helm - Upload model stopped working

Rasa Open Source
1

Hi,

I m using quick install/helm for rasa server’s deployment, and everything was working properly including model upload on server rasa x, until i upgraded to 2.8.2 (bug, not starting) and downgraded back to 2.7.0. After the deployment, since i m using local GIT and not github, i can’t train the model on server rasa x and i need to upload the model manually using Add model → Upload files…before i could upload the models without any problems, but now it’s failing…i tried to downgrade to 2.6.0, restart docker, delete namespace, images…without success.

The upload via API is not working.

I suspect the upgrade to 2.8.0 has messed everything up. Please advise if there is any alternative method to upload the model to docker pod, or how to solve this problem.

2

all pods are running, and no error logs:

Blockquote

[root@localhost fyp-chatbot]# kubectl get pods --namespace rasa NAME READY STATUS RESTARTS AGE rasa-redis-master-0 1/1 Running 0 24m rasa-nginx-7f4677647c-d8cmg 1/1 Running 0 24m rasa-postgresql-0 1/1 Running 0 24m rasa-rasa-x-8d69ff665-26svn 1/1 Running 0 24m rasa-rabbit-0 1/1 Running 0 24m rasa-app-6d79f897f-jjf2g 1/1 Running 0 24m rasa-db-migration-service-0 1/1 Running 1 24m rasa-rasa-production-76c6f46c69-hxzt8 1/1 Running 0 24m rasa-rasa-worker-666697f86b-gcmgz 1/1 Running 0 24m rasa-event-service-968bf56fb-496f5 1/1 Running 0 24m

Blockquote

3

Hi, can you provide more info?

  1. How did you upgrade it to 2.8.2? Commands that you execute would be useful
  2. Do you see any errors in the logs for the rasa-production pod or rasa-x pod?
  3. Do you use debug mode?
4

@tczekajlo, hi how are you?

1)- over time, I upgraded Rasa from 2.6 to 2.7 then 2.8 (then reverted back to 2.6 since i got error on 2.8 and rasa x is not working…):

  • pip3 install rasa rasa-x --upgrade --extra-index-url Simple Index --use- deprecated=legacy-resolver
  • pip3 install --upgrade rasa[spacy]
  • pip3 install --upgrade Spacy

2)- No errors under rasa-production. Kindly find below the logs of rasa-x pod:

Blockquote

[2021-08-16 13:09:02 +0000] - (sanic.access)[INFO][10.42.0.162:48272]: GET http://rasa-rasa-x-rasa-x.rasa.svc:5002/api/projects/default/models/tags/production?token=rasaXToken 404 160 [2021-08-16 13:09:03 +0000] - (sanic.access)[INFO][10.42.0.1:51776]: GET http://10.42.0.173:5002/ 200 3283 [2021-08-16 13:09:03 +0000] - (sanic.access)[INFO][10.42.0.1:51778]: GET http://10.42.0.173:5002/ 200 3283 [2021-08-16 13:09:12 +0000] - (sanic.access)[INFO][10.42.0.163:51530]: GET http://rasa-rasa-x-rasa-x.rasa.svc:5002/api/projects/default/models/tags/production?token=rasaXToken 404 160 [2021-08-16 13:09:12 +0000] - (sanic.access)[INFO][10.42.0.162:48352]: GET http://rasa-rasa-x-rasa-x.rasa.svc:5002/api/projects/default/models/tags/production?token=rasaXToken 404 160 [2021-08-16 13:09:13 +0000] - (sanic.access)[INFO][10.42.0.1:51856]: GET http://10.42.0.173:5002/ 200 3283 [2021-08-16 13:09:13 +0000] - (sanic.access)[INFO][10.42.0.1:51858]: GET http://10.42.0.173:5002/ 200 3283 [2021-08-16 13:09:22 +0000] - (sanic.access)[INFO][10.42.0.163:51618]: GET http://rasa-rasa-x-rasa-x.rasa.svc:5002/api/projects/default/models/tags/production?token=rasaXToken 404 160 [2021-08-16 13:09:22 +0000] - (sanic.access)[INFO][10.42.0.162:48440]: GET http://rasa-rasa-x-rasa-x.rasa.svc:5002/api/projects/default/models/tags/production?token=rasaXToken 404 160 [2021-08-16 13:09:23 +0000] - (sanic.access)[INFO][10.42.0.1:51944]: GET http://10.42.0.173:5002/ 200 3283 [2021-08-16 13:09:23 +0000] - (sanic.access)[INFO][10.42.0.1:51946]: GET http://10.42.0.173:5002/ 200 3283 [2021-08-16 13:09:32 +0000] - (sanic.access)[INFO][10.42.0.163:51700]: GET http://rasa-rasa-x-rasa-x.rasa.svc:5002/api/projects/default/models/tags/production?token=rasaXToken 404 160 [2021-08-16 13:09:32 +0000] - (sanic.access)[INFO][10.42.0.162:48522]: GET http://rasa-rasa-x-rasa-x.rasa.svc:5002/api/projects/default/models/tags/production?token=rasaXToken 404 160 [2021-08-16 13:09:33 +0000] - (sanic.access)[INFO][10.42.0.1:52026]: GET http://10.42.0.173:5002/ 200 3283 [2021-08-16 13:09:33 +0000] - (sanic.access)[INFO][10.42.0.1:52028]: GET http://10.42.0.173:5002/ 200 3283

Blockquote

3)- Yes, i am using debug mode.

Under local mode, i even uninstalled rasa and reinstalled it without success…

5

i was able to upload the models all time, then suddenly stopped working, i don’t know what’s wrong as there is no clue about the source of this problem.

6

local rasa:

  • i reinstalled rasa.
  • i retrained the model.

rasa server:

  • i deleted the name space and images, and deployed again.
  • i rebooted the host machine.
  • i tried multiple browsers…
7

Today, as alternative solution, i trained/activated the model using github, but when i type anything the bot doesn’t answer (keeps typing always, all pods are running, no errors…). I m sure there is something occurred in Rasa server, but i don’t know what it is or how to fix it. Note that everything was working properly then this problem started.

8

it seems quick install was installing the latest versions of Rasa (2.8.0 / 0.42) which have bug (not working also on local mode). I solved the problem by forcing the usage of 2.7.0 and 0.41.2 (RASA_X_VERSION and RASA_VERSION variables).