Rasa X 0.32.2 deleting model from DB because it couldn't find it on disk

[Deprecated] Rasa X Community Edition
1

In my GitLab CI pipeline I’m using the Rasa X API to upload the model and tag it as “production”. But then, I see that Rasa X doesn’t have any model tagged; and the one uploaded does not appear, even though curl successfully uploaded and tagged the model.

# Upload model to Rasa X
$ curl --write-out "\n" \
    --silent \
    --retry 3 \
    --retry-delay 5 \
    --max-time 10 \
    --retry-max-time 120 \
    --show-error \
    --fail \
    --form "model=@$MODEL_PATH" \
    "$API_URL/projects/default/models?api_token=$RASA_X_SECRET_TOKEN"

# Response: 201 Created
# {
#    "hash":"6f12301a27de66d989576db4f30b0642",
#    "model":"20201103-132614",
#    "path":"\/app\/models\/20201103-132614.tar.gz",
#    "project":"default",
#    "trained_at":1604409882.6870493889,
#    "version":"1.10.14",
#    "tags":[]
# }

# Tag model as "production"
$ curl --write-out "\n" \
    --silent \
    --retry 3 \
    --max-time 10 \
    --show-error \
    --fail \
    --request PUT \
    "$API_URL/projects/default/models/20201103-132614/tags/production?api_token=$RASA_X_SECRET_TOKEN"

# Response: 204 No Content

Checking the logs I found this:

rasa-x_1 | INFO:rasax.community.services.model_service:Deleting model '20201103-132614' from database since it could not be found on disk.

But then I check the disk and the model is there.

$ ls -l models/ | tail -1
-rw-r--r-- 1 1001 root 57890712 Nov  3 13:32 20201103-132614.tar.gz

Though it doesn’t appear in Rasa X.

Trying to tag it again as production returns 404:

$ curl --write-out "\n" \
    --silent \
    --retry 3 \
    --max-time 10 \
    --show-error \
    --fail \
    --request PUT \
    "$API_URL/projects/default/models/20201103-132614/tags/production?api_token=$RASA_X_SECRET_TOKEN" 
curl: (22) The requested URL returned error: 404 Not Found

On top of that, this doesn’t always happen. But I just don’t know what causes it. How do I prevent Rasa X from deleting a model that actually exists?

2

@zfunction ,

How did you deploy Rasa X?

Using docker-compose, quick-install script, or kubernetes?

3

It was deployed manually with docker-compose.

4

Hi @Arjaan, do you have any insight on why is this happening?

5

I have found that waiting until Rasa services are up (rasa-x, rasa-worker, rasa-production, etc.) before uploading and tagging the model helps to avoid this issue.