I have just setup rasa x in gcp vm (computer engine) via docker (not the manual install). The model spends some time trying to train and then fails.
Here are a few excerpts from the log. @erohmensing , do you have any thoughts?
Including @mloubser
Can you please provide the version you tried to install? And any custom changes you made in e.g. docker-compose.override.yaml?
No custom changes. Initially, I tried 0.38.x and then upgraded to 0.39.x. I’m wondering if the VM has to be configured in a particular way to use NGINX?
You should be good on the VM side if it has enough CPU and memory for your workload; how long does training carry on before you see this error?
Typically 1 hour.
Does the worker pod go down? Do you see any resource complaints? If you’re training on rasa-demo, it can take quite a time and quite a lot of memory to train successfully, so I suspect a resource constraint issue
Easiest way to check: switch to a different, small repo, and see if it can train succesfully using that dataset
It is unable to train on a small repo.
Does it fail faster then, or also only after. 1 hour?
Seems to still train for a comparable amount of time.