I followed the tutorial from Rasa Masterclass 12 to learn how to deploy the latest version of Rasa X in Google Cloud Kubernetes and connect this deployment via integrated version control to my bot.
After following the tutorial I successfully connected the bot via Git and uploaded a model using the API. After this a few of the pods started failing, with the error logs seen below.
To test my issue I even used just the basic bot from the rasa init command and pushed it to a git repository to make sure my bot code was not the issue and had the same issue.
Here is the config used:
This is what my pods look like after a short while:
Looking into the logs of for example the rasa-worker shows the following error: 2020-04-14 08:38:59 DEBUG rasa.core.agent - Found new model with fingerprint 3a9028fc4c7790d64f5b36f8bb132509. Loading… 2020-04-14 08:38:59 ERROR rasa.core.agent - Could not load model due to Not all required importable packages are installed. To use this pipeline, you need to install the missing dependencies. Please install the package(s) that contain the module(s): tensorflow_text.
This seems to be the same issue that I had when using Rasa x on my windows machine locally, where tensorflow_text is not able to be installed. How can this happen when deploying on the Google Cloud Kubernetes though? I simply used the basic settings for the clusters, which is called “Container Optimized OS”. I even tried changing this to ubuntu, but I had the same issue.