Rasa X failures frequently on OCP Kubernetes

Rasa X seems to be instable on Openshift platform. Some symptoms:

  1. Lot of liveness probe failures; especially pods rasa worker, rasa production (fig 1)
  2. Lot of memory consumption (fig 2)
  3. more than once insights report starts and won’t stop
  4. Rasa X pods restart

Versions

  1. Rasa X : 0,40.1
  2. Rasa: 2.3.4 full
  3. kubernetes: v1.19.0+263ee0d Openshift: 4.6.23

Model-sizes

  1. c.a. max five models in Rasa /models Tab, others deleted from Rasa X / models Tab
  2. Size on linux file server: c.a. 150 MB

We can’t figure out, whats the cause of the problem. We even can’t figure out if it’s Rasa X application (according to some Openshift colleques) or OCP, or whatever.

So, I’m interested in:

  1. Forum members with same issue
  2. Forum members with same configuration/versions
  3. Hints, tips, suggestions, anything that could solve the problem.

fig-1

fig-1

fig-2

fig-2

fig-3 image fig-3

Although I still haven’t found a solution, I noticed:

  1. this failure when no model is selected and running insights report
  2. No failure when running the insights and a model is available

ps: Althoug I understand that the model is neede foor creating the insigths, I don’t understand why the application hangs on it.

Nevertheless, we found a work-around: uninstall/install namespace, with deleting persitant claims

1 Like