Multiple Issue with RASA X helm chart deployment

Hi, We have been trying to deploy our bot using helm chart and also following the steps mentioned in Rasa Advanced Deployment Workshop | Udemy.

Steps taken are as follows:

sudo apt update
sudo apt install docker.io docker-compose
sudo snap install microk8s --classic
sudo usermod -a -G microk8s $USER
sudo chown -f -R $USER ~/.kube
sudo microk8s enable dns storage helm3 registry dashboard ingress
cd $HOME/.kube
sudo microk8s config > config

alias kubectl='microk8s.kubectl'
alias helm='microk8s.helm3'
alias k="kubectl --namespace ask-edna"
alias h="helm --namespace ask-edna"
source ~/.bashrc

kubectl create namespace ask-edna

helm repo add rasa-x https://rasahq.github.io/rasa-x-helm
h install --values values.yml ask-edna rasa-x/rasa-x

The values.yml file is as below:

debugMode: true
rasax:
    # initialUser is the user which is created upon the initial start of Rasa X
    initialUser:
        # password for the Rasa X user
        password: "Password1"
    # token Rasa X accepts as authentication token from other Rasa services
    token: "Password1"
    # jwtSecret which is used to sign the jwtTokens of the users
    jwtSecret: "Password1"
# rasa: Settings common for all Rasa containers
rasa:
    # token Rasa accepts as authentication token from other Rasa services
    token: "Password1"
    name: "xxxxxxxx/rasaxxxxxxxxx"
    # tag refers to the Rasa image tag. If empty `.Values.rasa.version-full` is used.
    tag: "1"
# RabbitMQ specific settings
rabbitmq:
    # rabbitmq settings of the subchart
    rabbitmq:
        # password which is used for the authentication
        password: "Password1"
# global settings of the used subcharts
global:
    # postgresql: global settings of the postgresql subchart
    postgresql:
        # postgresqlPassword is the password which is used when the postgresqlUsername equals "postgres"
        postgresqlPassword: "Password1"
    # redis: global settings of the postgresql subchart
    redis:
        # password to use in case there no external secret was provided
        password: "Password1"
app:
    name: "xxxxxxxx/customactionserver"
    tag: "1"
duckling:
  enabled: true
  # name of the Duckling image to use
  name: "rasa/duckling"
  # port on which duckling should run
  port: 8000
  # scheme by which duckling is accessible
  scheme: http
nginx:
  service:
    type: LoadBalancer
    port: 80
    externalIPs: [100.xx.xx.xx]

If you notice we are using 2 custom images hosted in dockerhub:

  1. for action server

  2. rasa, as we are using custom components: Now for this we have used the below dockerfile:

    FROM rasa/rasa:2.1.3-full

    USER root

    RUN pip install --trusted-host pypi.python.org fuzzywuzzy==0.18.0 rapidfuzz==0.13.4

We are able to access Rasax, upload and activate the model. But:

  1. Github integration is failing with “Something went wrong”: deploy key added with write access in github
  2. We are not getting a simple “hi” as a response back from the model
  3. kubectl get pods --namespace=ask-edna is showing but on trying to access the logs we are getting: k logs ask-edna-rasa-x-78d9799d4c-nh7tr

Found one possible cause: the custom rasa image file is not being created correctly… updated the docker file but still facing the same issue:

FROM rasa/rasa:2.1.3-full

USER root

COPY ./customComp /customComp/
ENV PYTHONPATH=$PYTHONPATH:/customComp/

RUN pip install --trusted-host pypi.python.org fuzzywuzzy==0.18.0 rapidfuzz==0.13.4

# Switch back to non-root to run code
USER 1001

project structure is as follows:

image

was following this Rasa X Docker ModuleNotFoundError with a Custom Component - Rasa X - Rasa Community Forum @mloubser any idea on how resolve this?

what’s going wrong during the creation of the image? is there an error message? Or is it not getting pulled during container creation?

@mloubser Thanks for taking some time out and replying.

To answer your question Nothing is wrong, the image is getting created correctly and pulled correctly at least all all the pods are up and running… my questions are:

  1. Is the way in which I am creating the image and copying the customcomp correct?
  2. What should i mention in the values.yml for the helm chart deployment in order to use this?

Because the problems that i have mentioned in the initial question still remains…

  1. No response from uploaded model – which i think probably due to the custom comp issue
  2. Cannot link github
  3. Can’t even check the logs – not sure why this is happening

Screenshots to these are given in the original question.

  1. It looks alright, but if you test your custom image locally, does it work? What does your config.yml look like?

  2. you will need to pass the new image name and tag like this: values.yml

rasa:
  name: <your custom image repo name>
  tag: <tag of your custom image to pull>

when you say you can’t check logs - what command are you running to check them? and what are you getting back?

  1. If I run the rasa image locally this is what i get:

config.yml in the dev looks like this, does this need to be specified while deploying using helm chart? If so then where do i specify it?:

  1. This is something which I am already doing… can you please check the initial question of this post? It has the values.yml that i am using… I was worried if the custom component needs to specified in the values.yml

  2. logs - screenshot is attached in the initial question of this thread, PFB the same

Can you use the docker image to train a model locally? To run a model locally? Any issues with the custom component won’t show up unless you try one of those.

No, the custom component doesn’t need to be specified in values.yml

(3) sounds like an issue connecting to your cluster. Check out Troubleshoot kubectl connection refused | by David O'Dell | Medium - this would not be caused by your custom component or anything like that

Can you use the docker image to train a model locally? To run a model locally? Any issues with the custom component won’t show up unless you try one of those.

How do I do that?

see Building a Rasa Assistant in Docker

@dearc as i mentioned to you when we discussed about, your docker image is still incorrect.

you need to add an entrypoint because your docker image is not running as per the logs. Also using Docker UI is not going to provide you useful logs, use the docker command line instead to run your images. Using UI, requires you to provide an entrypoint to your image which is missing in the above Dockerfile you provided.

Rebuild the below image and use it to run. also how exactly are you running the docker image? please try

docker run yourimagename rasa run

simply using the docker UI to run images will require you to add an entrypoint.

FROM rasa/rasa:2.1.3-full

USER root

COPY ./customComp /customComp/
ENV PYTHONPATH=$PYTHONPATH:/customComp/

RUN pip install --trusted-host pypi.python.org fuzzywuzzy==0.18.0 rapidfuzz==0.13.4

# Switch back to non-root to run code
USER 1001

ENTRYPOINT rasa run

please update it accordingly.