Right now, I can successfully deploy the rasa server and rasa custom action server to Kubernetes using Helm Charts. However, the deployment in Kubernetes only serves one user at a time.
What I want is to serve multiple users simultaneously so I set the ‘replicaCount’ in both values.yaml in both rasa server and rasa action server helm charts to test whether it can handle multiple users.
Nevertheless, it seems only can serve only one user at a time. First, when I use 2 different devices to go to the website, the chatbot widgets appear in both devices. Then, I started to chat with the bot, device A will respond but the widget in device B suddenly disappear while I am chatting.
Given this situation, I am wondering if the rasa action server does not link up with the rasa server correctly although I have already 3 replicas. What should I do to achieve the goal of scaling my bot.
@edward0422 ,
even one rasa instance can handle multiple users at the same time.
Few things to look at/
Tracker store- is your tracker store externalised to Redis, Postgres?
Session ID, when using the rest channel, there is a user object which should be a unique ID for each user/session depending how you would configure. In your widget, check if this user object is set to something unique to identify each conversation session
Widget behaviour is outside of the scope of the rasa setup, it might be some kind of bug in the widget itself.
what URL in the Rasa server are you to pointing for the rasa action server, is the one for the pod or the service. make sure you point towards the service url
Tracker store: I seems do not have any tracker store in the deployment but using lockstore instead. From the Helm Chart of rasa, 'applicationSettings.endpoints.lockStore"
lockStore:
# -- Enable endpoint for Lock Store
enabled: true
# -- Lock Store type
type: "redis"
# -- The url of your redis instance
url: ${REDIS_HOST}
# -- The port which redis is running on
port: ${REDIS_PORT}
# -- Password used for authentication
password: ${REDIS_PASSWORD}
# -- The database in redis which Rasa uses to store the conversation locks
db: "1"
As the command prompt recommends me to enable the lockstore when the ‘replicaCount’ is set to be larger than 1.
Yes, I configure the bot using rest channel. But, how can I check the sessionID when deploying to Kubernetes?
I am using the Botfront rasa chabot and do not know whether it has a bug or not. I think majority are using this widget
I config the ‘rasa-values.yaml’ as follows"
# rasa-values.yaml
rasa-action-server:
external:
# -- Determine if external URL is used
enabled: true
# -- URL to Rasa Action Server
url: "http://rasa-action-release-rasa-action-server/webhook"
Regarding Lock Store
Yes indeed this is to ensure there is no concurrent lock when you have multiple replicas.
However Tracker Store is not related to concurrency rather it preserves the state. If you don’t externalise the tracker, it would use InMemoryTrackerStore where the state is created in each of the pods. So in your case you must also externalise the tracker store. Since you are already running redis, you can use it as well to create a tracker store.
I am not sure about Botfront but maybe there is a kind of lock.
Regarding sessionID, since you are using botfront, i suppose it is creating them already.
you can --enable-api when running rasa server so you can access the tracker for more information
Could you also replace the action-server port from 80 to 5005. Usually 80 is a reserved port so it might be that your rasa server is not able to reach action server.
@souvikg10 In other words, I should both enable the tracker store and lock store? However, could they both install in an individual redis? Is it an redis container is created, and there are 2 pods, one is tracker store, another is lock store?
The rasa server is running with the command --enable-api --cors '*' in the pods.
I should have use the port 5055 for the rasa action server, here is the values.yaml for action server:
# Default values for rasa-action-server.
# This is a YAML-formatted file.
# -- (string) Override name of app
nameOverride: ""
# -- (string) Override the full qualified app name
fullnameOverride: ""
# -- Registry to use for all Rasa images (default docker.io)
## DockerHub - use docker.io/rasa
registry: rasaaci.azurecr.io
applicationSettings:
# -- Port on which Rasa Action Server runs
port: 5055
# -- Scheme by which the service are accessible
scheme: http
# -- Specify the number of Action Server replicas
replicaCount: 2
# -- Override the default arguments for the container
args: []
# -- Override the default command for the container
command: []
# -- Add extra environment variables
extraEnv: []
# - name: SOME_CUSTOM_ENV_VAR
# value: "custom value"
## Define the rasa image to work with
image:
# -- Action Server image name to use (relative to `registry`)
name: docker-rasa-action
# -- Action Server image tag to use
tag: "v5"
# -- Override default registry + image.name for Action Server
repository: ""
# -- Action Server image pullPolicy
pullPolicy: IfNotPresent
# -- Action Server repository pullSecret
## See https://kubernetes.io/docs/concepts/containers/images/#specifying-imagepullsecrets-on-a-pod
pullSecrets: []
# - name: "<SECRET>"
serviceAccount:
# -- Specifies whether a service account should be created
create: false
# -- Annotations to add to the service account
annotations: {}
# -- The name of the service account to use.
# If not set and create is true, a name is generated using the fullname template
name: ""
# -- Annotations to add to the action-server's pod(s)
podAnnotations: {}
# key: "value"
# -- Labels to add to the action-server's pod(s)
podLabels: {}
# key: "value"
# -- Annotations to add to the action-server deployment
deploymentAnnotations: {}
# key: "value"
# -- Labels to add to the action-server deployment
deploymentLabels: {}
# -- Defines pod-level security attributes and common container settings
## See: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/
podSecurityContext: {}
# fsGroup: 2000
# -- Allows you to overwrite the pod-level security context
securityContext: {}
# capabilities:
# drop:
# - ALL
# readOnlyRootFilesystem: true
# runAsNonRoot: true
# runAsUser: 1000
## Configuration for the service for the action-server
service:
# -- Set type of action-server service
type: ClusterIP
# -- Set port of action-server service (Kubernetes >= 1.15)
port: 80
# -- Annotations to add to the service
annotations: {}
# -- Specify the nodePort(s) value(s) for the LoadBalancer and NodePort service types
## Ref: https://kubernetes.io/docs/concepts/services-networking/service/#nodeport
nodePort:
# -- Exposes the Service externally using a cloud provider's load balancer
## Ref: https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer
loadBalancerIP:
# -- Enable client source IP preservation
## Ref: http://kubernetes.io/docs/tasks/access-application-cluster/create-external-load-balancer/#preserving-the-client-source-ip
externalTrafficPolicy: Cluster
## Configure the ingress resource that allows you to access the
## deployment installation. Set up the URL
## ref: http://kubernetes.io/docs/user-guide/ingress/
ingress:
# -- Set to true to enable ingress
enabled: false
# -- Ingress annotations
annotations: {}
# kubernetes.io/ingress.class: nginx
# kubernetes.io/tls-acme: "true"
# -- Labels to add to the ingress
labels: {}
# -- Ingress Path type
## Ref: https://kubernetes.io/docs/concepts/services-networking/ingress/#path-types
pathType: ImplementationSpecific
# -- Ingress path
path: /
# -- Hostname used for the ingress
hostname: chart-example.local
# -- TLS configuration for ingress
## See: https://kubernetes.io/docs/concepts/services-networking/ingress/#tls
tls: []
# - secretName: chart-example-tls
# hosts:
# - chart-example.local
# -- Any additional arbitrary paths that may need to be added to the ingress under the main host
extraPaths: {}
# - path: /*
# backend:
# serviceName: ssl-redirect
# servicePort: https
# -- Resource requests and limits
resources: {}
# We usually recommend not to specify default resources and to leave this as a conscious
# choice for the user. This also increases chances charts run on environments with little
# resources, such as Minikube. If you do want to specify resources, uncomment the following
# lines, adjust them as necessary, and remove the curly braces after 'resources:'.
# limits:
# cpu: 100m
# memory: 128Mi
# requests:
# cpu: 100m
# memory: 128Mi
## Autoscaling parameters for the Action Server Deployment
## See: https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/
autoscaling:
# -- Enable autoscaling
enabled: false
# -- Lower limit for the number of pods that can be set by the autoscaler
minReplicas: 1
# -- Upper limit for the number of pods that can be set by the autoscaler.
# It cannot be smaller than minReplicas.
maxReplicas: 20
# -- Fraction of the requested CPU that should be utilized/used,
# e.g. 70 means that 70% of the requested CPU should be in use.
targetCPUUtilizationPercentage: 80
# targetMemoryUtilizationPercentage: 80
# -- Allow the Action Server Deployment to be scheduled on selected nodes
## Ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#nodeselector
## Ref: https://kubernetes.io/docs/user-guide/node-selection/
nodeSelector: {}
# -- Tolerations for pod assignment
## Ref: https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/
tolerations: []
# -- Allow the Action Server Deployment to schedule using affinity rules
## Ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity
affinity: {}
# -- Allow the deployment to perform a rolling update
## ref: https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#strategy
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 1
maxUnavailable: 0
# -- Override default liveness probe settings
# @default -- Every 15s / 6 KO / 1 OK
## Ref: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/#container-probes
livenessProbe:
httpGet:
path: /health
port: http
scheme: HTTP
initialDelaySeconds: 15
periodSeconds: 15
successThreshold: 1
timeoutSeconds: 5
failureThreshold: 6
# -- Override default readiness probe settings
# @default -- Every 15s / 6 KO / 1 OK
## Ref: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/#container-probes
readinessProbe:
httpGet:
path: /health
port: http
scheme: HTTP
initialDelaySeconds: 15
periodSeconds: 15
successThreshold: 1
timeoutSeconds: 5
failureThreshold: 6
# -- Allow to specify init containers for the Action Server Deployment
## Ref: https://kubernetes.io/docs/concepts/workloads/pods/init-containers/
initContainers: []
# - name: init
# image: "busybox"
# command: ["bash", "-c"]
# args:
# - echo "init container"
# -- Specify additional volumes to mount in the action-server container
## Ref: https://kubernetes.io/docs/concepts/storage/volumes/
volumes: []
# - hostPath:
# path: <HOST_PATH>
# name: <VOLUME_NAME>
# -- Specify additional volumes to mount in the action-server container
volumeMounts: []
# - name: <VOLUME_NAME>
# mountPath: <CONTAINER_PATH>
# readOnly: true
Should I also change the port in service.port from 80 to 5055?
@edward0422 - Redis is a like a database, and with it’s own distributed system, you don’t need to spin up pods based on what tables you make.
for redis, i would create pods based on how much memory it needs. Tracker Store can be a large table so two replicas of redis should be enough to sustain both lock store(different table) and tracker store. you don’t need to assign these tables to individual pods, redis takes care of it.
and regarding the port indeed it is better to use ports like 5055 for action server
However, in the values.yaml for building helm chart of rasa server, it claims that the Tracker Store is recommended to build in Postgresql, see the comments below:
## You can use a Tracker Store to store your assistant's conversation history.
## See: https://rasa.com/docs/rasa/tracker-stores
##
## All environment variables used as values are added to the rasa-oss container automatically if `postgresql.install=true`.
trackerStore:
# -- Enable endpoint for Tracker Store
enabled: true
# -- Tracker Store type
type: sql
# -- The dialect used to communicate with your SQL backend
dialect: "postgresql"
# -- URL of your SQL server
url: ${DB_HOST}
# -- Port of your SStoreQL server
port: ${DB_PORT}
# -- The username which is used for authentication
username: ${DB_USER}
# -- The password which is used for authentication
password: ${DB_PASSWORD}
# -- The path to the database to be used
db: ${DB_DATABASE}
# -- Create the database for the tracker store.
# If `false` the tracker store database must have been created previously.
login_db: ${DB_DATABASE}
global:
postgresql:
# -- postgresqlUsername which should be used by Rasa to connect to Postgres
postgresqlUsername: "postgres"
# -- postgresqlPassword is the password which is used when the postgresqlUsername equals "postgres"
postgresqlPassword: "password"
# -- existingSecret which should be used for the password instead of putting it in the values file
existingSecret: ""
# -- postgresDatabase which should be used by Rasa
postgresqlDatabase: "rasa"
# -- servicePort which is used to expose postgres to the other components
servicePort: 5432
## PostgreSQL specific settings (https://hub.helm.sh/charts/bitnami/postgresql/10.3.18)
postgresql:
# -- Install PostgreSQL
install: true
## Use external PostgreSQL installation
## This section is not a part of the PostgreSQL subchart
external:
# -- Determine if use an external PostgreSQL host
enabled: false
# -- External PostgreSQL hostname
## The host value is accessible via the `${DB_HOST}` environment variable
host: "external-postgresql"
Therefore. I install postgresql and enable it for tracker store. Would it be the same?
Regarding to the port, the rasa action server cannot work if I change the service.port from 80 to 5055 . I guess the 80 port is for the service and allow the rasa server to communicate with it. I only keep applicationSettings.port to be 5055 which let the rasa action server runs on 5055
@edward0422 you can other ports for your service as well, since it is running on the same cluster as the rasa server, you can assign another port for the service like 5055, only reason i am saying this is because 80 for some cluster might be reserved. and yeah postgres/redis both are supported for tracker store. it’s the same.
