Replicas in Kubernetes Deployment

Right now, I can successfully deploy the rasa server and rasa custom action server to Kubernetes using Helm Charts. However, the deployment in Kubernetes only serves one user at a time.

What I want is to serve multiple users simultaneously so I set the ‘replicaCount’ in both values.yaml in both rasa server and rasa action server helm charts to test whether it can handle multiple users.

Nevertheless, it seems only can serve only one user at a time. First, when I use 2 different devices to go to the website, the chatbot widgets appear in both devices. Then, I started to chat with the bot, device A will respond but the widget in device B suddenly disappear while I am chatting.

Given this situation, I am wondering if the rasa action server does not link up with the rasa server correctly although I have already 3 replicas. What should I do to achieve the goal of scaling my bot.


@edward0422 , even one rasa instance can handle multiple users at the same time. Few things to look at/

  • Tracker store- is your tracker store externalised to Redis, Postgres?
  • Session ID, when using the rest channel, there is a user object which should be a unique ID for each user/session depending how you would configure. In your widget, check if this user object is set to something unique to identify each conversation session
  • Widget behaviour is outside of the scope of the rasa setup, it might be some kind of bug in the widget itself.
  • what URL in the Rasa server are you to pointing for the rasa action server, is the one for the pod or the service. make sure you point towards the service url

@souvikg10 Hello, thanks for answering me. Because I am still new to rasa so that I may not quite understand.

Currently, I am using the Botfront rasa chabot

  1. Tracker store: I seems do not have any tracker store in the deployment but using lockstore instead. From the Helm Chart of rasa, 'applicationSettings.endpoints.lockStore"
      # -- Enable endpoint for Lock Store
      enabled: true

      # -- Lock Store type
      type: "redis"

      # -- The url of your redis instance
      url: ${REDIS_HOST}

      # -- The port which redis is running on
      port: ${REDIS_PORT}

      # -- Password used for authentication
      password: ${REDIS_PASSWORD}

      # -- The database in redis which Rasa uses to store the conversation locks
      db: "1"

As the command prompt recommends me to enable the lockstore when the ‘replicaCount’ is set to be larger than 1.

  1. Yes, I configure the bot using rest channel. But, how can I check the sessionID when deploying to Kubernetes?

  2. I am using the Botfront rasa chabot and do not know whether it has a bug or not. I think majority are using this widget

  3. I config the ‘rasa-values.yaml’ as follows"

# rasa-values.yaml
    # -- Determine if external URL is used
    enabled: true
    # -- URL to Rasa Action Server
    url: "http://rasa-action-release-rasa-action-server/webhook"

As you can see above, the service name of rasa action server is ‘rasa-action-release-rasa-action-server’

Hopes you can help me. Thanks!

Regarding Lock Store Yes indeed this is to ensure there is no concurrent lock when you have multiple replicas. However Tracker Store is not related to concurrency rather it preserves the state. If you don’t externalise the tracker, it would use InMemoryTrackerStore where the state is created in each of the pods. So in your case you must also externalise the tracker store. Since you are already running redis, you can use it as well to create a tracker store.

I am not sure about Botfront but maybe there is a kind of lock. Regarding sessionID, since you are using botfront, i suppose it is creating them already.

you can --enable-api when running rasa server so you can access the tracker for more information

Could you also replace the action-server port from 80 to 5005. Usually 80 is a reserved port so it might be that your rasa server is not able to reach action server.

@souvikg10 In other words, I should both enable the tracker store and lock store? However, could they both install in an individual redis? Is it an redis container is created, and there are 2 pods, one is tracker store, another is lock store?

The rasa server is running with the command --enable-api --cors '*' in the pods.

I should have use the port 5055 for the rasa action server, here is the values.yaml for action server:

# Default values for rasa-action-server.
# This is a YAML-formatted file.

# -- (string) Override name of app
nameOverride: ""

# -- (string) Override the full qualified app name
fullnameOverride: ""

# -- Registry to use for all Rasa images (default
## DockerHub - use

  # -- Port on which Rasa Action Server runs
  port: 5055

  # -- Scheme by which the service are accessible
  scheme: http

# -- Specify the number of Action Server replicas
replicaCount: 2

# -- Override the default arguments for the container
args: []

# -- Override the default command for the container
command: []

# -- Add extra environment variables
extraEnv: []
#   value: "custom value"

## Define the rasa image to work with
  # -- Action Server image name to use (relative to `registry`)
  name: docker-rasa-action

  # -- Action Server image tag to use
  tag: "v5"

  # -- Override default registry + for Action Server
  repository: ""

  # -- Action Server image pullPolicy
  pullPolicy: IfNotPresent

  # -- Action Server repository pullSecret
  ## See
  pullSecrets: []
  #   - name: "<SECRET>"

  # -- Specifies whether a service account should be created
  create: false

  # -- Annotations to add to the service account
  annotations: {}

  # -- The name of the service account to use.
  # If not set and create is true, a name is generated using the fullname template
  name: ""

# -- Annotations to add to the action-server's pod(s)
podAnnotations: {}
  # key: "value"

# -- Labels to add to the action-server's pod(s)
podLabels: {}
  # key: "value"

# -- Annotations to add to the action-server deployment
deploymentAnnotations: {}
  # key: "value"

# -- Labels to add to the action-server deployment
deploymentLabels: {}

# -- Defines pod-level security attributes and common container settings
## See:
podSecurityContext: {}
  # fsGroup: 2000

# -- Allows you to overwrite the pod-level security context
securityContext: {}
  # capabilities:
  #  drop:
  #    - ALL
  #  readOnlyRootFilesystem: true
  #  runAsNonRoot: true
  #  runAsUser: 1000

## Configuration for the service for the action-server
  # -- Set type of action-server service
  type: ClusterIP

  # -- Set port of action-server service (Kubernetes >= 1.15)
  port: 80

  # -- Annotations to add to the service
  annotations: {}

  # -- Specify the nodePort(s) value(s) for the LoadBalancer and NodePort service types
  ## Ref:

  # -- Exposes the Service externally using a cloud provider's load balancer
  ## Ref:

  # -- Enable client source IP preservation
  ## Ref:
  externalTrafficPolicy: Cluster

## Configure the ingress resource that allows you to access the
## deployment installation. Set up the URL
## ref:
  # -- Set to true to enable ingress
  enabled: false

  # -- Ingress annotations
  annotations: {}
  # nginx
  # "true"

  # -- Labels to add to the ingress
  labels: {}

  # -- Ingress Path type
  ## Ref:
  pathType: ImplementationSpecific

  # -- Ingress path
  path: /

  # -- Hostname used for the ingress
  hostname: chart-example.local

  # -- TLS configuration for ingress
  ## See:
  tls: []
  #  - secretName: chart-example-tls
  #    hosts:
  #      - chart-example.local

  # -- Any additional arbitrary paths that may need to be added to the ingress under the main host
  extraPaths: {}
  # - path: /*
  #   backend:
  #     serviceName: ssl-redirect
  #     servicePort: https

# -- Resource requests and limits
resources: {}
# We usually recommend not to specify default resources and to leave this as a conscious
# choice for the user. This also increases chances charts run on environments with little
# resources, such as Minikube. If you do want to specify resources, uncomment the following
# lines, adjust them as necessary, and remove the curly braces after 'resources:'.
# limits:
#   cpu: 100m
#   memory: 128Mi
# requests:
#   cpu: 100m
#   memory: 128Mi

## Autoscaling parameters for the Action Server Deployment
## See:
  # -- Enable autoscaling
  enabled: false

  # -- Lower limit for the number of pods that can be set by the autoscaler
  minReplicas: 1

  # -- Upper limit for the number of pods that can be set by the autoscaler.
  # It cannot be smaller than minReplicas.
  maxReplicas: 20

  # -- Fraction of the requested CPU that should be utilized/used,
  # e.g. 70 means that 70% of the requested CPU should be in use.
  targetCPUUtilizationPercentage: 80
  # targetMemoryUtilizationPercentage: 80

# -- Allow the Action Server Deployment to be scheduled on selected nodes
## Ref:
## Ref:
nodeSelector: {}

# -- Tolerations for pod assignment
## Ref:
tolerations: []

# -- Allow the Action Server Deployment to schedule using affinity rules
## Ref:
affinity: {}

# -- Allow the deployment to perform a rolling update
## ref:
  type: RollingUpdate
    maxSurge: 1
    maxUnavailable: 0

# -- Override default liveness probe settings
# @default -- Every 15s / 6 KO / 1 OK
## Ref:
    path: /health
    port: http
    scheme: HTTP
  initialDelaySeconds: 15
  periodSeconds: 15
  successThreshold: 1
  timeoutSeconds: 5
  failureThreshold: 6

# -- Override default readiness probe settings
# @default -- Every 15s / 6 KO / 1 OK
## Ref:
    path: /health
    port: http
    scheme: HTTP
  initialDelaySeconds: 15
  periodSeconds: 15
  successThreshold: 1
  timeoutSeconds: 5
  failureThreshold: 6

# -- Allow to specify init containers for the Action Server Deployment
## Ref:
initContainers: []
#  - name: init
#    image: "busybox"
#    command: ["bash", "-c"]
#    args:
#      - echo "init container"

# -- Specify additional volumes to mount in the action-server container
## Ref:
volumes: []
#   - hostPath:
#       path: <HOST_PATH>
#     name: <VOLUME_NAME>

# -- Specify additional volumes to mount in the action-server container
volumeMounts: []
#   - name: <VOLUME_NAME>
#     mountPath: <CONTAINER_PATH>
#     readOnly: true

Should I also change the port in service.port from 80 to 5055?


@edward0422 - Redis is a like a database, and with it’s own distributed system, you don’t need to spin up pods based on what tables you make.

for redis, i would create pods based on how much memory it needs. Tracker Store can be a large table so two replicas of redis should be enough to sustain both lock store(different table) and tracker store. you don’t need to assign these tables to individual pods, redis takes care of it.

and regarding the port indeed it is better to use ports like 5055 for action server


However, in the values.yaml for building helm chart of rasa server, it claims that the Tracker Store is recommended to build in Postgresql, see the comments below:

    ## You can use a Tracker Store to store your assistant's conversation history.
    ## See:
    ## All environment variables used as values are added to the rasa-oss container automatically if `postgresql.install=true`.
      # -- Enable endpoint for Tracker Store
      enabled: true

      # -- Tracker Store type
      type: sql

      # -- The dialect used to communicate with your SQL backend
      dialect: "postgresql"

      # -- URL of your SQL server
      url: ${DB_HOST}

      # -- Port of your SStoreQL server
      port: ${DB_PORT}

      # -- The username which is used for authentication
      username: ${DB_USER}

      # -- The password which is used for authentication
      password: ${DB_PASSWORD}

      # -- The path to the database to be used
      db: ${DB_DATABASE}

      # -- Create the database for the tracker store.
      # If `false` the tracker store database must have been created previously.
      login_db: ${DB_DATABASE}
    # -- postgresqlUsername which should be used by Rasa to connect to Postgres
    postgresqlUsername: "postgres"

    # -- postgresqlPassword is the password which is used when the postgresqlUsername equals "postgres"
    postgresqlPassword: "password"

    # -- existingSecret which should be used for the password instead of putting it in the values file
    existingSecret: ""

    # -- postgresDatabase which should be used by Rasa
    postgresqlDatabase: "rasa"

    # -- servicePort which is used to expose postgres to the other components
    servicePort: 5432

## PostgreSQL specific settings (
  # -- Install PostgreSQL
  install: true

  ## Use external PostgreSQL installation
  ## This section is not a part of the PostgreSQL subchart
    # -- Determine if use an external PostgreSQL host
    enabled: false

    # -- External PostgreSQL hostname
    ## The host value is accessible via the `${DB_HOST}` environment variable
    host: "external-postgresql"

Therefore. I install postgresql and enable it for tracker store. Would it be the same?

Regarding to the port, the rasa action server cannot work if I change the service.port from 80 to 5055 . I guess the 80 port is for the service and allow the rasa server to communicate with it. I only keep applicationSettings.port to be 5055 which let the rasa action server runs on 5055

@souvikg10 Anything should I try?

@edward0422 you can other ports for your service as well, since it is running on the same cluster as the rasa server, you can assign another port for the service like 5055, only reason i am saying this is because 80 for some cluster might be reserved. and yeah postgres/redis both are supported for tracker store. it’s the same.