Add conversation persistence with load balancer (nginx, redis, found possible solution)

Rasa Open Source
1

Hello everyone

I am trying to deploy multiple opensource rasa servers through a load balancer (nginx) with reverse proxy

I currently have three instances deployed on Google Cloud Engine:

  • Instance 1: NGINX
  • Instance 2: Server 1 (server+actions)
  • Instance 3: Server 2 (server+actions)

If I use only the load balancer instance and a server, everything works fine, but when I introduce the second rasa server in the load balancer, the bot responds but not as it should, an example:

I tell it my name
i ask who I am
bot tell me that I have not told who i am yet
i ask again
bot answer me correctly.
I ask again
bot tell me that I have not told who i am yet

This is obviously the behavior of the load balancer, which alternates the servers, how could I make said connection permanent for x amount of time? Would this be correct?

I have read in other posts about the use of Redis but it is not clear to me how to introduce it in the deployment, would I have to install it in the load balancer and connect the servers to it, or would I have to install it in each server?

2

Nevermind, I just found the solution after too much forums and try-error (I always find the solution after asking, it doesn’t matter if I spend a whole day trying it before, it’s asking and hitting the nail on the head)

Architecture:

  • Instance 1:

    • nginx container (load balancer and reverse proxy)
    • redis container (rasa servers will connect to this db in 6379 port)

  • Instance 2:

    • rasa server container
    • rasa actions container

  • Instance 3:

    • rasa server container
    • rasa actions container (i have to see if i can set only one action server)

Docker-compose redis file:

version: '3.0'
services:
  redis:
    image: redis:latest
    container_name: redis
    ports:
      - 6379:6379

Now, each rasa server endpoint has to set this configuration:

tracker_store:
    type: redis
    url: load-balancer-ip (preferable the internal ip, where the redis container is deployed)
    port: 6379
    db: 1
    password: (yes, this is empty, no authentication will be done)


lock_store:
    type: redis
    url: load-balancer-ip (preferable the internal ip, where the redis container is deployed)
    port: 6379
    db: 0
    password:
    key_prefix: rasa

#tracker_store:
#  type: sql

Then, when you start both rasa server, the nginx and redis containers, you can access to the redis container with:

  • docker exec -it id /bin/bash

Then run:

  • redis-cli

Once in redis interface, run

  • select 1 (this is the tracker database)

Run:

  • keys *

Now you will see all the tracker id of the user who talked to the bot, you can see the conversation with:

  • get tracker: id

In this way, although the telegram request is balanced between the servers, the database will be unified and therefore, each user will have a record of their history, allowing interactions not to be mixed

Kudos to Rasa course, rasa training, rasa interview, docker tracker store redis service deployment of Rasa practical series

Also, for bonus configuration, this is my nginx configuration (i know i can do it better)

  • default.conf
#/etc/nginx/conf.d/default.conf
upstream lb {
  include servers;
}

# http access with reverse proxy
server {
  listen 80;

  # reverse proxy
  location / {
    proxy_pass http://lb;
  }
}

# https access with reverse proxy
server {
  listen 443 ssl;

  # set ssl certificates for https access
  ssl_certificate /etc/letsencrypt/live/your-ld-domain/fullchain.pem;
  ssl_certificate_key /etc/letsencrypt/live/your-ld-domain/privkey.pem;
  ssl_trusted_certificate /etc/letsencrypt/live/your-ld-domain/fullchain.pem;

  # reverse proxy
  location / {
    proxy_pass  http://lb;
  }
}
  • servers file (the include reference)
server ip-1:5005;
server ip-2:5005;

The ssl certificates has been created with certbot and letscrypt, you will need to use a domain (aws ec2 domains does not work Policy forbids issuing for name on Amazon EC2 domain - #4 by jsha - Server - Let's Encrypt Community Support)

It would be nice if someone can confirm this solution to mark it as Solution, as anyone has responded the thread yet

Regards, Raúl.