Rasa docker scaling

Hi I am trying to scale my docker rasa container using nginx as reverse proxy but I am getting too many failed requests. Can you help me with this? I have also made a custom tracker store that is hosted on ec2. I am using mysql as the database.

This is my Dockerfile

From python:3.8

RUN python -m pip install rasa==2.8.2 mysqlclient protobuf==3.20.1

WORKDIR /app
COPY . . 

RUN rasa train 

# set the user dont use as root
USER 1001
# set endpoint for interactive shells
ENTRYPOINT ["rasa"]
#  command to run when container is called to run 
CMD ["run", "--enable-api", "--port", "5005"]

This is my docker compose file

version: '2.0'
services:
  rasa:
    image: rasa/test
    # ports:
    #   - 5005:5005
    volumes:
      - ./:/app
    command:
      - run
  nginx:
    image: nginx:latest
    volumes: 
      - ./conf.d:/etc/nginx/conf.d
    depends_on:
      - rasa
     
    ports:
      - 3000:3000

and lastly this is my conf.d file

server{
    listen 3000;
    location / {
        proxy_pass http://rasa:5005;

    
    } 


}


the instance I am using is t2.large on aws

and this is the command that I am using for scaling

docker-compose up --scale rasa=2

Also I am very new to scaling docker containers with nginx.

Any help regarding this would be greatly appreciated

I increased the container number to 4 and the response that I got was much better than the previous one

But How do I reduce the average response time?. It is still 40 seconds

If anyone is able to help me I will be very thankful

Kubernetes and the helm chart are recommended for scaling Rasa deployments.

Regarding the response time, you’d need to provide more information. Jaeger and OpenTelemtry can be used for detailed performance tracking. There’s a post on this here and an older docker image I had built for this purpose.