Anyone with experience troubleshooting AWS Deployment issues with Elastic Beanstalk for Rasa? [Solved]

I’m wondering if anyone has run into similar issues before. I’m not 100% sure if it’s an issue related to Rasa, or if it’s in the Elastic Beanstalk instance that I have spun up.

Essentially, I’m running Rasa Core, Rasa NLU, and the Action server off of Docker with my own docker-compose.yml as the examples in the docs have mentioned (and have worked fine on local). I’ve deployed this to a Multi-container Docker Elastic Beanstalk instance. I was able to successfully deploy the project with a single instance load balancer and each of the server’s ports allowed through the security group (just for testing). I can reach out to Rasa NLU via [EC2 instance IP]:5000 or the action server via [EC2 instance IP]:5055 from my browser, but when I try and hit the Core server, I get the below error message in the logs:

 Failed to parse text 'this is a test' using rasa NLU over http. Error: 
 HTTPConnectionPool(host='127.0.0.1', port=5000): Max retries exceeded with url: //parse? 
 model=default&project=&q=this+is+a+test&message_id=d7d6863006c845ffaae7a0a4c7e51c76 
 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 
 0x7f4cd55fbcc0>: Failed to establish a new connection: [Errno 111] Connection refused',))

If I ssh into the instance, and curl http://127.0.0.1:5000 it works fine.

I’ve tinkered around with the rasa setup (i.e. changing endpoints from something like http://rasa-nlu:5000 to http://127.0.0.1:5000 since that’s what the container maps to in the instance), but nothing’s worked, and I’m guessing it’s something related to the security measures that Elastic Beanstalk has.

Anyone have an idea for something I can change?

how does your endpoints config for Rasa Core look like?

Sorry @Tobias_Wochinger , forgot I hadn’t posted the solution to this! I was able to get it fixed without much issue, but this is what my config/endpoints.yml looks like:

nlu:
  url: "http://rasa-nlu:5000/"

action_endpoint:
  url: "http://action-server:5055/webhook"

and this is what my Dockerrun.aws.json looks like now:

{
  "AWSEBDockerrunVersion": 2,
  "containerDefinitions": [
    {
      "name": "rasa-core",
      "image": "rasa/rasa_core:stable",
      "essential": true,
      "memory": 900,
      "memoryReservation": 300,
      "command": [
        "start",
        "--core",
        "model",
        "-c",
        "rest",
        "--endpoints",
        "config/endpoints.yml",
        "-u",
        "live/", // note the live specifier for AWS, docker-compose will point to local/
        "-o",
        "logs/server.log",
        "-vv"
      ],
      "mountPoints": [
        {
          "containerPath": "/app/model",
          "sourceVolume": "Rasa_Core"
        },
        {
          "containerPath": "/app/config",
          "sourceVolume": "Config"
        },
        {
          "containerPath": "/app/logs/",
          "sourceVolume": "Rasa_Core_Logs"
        }
      ],
      "portMappings": [
        {
          "containerPort": 5005,
          "hostPort": 80
        }
      ],
      "links": [
        "rasa-nlu",
        "action-server"
      ]
    },
    {
      "name": "rasa-nlu",
      "image": "rasa/rasa_nlu:stable-full",
      "essential": true,
      "memory": 300,
      "memoryReservation": 100,
      "command": ["start", "--path", "models", "--response_log", "logs/", "-v"],
      "mountPoints": [
        {
          "containerPath": "/app/models/",
          "sourceVolume": "Rasa_NLU"
        },
        {
          "containerPath": "/app/logs",
          "sourceVolume": "awseb-logs-rasa-nlu"
        }
      ],
      "portMappings": [
        {
          "containerPort": 5000,
          "hostPort": 5000
        }
      ]
    },
    {
      "name": "action-server",
      "image": "rasa/rasa_core_sdk:latest",
      "essential": true,
      "memory": 90,
      "memoryReservation": 30,
      "command": ["start", "--actions", "actions"],
      "mountPoints": [
        {
          "containerPath": "/app/actions",
          "sourceVolume": "Actions"
        }
      ],
      "portMappings": [
        {
          "containerPort": 5055,
          "hostPort": 5055
        }
      ]
    }
  ],
  "volumes": [
    {
      "host": {
        "sourcePath": "/var/app/current/models/rasa_core"
      },
      "name": "Rasa_Core"
    },
    {
      "host": {
        "sourcePath": "/var/app/current/logs/core"
      },
      "name": "Rasa_Core_Logs"
    },
    {
      "host": {
        "sourcePath": "/var/app/current/config"
      },
      "name": "Config"
    },
    {
      "host": {
        "sourcePath": "/var/app/current/models/rasa_nlu"
      },
      "name": "Rasa_NLU"
    },
    {
      "host": {
        "sourcePath": "/var/app/current/actions"
      },
      "name": "Actions"
    }
  ]
}

This is what my docker-compose.yml looks like:

version: '3.0'

services:
  rasa-core:
    image: rasa/rasa_core:stable
    ports:
      - "5005:5005"
    networks: ['rasa-network']
    volumes:
      - ./data/stories:/app/stories
      - ./models/rasa_core:/app/model
      - ./config:/app/config
      - ./logs/core:/app/logs
      - ./:/app/project
    command:
      - start
      - --core
      - model
      - -c
      - rest
      - --endpoints
      - config/endpoints.yml
      - -u
      - local/
      - -o
      - /app/logs/server.log
      - -vv
    depends_on:
      - rasa-nlu
      - action-server
  rasa-nlu:
    image: rasa/rasa_nlu:stable-full
    volumes:
      - ./models/rasa_nlu:/app/models/
      - ./logs/nlu:/app/logs
      - ./config:/app/config
      - ./:/app/project
    ports:
      - "5000:5000"
    networks: ['rasa-network']
    command:
      - start
      - --path
      - models
      - --response_log
      - logs/
      - -v
  action-server:
    image: rasa/rasa_core_sdk:latest
    volumes:
      - ./actions:/app/actions
    ports:
      - "5055:5055"
    networks: ['rasa-network']
    command:
      - start
      - --actions
      - actions

networks: {rasa-network: {}}

And this is my folder structure:

project/
├── Dockerrun.aws.json # for AWS deploy
├── docker-compose.yml # for local dev
├── domain.yml
├── policies.yml
├── actions
│   └── # some actions here
├── config
│   ├── endpoints.yml
│   └── nlu_config.yml
├── data
│   ├── nlu
│   │   └── # some NLU data here
│   └── stories
│       └── # some story data here
└── models
    ├── rasa_core
    │   ├── live
    │   │   └── # live core model here
    │   └── local
    │       └── # local core models here
    └── rasa_nlu
        ├── live
        |   └── # live nlu model here
        └── local
            └── # local nlu models here

This actually might be worth adding into the documentation as a tutorial, since it’s the easiest folder structure to deploy to AWS Elastic Beanstalk with a local environment with docker-compose and live environment with AWS. I’ll see if I can find some time to create a tutorial and create a pull request.

Awesome work! @niveK

@niveK Can you please share the updated versions of the scripts for the latest RASA version 1.1.4?

1 Like

Were you able to get a working rasa 1.1.4 Dockerrun.aws.json file?

Nope

try something like this:

     {
      "AWSEBDockerrunVersion": 2,
      "containerDefinitions": [
        {
          "name": "rasa",
          "image": "rasa/rasa:latest-full",
          "essential": true,
          "memory": 900,
          "memoryReservation": 300,
          "command": [
            "run",
            "--log-file",
            "serverlog.log",
            "-vv"
          ],
          "mountPoints": [
            {
              "containerPath": "/app",
              "sourceVolume": "Rasa"
            }
          ],
          "portMappings": [
            {
              "containerPort": 5005,
              "hostPort": 80
            }
          ],
          "links": [
            "action_server"
          ]
        },
        {
          "name": "action_server",
          "image": "rasa/rasa-sdk:latest",
          "essential": true,
          "memory": 90,
          "memoryReservation": 30,
          "mountPoints": [
            {
              "containerPath": "/app/actions",
              "sourceVolume": "Actions"
            }
          ],
          "portMappings": [
            {
              "containerPort": 5055,
              "hostPort": 5055
            }
          ]
        }
      ],
      "volumes": [
        {
          "host": {
            "sourcePath": "/var/app/current"
          },
          "name": "Rasa"
        },
        {
          "host": {
            "sourcePath": "/var/app/current/actions"
          },
          "name": "Actions"
        }
      ]
    }
1 Like

I’ve been working on a previous version, so I haven’t had to set up the Rasa 1.0 deployment process, @KarthiAru. This like it should work pretty well for you.

Hi, any chance you make this tutorial or atleast send these files again since probably a lot of things changed since you posted this.

I’m currently in the same process of deploying my rasa application to AWS so this would be very helpful to me. :slight_smile:

Sure! I’ll try and get to creating a quick tutorial next week, don’t really have a lot of time right now :slight_smile:

2 Likes

Hi @niveK did you get a chance to create the tutorial? Thanks

I haven’t, no. Here’s a Dockerrun.aws.json with the latest release.

{
	"AWSEBDockerrunVersion": 2,
	"containerDefinitions": [
		{
			"name": "rasa",
			"image": "rasa/rasa:1.9.5",
			"essential": true,
			"memoryReservation": 400,
			"command": [
				"run",
				"--log-file",
				"/logs/rasa.log"
			],
			"mountPoints": [
				{
					"containerPath": "/app",
					"sourceVolume": "App"
				},
				{
					"containerPath": "/logs",
					"sourceVolume": "awseb-logs-rasa"
				}
			],
			"portMappings": [
				{
					"containerPort": 5005,
					"hostPort": 80
				}
			],
			"links": [
				"action-server",
				"duckling"
			]
		},
		{
			"name": "action-server",
			"image": "rasa/rasa-sdk:1.9.0",
			"essential": true,
			"memoryReservation": 100,
			"command": [
				"start",
				"--actions",
				"actions"
			],
			"mountPoints": [
				{
					"containerPath": "/app/actions",
					"sourceVolume": "Actions"
				}
			],
			"portMappings": [
				{
					"containerPort": 5055,
					"hostPort": 5055
				}
			]
		},
		{
			"name": "duckling",
			"image": "rasa/duckling:latest",
			"essential": true,
			"memoryReservation": 100,
			"portMappings": [
				{
					"containerPort": 8000,
					"hostPort": 8000
				}
			]
		}
	],
	"volumes": [
		{
			"host": {
				"sourcePath": "/var/app/current"
			},
			"name": "App"
		},
		{
			"host": {
				"sourcePath": "/var/app/current/actions"
			},
			"name": "Actions"
		}
	]
}

Note: you’ll need a .ebextensions/ directory in your project root with 2 files for this to work.

.ebextensions/elb-listener.config

option_settings:
  aws:elb:listener:80:
    ListenerProtocol: HTTP
    InstanceProtocol: HTTP
    InstancePort: 80

.ebextensions/elb-ingress.config

Resources:
  port80SecurityGroupIngress:
    Type: AWS::EC2::SecurityGroupIngress
    Properties:
      GroupId: {"Fn::GetAtt" : ["AWSEBSecurityGroup", "GroupId"]}
      IpProtocol: tcp
      ToPort: 80
      FromPort: 80
      SourceSecurityGroupName: { "Fn::GetAtt": ["AWSEBLoadBalancer", "SourceSecurityGroup.GroupName"] }

Hi @niveK. I’ve been trying to implement your most recent post. Unfortunately, I’m getting this error on Beanstalk:

Service:AmazonCloudFormation, Message:Template error: instance of Fn::GetAtt references undefined resource AWSEBLoadBalancer

Did you come across the same issue?

This is my Dockerrun.aws.json. In my case, rasa is not at var/app/current but at var/app/current/rasa because my project structure is

- application
-- src
-- telegram
-- flask 
-- rasa

Dockerrun.aws.json:

{
    "AWSEBDockerrunVersion": 2,
    "volumes": [
        {
          "host": {
                "sourcePath": "/var/app/current/rasa"
            },
            "name": "Rasa"
        }
      ],
    "containerDefinitions": [
        {
            "name": "telegram",
            "image": "s0288/telegram",
            "hostname": "telegram",
            "essential": true,
            "memory": 900
        },
        {
            "name": "flask",
            "image": "s0288/flask",
            "hostname": "flask",
            "essential": false,
            "portMappings": [
                {
                    "containerPort": 5000,
                    "hostPort": 5000
                }
            ],
            "memory": 900
        },
        {
            "name": "rasa",
            "image": "rasa/rasa:1.10.3-full",
            "hostname": "rasa",
            "essential": false,
            "memory": 900,
            "memoryReservation": 512,
            "command": ["run"],
            "mountPoints": [
                {
                  "containerPath": "/app",
                  "sourceVolume": "Rasa"
                }
              ],
            "portMappings": [
                {
                    "containerPort": 5005,
                    "hostPort": 80
                }
            ],              
            "links": ["flask"]
        }
    ]
}

And my elb-ingress.config file:

Resources:
  port80SecurityGroupIngress:
    Type: AWS::EC2::SecurityGroupIngress
    Properties:
      GroupId: {"Fn::GetAtt" : ["AWSEBSecurityGroup", "GroupId"]}
      IpProtocol: tcp
      ToPort: 80
      FromPort: 80
      SourceSecurityGroupName: { "Fn::GetAtt": ["AWSEBLoadBalancer", "SourceSecurityGroup.GroupName"] }

Thanks.

Apparently my environment wasn’t set up for load balancing yet. After changing the environment type, I got a new error:

Application update failed at 2020-06-28T21:20:52Z with exit status 1 and error: Hook /opt/elasticbeanstalk/hooks/appdeploy/enact/03start-task.sh failed.

Will do some further research…

Hi Alex,

Sorry, have been out sick for a few days and didn’t see this. Yes, I’ve seen that there are issues with having a load balancer set up for a multi-container Docker on the latest platform version; I’m still on an older version, but I’ll need to upgrade soon and deal with that issue. Unfortunately there’s little documentation on this change in the AWS docs that I could find. Let me know if you figure it out, I’d be curious to see what the solution would be.

Hi guys.

Just a quick update on my earlier issue as it may help someone. In the end, I found AWS frustrating and decided to not spend more time on it. I switched to DigitalOcean and use a Linux-based Droplet to run my docker-compose.yml file directly.

It’s a bit more hacky / less polished, but works for my purposes.

1 Like

Also, in case anyone was curious, I’m planning on moving over to an AWS Fargate cluster since I began baking in the necessary files into a custom image each for the rasa and rasa-sdk resources. That meant I only had to point a Dockerrun.aws.json to those images, which removes much of the need for an AWS EB application. I’d recommend moving away from Elastic Beanstalk as soon as you can.