Serving Multiple Bots from Django Backend

Rasa Open Source
21

Hi @samwell,

Its working out quite well in a production environment so far, but still requires testing under load.

I ended up using Flask as opposed to Django which I initially posted about just because its simpler, so that’s what I would recommend. Works fine with multiple bots, as long as you’ve got the processing power/memory.

The trickiest part for my team was setting up the proruction environment, but that issue isn’t so much related to Flask/Rasa. Flask and Rasa seems to work together quite smoothly in my experience.

22

@raymond thats great to hear. By any chance do you have discord, slack, or any other way I can pick your brain about this (if you don’t mind of-course). I’m trying to do the same thing but I really want to go for a serverless implementation.

The end goal being that I’m able to run several bots (each with their own data directories, actions, and models) through just a simple interface - maybe a POST with a message and a botId which then determines which of the bot configurations to pick up and use for this conversation. I’m very very new to Rasa so my apologies if any of this sounds ridiculous, still in the early stages of my research :slight_smile:

Any advice from a veteran to a novice would be awesome!

23

Hey @samwell ,

I’m happy to answer any questions either here, or if there’s a private message feature on this forum (couldn’t find one when I looked) I can PM you my email address. Otherwise you could start a new thread and tag me which will give me an email alert.

Running several bot models with separate data directories/actions etc is manageable with Flask the way you describe. Have the request to the server include an identifier such as a name specifying which model it should interact with, then query the model accordingly and send back a response.

24

Hi @raymond, thanks again for the quick reply. It doesn’t look like there is a DM option and when I created a new post it wouldn’t let me tag you as you’re not already in that thread, so I’ll just continue in here.

Can you tell me about how you were able to set up your system? So you make a POST request with the message and some meta data to your flask endpoint, that controller or method then queries or pings your rasa server? Or do you have rasa running right on the same server as the flask server and it runs some command locally?

I’ve looked through the docs but can’t really find one that helps this particular use case, do you by chance have any specific links that could describe how this should be done?

Any help is appreciated as always :slight_smile:

25

Hey, sorry for the late reply.

This is how I have it set up: I have a flask RESTful API server with an endpoint which takes a POST request thst includes an identifier for the bot, an identifier for the client, and the message. When a request is received, the flask app will look at the identifier and query the Rasa bot using the Rasa python API. The response is then sent back to the client. For specifics on the Rasa API: Agent

So to answer your question of whether I have flask and Rasa running on the same server, the answer is yes since Im running Rasa from within the Flask app. The above link has more info on which methods to use.

Let me know if you want anything clarified further.

26

Please help me to work with RASA multiple bot.

27

hi… how much time it is taking to load the single model

28

@adrianhumphrey111

can you please share complete solution of multiple bot running using flask

29

Hello Shayan, how much ram do you request for 50 models??

30

@raymond can you please tell me how does your production environment looks like and how many bots are you serving this would really help a lot?

31

Hi @raymond, do you load the models using agent.load() and then run handle_message directly on the corresponding bot or do you have a function which returns the appropriate agent object and then run the handle_message on it?

Eg. Scenario 1:

my_agent_1 = Agent.load('path/to/bot1')
my_agent_2 = Agent.load('path/to/bot2')

if id == 1:
response = my_agent_1.handle_message()
elif id == 2:
response = my_agent_2.handle_message()

Or Scenario 2:

all_agents = {}
all_agents["1"] = Agent.load('path/to/bot1')
all_agents["2"] = Agent.load('path/to/bot2')

def get_agent(id):
    return all_agents[id]

agent = get_agent(id)
response = agent.handle_message()

And, if you have tried both, is there any disproportional performance impact for either Scenario when dealing with >20 bots?