Comparing two deployment approaches: single bot vs multi bot

Rasa Open Source
1

Hello!

Several scrum teams at Dialogue (dialogue.co @moaazsidat @Lp-dialogue @mbelang ) are building different chatbots for our system. We’re considering two deployment approaches and wanted to get your take:

The first approach is to deploy a single instance of rasa stack, have a single domain.yml and single story.md file. The second approach is to deploy multiple instances of rasa stack. Each rasa runtime would be responsible for a single bot.

We’ve come up with the following list of pros/cons of each approach.

Do think about this in the right way? Did we miss any pros/cons? What’s been your experience in building multiple bots with different scrum teams?

For the purpose of this discussion, consider two bots - one performs a mental health assessment, the other books an appointment.

Single bot

A design where a single rasa core bot manages all dialogue flows through a single instance of rasa core and an actions server – and through it, share a single domain and stories file.

Pros

  • this is very much how Rasa is natively modelled today
  • let’s us train/optimize across different starting intents in the long term. Use case “I’m feeling suicidal” mentioned in mental health bot and that influences a decision made by the appointment bot later in the flow.
  • all of the utterances are stored in a single git repo, making it easier to manage when it comes to figuring out where specific conversations are
  • ML: the longer the flow controlled by the bot, the more it can “learn” and have a big impact on flow optimization. If the bot controls steps 1 to 10, it can learn to ask or not to ask specific questions on 10 steps knowing the whole flow. If we reduce this scope, it’s harder to learn since it will maximize a ‘sub-objective’ that is not the holistic one.

Cons

  • harder to integrate new flows and ensure existing flows remain consistent
  • harder to manage lots of intents/actions/slots that span multiple flows in a single domain/story
  • harder to have multiple teams working concurrently
  • possibility of intent/slot clash (there are no namespaces)

Multi bots

A design where bots are independent, where each run their own rasa core and action server, have their own domain and stories file. This requires some system of orchestration to engage/disengage these bots.

Pros

  • aligns more closely with a single-responsibility principle, each bot is self-contained
  • easier to manage different bots and focus on optimizing functions that each performs
  • allows for a more plug-and-play mechanism for building bots
  • allows us to scale bots’ resources individually depending on each’s traffic (in the long run)

Cons

  • unclear how we can leverage rasa to optimize across bots if each is separated
  • how do you transfer state and context of the conversation between bots
  • won’t be able to leverage rasa tooling across the entire conversation (will still be able to on a per-bot basis)
  • higher operating cost, as these are long-lived services, not scale to zero

Your thoughts are welcome!

Thanks, Alexis

4 Likes
2

Disclaimer: I’m pretty new to the Rasa stack, mostly played around with the demos and gone over documentation, so take that for what you will.

I think you could transfer state from one bot to the other by getting all of the events from both bots’ trackers, find which events are new, and before the start of the interaction with bot B you can append the new events to bot B’s tracker. You can use https://rasa.com/docs/core/server/#operation/getTracker to get the events from the current interaction and as soon as you determine to transfer to bot B use https://rasa.com/docs/core/server/#operation/appendEvent to append those events. The user probably doesn’t even need to know they were “transferred” to a new bot.

I’m not sure if the stuff/events endpoint takes multiple events or not, I haven’t tried it out yet. So you might need to iterate over the events returned by the stuff/tracker endpoint and do them one at a time. I’d assume it is a pretty fast operation though. Plus it really seems like it should take multiple, so I’d assume it does, but that might just be because I’m trying to do that wrong and there is a better way.

This endpoint exists too, but says not to use it since it replaces ALL event with only those passed it, but as long as you always pass the entire set of events from the tracker I think it would be fine. Though it would not scale well as the number of events grows. https://rasa.com/docs/core/server/#operation/replaceEvents

3

@alexissmirnov I just started working on the same problem! Im interested in the multi bot approach but Im trying to think ahead of time how we will manage context and sequential conversation/forms across the bots and if there should be one central bot in front of the other bots that is responsible for routing requests between the other bots… Did your team end up taking the Multi-bot approach?

4

@tatianaf I just started working on the same problem! Im interested in the multi bot approach. Can you let me know if your teams have taken up Multi-bot approach? If yes, can you share details on your approach.

Thank’s In Advance !!!