Is RASA really scalable?

Rasa Open Source
1

Hi,

Its been an year i am working with RASA and in a way i found it very helpful when building an FAQ like bot where the user gives a question and bot gives the answer, But this whole scenario changes when you want to built a contextual bot where the bot has to gather multiple information before providing a result.Currently i am building a bot for production where it is very tedious with writing stories to handle edge cases, doing some custom handling inside my formaction to not let my bot break.

Coming to scalability

I started with an FAQ bot which was piece of cake with RASA NLU and Core. Then I had to add a context based intent into the current bot , now it needs to handle both FAQ and context based Intents in same bot.Bot works like (User testing)

  • User : Hi
  • Bot : Hey boss
  • User : how to create an account
  • Bot : (Info on how to create an account)
  • User: So i want to transfer money
  • Bot : Please let me know your account number
  • User : how to create an account
  • Bot : I am sorry i didn’t userstand ## edge case1
  • User : tell me about how to create account
  • Bot : I am sorry i didn’t userstand (User not happy :frowning: ) So i again wrote some stories to handle occurance of such cases a number of times. (Next User test)
  • User : Hi
  • Bot : Hey boss
  • User: So i want to transfer money
  • User : how to create an account
  • Bot : (Info on how to create an account) your were in between a money transaction so please enter the account number?
  • User: No i would like to stop
  • Bot: I am sorry i didn’t userstand
  • User : I said stop
  • Bot : I am sorry i didn’t userstand (User in now furious :triumph:) Again i had to write stories not just one but few combination to handle this scenario (Training time went from 10 mins to 1 hours so unit testing at this point of time is very hard).I Wrote few utilities to automate testing to check every time the previous working things are still working (User test again)
  • User : Hi
  • Bot : Hey boss
  • User: So i want to transfer money
  • User : how to create an account
  • Bot : (Info on how to create an account) your were in between a money transaction so please enter the account number?
  • User: No i would like to stop
  • Bot: It seems you want to stop please press (yes) or (no)
  • User: no
  • Bot: please enter the account number?
  • User : what are different credit cards available
  • Bot:(Answer) your were in between a money transaction so please enter the account number?
  • User: before that i want to know minimum requirement for credit card
  • Bot:(Answer)your were in between a money transaction so please enter the account number?
  • User: cancel it
  • Bot: sorry i didn;t understand (User has lost it now :rage:) Every time the size of my story and code is increasing to handle these requirement.

This is just the case of single formaction. I have more such intents , 50+ FAQ intents, along with small talk intents to handle.

I have also found every time the model trains there are some slight difference in the way the policy predicts the next actions.

Is there anyway to solve the above problems @amn41 @MetcalfeTom @akelad @erohmensing

I love :heart: RASA i really do but i seek a solution for my above issues.

2

@dakshvar22 maybe this is a good use case for your branch? @Anand_Menon Daksh’s new stuff may help you out.

3

What is this new stuff? @dakshvar22

4

@Anand_Menon We have been building a “response selector” component which can come in handy while dealing with open doman intents/small talk utterances as you mentioned you have in your use case. You can read more about this feature here. This is still an experimental feature and in active development. Feel free to go through it and try it out on your data particularly for small talk intents. We also have a separate thread for feedback on this particular feature.

5

Well this look great and promising :slight_smile:. I would definitely try this one and i hope this works in older rasa version since i have not migrated to rasa 1.0.

6

@Anand_Menon You would have to checkout the specific branch mentioned in the introduction of the feature. Unfortunately I can’t guarantee this to work for versions before Rasa 1.0

7

@Anand_Menon You would definitely need to upgrade to specific version of branch which is definitely post Rasa 1.0 .

8

At the moment we are not planning to migrate to RASA 1.0