Unrelevant input and emails

Rasa Open Source
1

Hey guys and gals,

I am trying to build a bot that should be used to help our Costumer Support by answering the simple questions, before people actually bother real humans :wink: So far we don’t have real user input but we have emails from users. Do you guys think it would make sense to:

a) scan the emails for usable questions that can be used for the intents b) send the emails to the chatbot to see if it gives a valid answer (or maybe send the emails sentence by sentence? Because emails can be multi threaded… What do y’all think?) c) use parts of the mails, that can not be answered to create an in-domain-but-out-of-scope question? Would that be usefull?

Furthermore I was also wondering, if there is a way to get a look behind the curtains, so to say? We observed, that in many cases the intent classification assigns some intent with a rather high confidence to texts that do not fit any intent, but are a bit longer. Is this expected behaviour? Would it be possible to kindof see the “weight” of certain words that might trigger this missclassification or do I simply need more data and hope for the best?

i hope to get some feedback :wink: If you need to know more info ,you need only ask ^^

Best Dominik

2

I think this might be a pretty broad question but when I was recently creating a demo helpdesk assistant example I just used forms and the inform intent to setup how the dialog flow went. This way I didn’t need to have a bunch of training data for the questions just ask them their issue then use the from_text option we have in the sdk to set that as an entity.

You can see a basic example at GitHub - btotharye/helpdesk_assistant: Basic demo use case showing Rasa X with Service Now API calls to open incidents. I’m using forms with a custom action code to handle everything.

Not sure if that helps on that front.

As far as the other question you might want to look more at the --debug flag for more info where it normally will show some of the confidence information and you can also run tests against your data for further information, we have some information at Evaluating Models on some of the different commands you can perform.