How do you measure the success (rate) of your bot?

Rasa Open Source
1

Hey there,

since we’re currently working on improving our Rasa demo bot that lives in our docs we started measuring the success rate of the bot to see how it improves from week to week. That made us curious how you all measure the success of your conversational assistant? Anything that worked well and was good to measure, anything that you tried that didn’t work at all?

We’re excited to learn how you handle this not so easy question :slight_smile:

Oh, and if you have ideas what we can do to improve our bot and which features we can add, we’re happy about an answer in this discussion.

Cheers Linda

1 Like
2

Hi Community!

I am trying to find out exactly how this could be achieved. The approach I’m currently investigating is a setup as the Sara Demo bot, where each response is followed by a feedback positive/negative intent. But then to parse the tracker object to extract number of engagements and calculate the success rate as #of responses/ #of responses with positive feedback. Can you please share your approach? And has anyone successfully applied what i’ve just mentionned?

Thanks, Andrés.