This is my first “contribution” to the forum. Please be kind
We are running a bot and we optimize the intents in a separate database (think of Excel File).
In this file we save all user messages and bot answers incl. intent, confidence scores, ….
We need this file/database to run analytics, KPIs and so on.
We mark the messages that we want to optimize.
We have a field “bot_intent_wrong” and bot_intent_correction
In case we think the intent was detected wrongly we put in the database e.g “bot_intent_wrong” = “1” (1=wrong, 2=correct) and e.g. bot_intent_correction = “NewCorrectIntent”
Example A
User_message = “Hello there”
Bot_intent = “time.asking”
Bot_intent_wrong = ”1” (=wrong)
Bot_intent_correction = “greeting”
Example B
User_message = “how is the weather?”
Bot_intent = “hoisweather”
Bot_intent_wrong = ”2” (=correct)
Bot_intent_correction = “”
We have several thousand of those “corrected intents” in our database.
Now to my question: is there a way that we can automatically train the bot? e.g. can we run a script or a Rasa function that reads all the corrected messages/intents and trains/puts it into the bot?
Doing this manually would be a huge workload, so we try to optimize this process.
How do you train/optimize your bot? Especially in big projects?
Basically, you write a script to get the "User_message" from your excel and put it under the "Bot_intent_correction" intent in the nlu training file of rasa if the "Bot_intent_wrong" is 1.
Rasa will use this importer to import data from your excel at the time of training.
Hey @saurabh-m523 I am looking for a similar approach on how to automate my training data instead of manually writing intents in the NLU.md file.Can you tell what is the way to prepare automated training in RASA?
Thank you very much!
one more question: As we correct intents, that are obviously coming from the bot, does it make sense/is it possible to reffer to the old/original message in Rasa.
so instead of “just” saying “Hello there” = “greeting”
referring to the original “Hello there” message and actually correcting the “time.asking” intent to “greeting”
is there a difference between actually correcting a message/intent or “stating” intents?
Hi @MikeHH! I’m not sure if I understand your question correctly.
There is no such thing as correcting the intent of a message. What we do is, once we have figured out that the bot has wrongly predicted the intent of some message, we put those messages under their correct intent in the training file and then simply retrain the bot.
Hi @eashan_27! Could you please elaborate on the automatic training part?
Preparing the labeled data for training is always the difficult part. In most cases we don’t have enough conversational data. So the recommended approach is to start with whatever data you have and then give your bot to real users and then collect data and improve your bot. It’s an iterative process.
Now for automating the training, you’ll still need the labeled data (in any format is fine). So if you are asking for automatically labeling the data then that is not possible. Although, if you already have the labeled data then that can be converted into rasa training format.
Have you tried Rasa X to review conversations, tag “intent_wrong” and any other tags you want to use and then update the training data? From there you can push to a git repo and run training as part of your CI/CD pipeline.
Hi Greg, you mean the manual approch? Where you can tag each message? Yes. But we need a way to use the external data, as there are a couple of thousand messages.
In Rasa X you can filter conversations by confidence level and many other filtering options to focus on longer conversations or on specific intents, actions, etc. We recently in the NLU inbox also by confidence level also.
I’m not sure what you mean by “external data”? Normally, you would have Rasa X connected to your production Rasa OSS instance to capture actual user messages.
