Rasa-X Interactive-Learning stops getting users annotations when NLU is too loaded

Hi,
We’re working with RasaX (with CustomActions) and use the Interactive-Learning scheme for tuning the NLU performance.
Everything appears to work just fine, until at some point, the NLU stops receiving user’s annotations.
For example, that means the nothing happens when users click on the “Correct” button for some intent classification.
This issue holds us from continue using RasaX in Interactive-Learning scheme.
Is that a known issue? Can you please support us?
Thanks ahead,
Newt Team

1 Like

Hi @sgino209, I haevn’t seen that issue yet. When you say until at some point do you mean during the workflow of annotating multiple conversations in IL? Or after some number of messages in a long conversation?

Which version of Rasa X are you running?

Hi Ella,
Thanks for your quick response.
We are running Rasa 1.10.6.
Yes, by “some point” we mean that interactive-learning annotation works well (=RasaX gets our annotations properly), until RasaX start to ignore our annotations.
For example, clicking on “correct” does not dismiss the corresponding annotation.
Thanks,
Newt Team

Thanks, let me see if I can get that to happen on my end :slightly_smiling_face: Which version of Rasa X with your rasa 1.10.6 though? You can find it at http://<rasa x url>/api/version.

I was able to get it to happen for me once, but not a second time. I’m reporting it and we’ll take a look into it – if you have any idea of a specific action causes it to go from reponsive to unresponsive, or have a screen recording of this happening during your workflow, that’d be a helpful addition!

Cheers :slight_smile:

Hi Ella,
Very well, thanks for your reply.
We couldn’t find a deterministic sequence which leads for that faulty behaviour as well.
We’ll report back with specific reproduction commands, once we’ll imply such.
Hope you’ll be able to catch it before that.
Thanks,
Newt Team

1 Like