Annotating NLU in Rasa X in CDD

[Deprecated] Rasa X Community Edition
1

In CDD, when annotating, I’d liked to tag NLU data in Rasa X in NLU inbox.

Of course, I can tag them in the Conversations tab.
But I experienced, that tagging NLU apart from Conversations is also usefull.
It would even be better if filtering on NLU is possible, either on Conversations tab or in NLU inbox.

This helps me to:

  1. Tag NLU to be discussed with others (what kind of intent, is it a intent we want to add, wat does this mean for other data in the traing set)
  2. Find already examined but not yet decided on NLU examples.
  3. Cluster NLU-training data before deciding upon it

Some examples to illustrate the need for at least two-step examination, i.e. roughly and discussion:

image
image1107×621 21 KB
Its not necessary to understand the Dutch :slightly_smiling_face:: just want to show some intents, I want to tag to either discuss with collaeges later on or sleep a few nights on it.

Question and suggestion
Wonder what possibility Rasa X already offers, that I’ve probably missed.
And, if not already available, annotating NLU data could be an valuable new functionallity.

2

Thanks for the suggestion!

Unfortunately we only have tagging available in conversations at the moment, but improving some of the tasks you mention is something we’re looking at. I’ll share your suggestions with the team, but it would also help if you had time to answer some questions:

  • You mention filtering in the NLU, what kind of things would you like to be able to do?
  • You mention tagging for discussion with other colleagues, how many of you are there and how do you discuss/decide on changes to NLU training data?
3

  1. Filtering on:

    1. NLU not available in training data
    2. Intents (this one is already available in Rasa X)
    3. Intent confidence
    4. Tags
    5. Entities
    6. Entity confidence
    7. Review status

  2. Discussing with colleaques needs som explanation.
    At the moment I am the only one using Rasa and Rasa X, exploring Rasa and chat and feeding my organisation with my experiences. I’ve exposed the chatBot to some colleqaues as to get some real conversations. Exposion through Mattermost and testuser channel.
    I’m waiting for my organisation to decide on starting with a real Rasa project. I expect by then, that at least three DevOps and two or more SME’s, both domain and communication/chat experts, are collaborating in a scrum-team

The tags and filters then can be used in two ways:

  1. First reviewer and second reviewer or approver, all using Rasa X.
  2. A meeting in a reference group, where one of the scrum-team members leads the discussion, based on the filtered tags
4

Thanks! This is helpful information. This fits reasonably well with the idea of tags for NLU that we’re working on at the moment.

1 Like