Confusion matrix entities

If I am well, Rasa doesn’t have a confusion matrix for entity evaluation / visualizing.

Of course I can make one myself, or find a suitable one on the web.

But just wondering: why did Rasa decide to not have incorparated a entity confusion matrix?

Rasa 1.10.3. I know there are newer Rasa versions, but they don’t seem to have a entity confusion matrix either.

hi Herman - thanks for the question! You’re right we don’t have one. How did you imagine this looking? I wouldn’t normally think of entity recognition as a multiclass classification problem, so there’s a bit of freedom in how you visualize this

Hi Alan,

I just thought about it, because Rasa entity-errors report (f.e. DIETClassifier_errors.json) can be quite overwhelming in its info, especially when there are a lot of errors. Visualization could help, to focus on severe confusions. And indeed, because of all the possible combinations, there comes the freedom in.

Whenfantasizing a little, maybe it could look like these examples.

entities image

values-entities image

To be honest, I’d already worked a little bit on it. But for now, my experience and programming skills seem to be insufficient to get it as I want it to be.

Once again, I just ran into it. But I guess you’ve got higher priorities at Rasa. So no need to spend a lot of time on it.

Nevertheless: thanks for your respond.