I want to try out different approaches in labeling entities within the same intent. For this I want to create 2 nlu files, one per approach, train two models and then compare the performance of them. The first approach uses a lot of entities, the second one tries to summarize the needed data info few entities.
I saw that with
rasa test nlu you can get some nice confusion matrices, to analyse which entities are confused with each other. However I need to know if the whole sentence with multiple entities is labeled correctly. If yes, this sentence should be counted as successful and in the end I have a result that may look like this: “597 out of 1000 sentences are labeled with the correct entities.”
Is there a way to do this in Rasa? Or is there an easy way to accomplish this result? Thank you for any suggestions!