End to end testing runs nlu test on training data

Rasa Open Source
1

When I run the command “rasa test”, rasa runs and end to end test for the core part and prints out the results, but for the nlu part, it prints the results performed over the training data (data/nlu.md). Is it normal? How can I get a complete test over the end to end files? If I open the “DIETClassifier_errors.json” file there are reported sentences only present in the training data.

Rasa version: Rasa 1.10.10

2

Not exactly sure what you mean. There is no end to end test for the NLU data. This is just possible with the Core data.

How can I get a complete test over the end to end files?

Can you specify what exactly you mean by this? How does your data look like and what would you expect?

3

Ok thanks, I didn’t get that the NLU part cannot be tested on the end2end stories. I think that could be useful the possibility to save in a JSON file the core test results. Now it’s printed only in console

4

Can you maybe share an example or a screenshot? Still not 100% sure what you mean.

5

Previously, I thought that running rasa test would be performed an end2end evaluation for both the NLU and CORE parts. For example for the NLU part just using the sentences present in the end2end stories, but now I understand that is not necessary since the CORE part evaluates both entities and intents present in the end2end stories. Anyway, I think that a report file for the end2end evaluation would be useful.