Why is the test set trained for evaluation purpose?

Rasa Open Source
1

Hello,

I am trying to evaluate model using test data set:

python -m rasa_nlu.evaluate --data Test_dataset.json --model nlu_model

Test set has around 20% of samples compared with a train set. For example, if the train set has 6 samples per intent, then the train set will have only 1.

In this case, I got a warning:

\rasa_nlu\training_data\training_data.py:1 91: UserWarning: Intent 'TestIntent' has only 1 training examples!
Minimum is 2, training may fail.

Why is the test set trained for evaluation purpose if the training has already been performed on the training dataset?

2

thanks for noticing it, it shouldn’t emit this warning during evaluation. It happens, because it uses the same script to create training data object from the file. You can ignore this warning during evaluation

Could you please create GitHub issue?

3

Thank you! Created GitHub issue