Rasa nlu test starts training instead of test

Rasa Open Source
1

When I try to test my trained model apparently it starts to train all over again.

rasa on  master [✘!?] via .env at ☸️  docker-desktop 
➜ rasa test nlu -u train_test_split/test_data.json --model models/nlu-20200708-110610.tar.gz --cross-validation --confmat CONFMAT
2020-07-08 11:59:37 INFO     rasa.cli.test  - Test model using cross validation.
2020-07-08 11:59:37 INFO     rasa.nlu.utils.spacy_utils  - Trying to load spacy model with name 'pt_core_news_sm'
2020-07-08 11:59:42 INFO     rasa.nlu.components  - Added 'SpacyNLP' to component cache. Key 'SpacyNLP-pt_core_news_sm'.
/Users/****/Desktop/dev/rasa/.env/lib/python3.7/site-packages/rasa/utils/common.py:363: UserWarning: Entity entity '****' has only 1 training examples! The minimum is 2, because of this the training may fail.
2020-07-08 11:59:43 INFO     rasa.nlu.model  - Starting to train component SpacyNLP
2020-07-08 11:59:44 INFO     rasa.nlu.model  - Finished training component.

What I did:

  • splitted my train data into train and test data using rasa data split

obs: my train data is formatted as json

anyone knows why that happens?

edit1: after trying with my data in the markdown format, i got this following error:

**rasa** **on** ** master** **[✘!?]** **via** **.env** **at** **☸️ docker-desktop** **took** **10s**  

**➜** rasa test -u train_test_split/test_data.md --cross-validation --confmat CONFMAT --config config.yml

Unable to test: could not find a Core model. Use 'rasa train' to train a Rasa model and provide it via the '--model' argument.

2020-07-08 13:06:40 **INFO** rasa.nlu.components - Added 'SpacyNLP' to component cache. Key 'SpacyNLP-pt_core_news_sm'.

Processed Story Blocks: 0it [00:00, ?it/s]

/Users/diegosilva/Desktop/dev/rasa/.env/lib/python3.7/site-packages/rasa/utils/common.py:363: UserWarning: There is no starting story block in the training data. All your story blocks start with some checkpoint. There should be at least one story block that starts without any checkpoint.

More info at https://rasa.com/docs/rasa/core/stories/#stories

2020-07-08 13:06:42 **INFO** rasa.core.test - Evaluating 0 stories

Progress:

0it [00:00, ?it/s]

2020-07-08 13:06:42 **INFO** rasa.core.test - Finished collecting predictions.

2020-07-08 13:06:42 **WARNING**  rasa.nlu.test - No labels to evaluate. Skip evaluation.

Traceback (most recent call last):

File "/Users/diegosilva/Desktop/dev/rasa/.env/bin/rasa", line 10, in <module>

sys.exit(main())

File "/Users/diegosilva/Desktop/dev/rasa/.env/lib/python3.7/site-packages/rasa/__main__.py", line 92, in main

cmdline_arguments.func(cmdline_arguments)

File "/Users/diegosilva/Desktop/dev/rasa/.env/lib/python3.7/site-packages/rasa/cli/test.py", line 159, in test

run_core_test(args)

File "/Users/diegosilva/Desktop/dev/rasa/.env/lib/python3.7/site-packages/rasa/cli/test.py", line 91, in run_core_test

additional_arguments=vars(args),

File "/Users/diegosilva/Desktop/dev/rasa/.env/lib/python3.7/site-packages/rasa/test.py", line 140, in test_core

rasa.core.test(stories, _agent, out_directory=output, **kwargs)

File "uvloop/loop.pyx", line 1456, in uvloop.loop.Loop.run_until_complete

File "/Users/diegosilva/Desktop/dev/rasa/.env/lib/python3.7/site-packages/rasa/core/test.py", line 546, in test

completed_trackers, agent, fail_on_prediction_errors, e2e

File "/Users/diegosilva/Desktop/dev/rasa/.env/lib/python3.7/site-packages/rasa/core/test.py", line 492, in collect_story_predictions

in_training_data_fraction = _in_training_data_fraction(action_list)

File "/Users/diegosilva/Desktop/dev/rasa/.env/lib/python3.7/site-packages/rasa/core/test.py", line 445, in _in_training_data_fraction

return len(in_training_data) / len(action_list)

ZeroDivisionError: division by zero
2

I’ve managed to evaluate. I thought I could evaluate with the data in the json format but I really must have stories in order to do that.

3

Hello, nice that you found a solution, I am trying to achieve a similar objective, basically I would like to split data into train/test, train calculate F1 score on both training and test, and possibly plot the learning curves. I appreciate it if you share some of your code, if possible, that I can use a start.

4

The super class of ZeroDivisionError is ArithmeticError. This exception raised when the second argument of a division or modulo operation is zero. In Mathematics, when a number is divided by a zero, the result is an infinite number. It is impossible to write an Infinite number physically. Python interpreter throws “ZeroDivisionError: division by zero” error if the result is infinite number. While implementing any program logic and there is division operation make sure always handle ArithmeticError or ZeroDivisionError so that program will not terminate.

try:
    z = x / y
except ZeroDivisionError:
    z = 0

Or check before you do the division:

if y == 0:
    z = 0
else:
    z = x / y

The latter can be reduced to:

z = 0 if y == 0 else (x / y)