Policy comparison mode doesn't generate models

Following the documentation (Evaluating Models) I ran the following command to compare the policies’ performance:

rasa train core -c config/keras.yml config/embbeding.yml -d domain.yml -s data/100/stories.md --out core_comparison_results/ --runs 3 --percentages 0 25 50 75 95 --augmentation 10

After this was complete, the documentation indicates to run a “test” command between the models created. Here it comes the blocker, after training the models, rasa did not create any model it just created folders for each run storing the parameters used.

there are no trained models in core_comparison_results folder?

This is how the result folder looks like: 39

yep, that are the models, but they should be compressed into .tar.gz files. What version of rasa are you using?

1.1.4

there is actually a bug, the folders inside should be compressed, could you please create an issue?

To use it right now, you can train 1 model, then look what is inside .tar.gz model file, then by analogy compress the files in your screenshot

Thanks, i will :slight_smile: