# Policy comparison mode doesn't generate models

**URL:** <https://forum.rasa.com/t/policy-comparison-mode-doesnt-generate-models/12728>\
**Category:** Rasa Open Source\
**Created:** [July 10, 2019, 7:55am UTC](https://forum.rasa.com/t/policy-comparison-mode-doesnt-generate-models/12728 "2019-07-10T07:55:00Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![guillecarc](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/guillecarc/32/2140_2.png) [@guillecarc](https://forum.rasa.com/u/guillecarc)\
**Post date:** [July 10, 2019, 7:55am UTC](https://forum.rasa.com/t/policy-comparison-mode-doesnt-generate-models/12728/1 "2019-07-10T07:55:00Z")

</div>

Following the documentation ([Evaluating Models](https://rasa.com/docs/rasa/user-guide/evaluating-models/#comparing-policies)) I ran the following command to compare the policies’ performance:

rasa train core -c config/keras.yml config/embbeding.yml -d domain.yml -s data/100/stories.md --out core\_comparison\_results/ --runs 3 --percentages 0 25 50 75 95 --augmentation 10

After this was complete, the documentation indicates to run a “test” command between the models created. Here it comes the blocker, after training the models, rasa did not create any model it just created folders for each run storing the parameters used.

---

<div class="post-metadata">

**Author:** ![Ghostvv](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/ghostvv/32/211_2.png) [@Ghostvv](https://forum.rasa.com/u/Ghostvv)\
**Post date:** [July 10, 2019, 8:51am UTC](https://forum.rasa.com/t/policy-comparison-mode-doesnt-generate-models/12728/2 "2019-07-10T08:51:01Z")

</div>

there are no trained models in `core_comparison_results` folder?

---

<div class="post-metadata">

**Author:** ![guillecarc](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/guillecarc/32/2140_2.png) [@guillecarc](https://forum.rasa.com/u/guillecarc)\
**Post date:** [July 10, 2019, 9:07am UTC](https://forum.rasa.com/t/policy-comparison-mode-doesnt-generate-models/12728/3 "2019-07-10T09:07:23Z")

</div>

This is how the result folder looks like:

 ![39](https://europe1.discourse-cdn.com/flex013/uploads/rasa/original/2X/c/c406279cafe25fd2795783414ccd73c0c564f86b.jpeg)

---

<div class="post-metadata">

**Author:** ![Ghostvv](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/ghostvv/32/211_2.png) [@Ghostvv](https://forum.rasa.com/u/Ghostvv)\
**Post date:** [July 10, 2019, 11:54am UTC](https://forum.rasa.com/t/policy-comparison-mode-doesnt-generate-models/12728/4 "2019-07-10T11:54:07Z")

</div>

yep, that are the models, but they should be compressed into `.tar.gz` files. What version of rasa are you using?

---

<div class="post-metadata">

**Author:** ![guillecarc](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/guillecarc/32/2140_2.png) [@guillecarc](https://forum.rasa.com/u/guillecarc)\
**Post date:** [July 10, 2019, 12:07pm UTC](https://forum.rasa.com/t/policy-comparison-mode-doesnt-generate-models/12728/5 "2019-07-10T12:07:27Z")

</div>

1.1.4

---

<div class="post-metadata">

**Author:** ![Ghostvv](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/ghostvv/32/211_2.png) [@Ghostvv](https://forum.rasa.com/u/Ghostvv)\
**Post date:** [July 10, 2019, 2:01pm UTC](https://forum.rasa.com/t/policy-comparison-mode-doesnt-generate-models/12728/6 "2019-07-10T14:01:58Z")

</div>

there is actually a bug, the folders inside should be compressed, could you please create an issue?

To use it right now, you can train 1 model, then look what is inside `.tar.gz` model file, then by analogy compress the files in your screenshot

---

<div class="post-metadata">

**Author:** ![guillecarc](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/guillecarc/32/2140_2.png) [@guillecarc](https://forum.rasa.com/u/guillecarc)\
**Post date:** [July 10, 2019, 3:26pm UTC](https://forum.rasa.com/t/policy-comparison-mode-doesnt-generate-models/12728/7 "2019-07-10T15:26:09Z")

</div>

Thanks, i will 🙂