# Evaluation/Test stories format questions

**URL:** <https://forum.rasa.com/t/evaluation-test-stories-format-questions/6171>\
**Category:** Rasa Open Source\
**Created:** [March 19, 2019, 9:57pm UTC](https://forum.rasa.com/t/evaluation-test-stories-format-questions/6171 "2019-03-19T21:57:25Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![MaZe](https://avatars.discourse-cdn.com/v4/letter/m/e9a140/32.png) [@MaZe](https://forum.rasa.com/u/MaZe)\
**Post date:** [March 19, 2019, 9:57pm UTC](https://forum.rasa.com/t/evaluation-test-stories-format-questions/6171/1 "2019-03-19T21:57:25Z")

</div>

Hello,

While creating test scripts for the bot we ran across multiple bizarre behaviors. We follow the interactive training to generate scripts (stories), to make sure that the stories are in a good format for Rasa. But when we perform end to end evaluation we noticed that the bot predicts some very strange stories.

For example:

- request\_quote: I would like to get a quote
  - utter\_quote\_form
  - quote\_form
  - form{“name”: “quote\_form”}
  - slot{“requested\_slot”: “know\_product\_type”}

- form: ask\_whyneedinfo: Why would you need to know this?
  - form: quote\_form
  - form: action\_listen
  - form: quote\_form
  - form: quote\_form

The idea is to handle uncooperative user’s behavior and we have a specific action that gets call when the intent (ask\_whyneedinfo) is detected. But in the test story format, adding this action would result in a failed test. And as you can see quote\_form (the form action), gets called three times, following the user message in this evaluation script. Also, action\_listen as the second action makes very little sense, as generally action\_listen should be the last action before user message, and we have no instance of action

The issue we have is that, while this test goes through with no issues, we do not understand why the correct story is not the following:

- request\_quote: I would like to get a quote
  - utter\_quote\_form
  - quote\_form
  - form{“name”: “quote\_form”}
  - slot{“requested\_slot”: “know\_product\_type”}

- form: ask\_whyneedinfo: Why would you need to know this?
  - form: quote\_form
  - action\_whyneedinfo
  - quote\_form
  - action\_listen

Which would make more sense. If anyone has experience with how the evaluation stories are/should be written we would appreciate it.

Thanks.

---

<div class="post-metadata">

**Author:** ![Remy](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/remy/32/2030_2.png) [@Remy](https://forum.rasa.com/u/Remy)\
**Post date:** [March 20, 2019, 9:38am UTC](https://forum.rasa.com/t/evaluation-test-stories-format-questions/6171/2 "2019-03-20T09:38:49Z")

</div>

You could try placing the action right below the ask\_whyneedinfo intent, so it gets called first, before the form. Now it looks like it messes up the flow of the conversation. The correct story would then be

form: ask\_whyneedinfo: Why would you need to know this?

- action\_whyneedinfo
- form: quote\_form
- quote\_form
- action\_listen [/quote]