# Test scores for failed intents

**URL:** <https://forum.rasa.com/t/test-scores-for-failed-intents/45907>\
**Category:** Rasa Open Source\
**Created:** [July 19, 2021, 4:07pm UTC](https://forum.rasa.com/t/test-scores-for-failed-intents/45907 "2021-07-19T16:07:57Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![laboratory](https://avatars.discourse-cdn.com/v4/letter/l/df705f/32.png) [@laboratory](https://forum.rasa.com/u/laboratory)\
**Post date:** [July 19, 2021, 4:07pm UTC](https://forum.rasa.com/t/test-scores-for-failed-intents/45907/1 "2021-07-19T16:07:57Z")

</div>

When I test my stories in `test_stories.yml`, is there a way we can see the probabilities returned for each test case (especially the failed tests)?

In `failed_test_stories.yml` I just see something like

```auto
- intent: greet # predicted nlu_fallback: hello there!

```

Can I also see the probability that was returned so I know in what direction to work the model? Does this exist already?

---

<div class="post-metadata">

**Author:** ![jupyterjazz](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/jupyterjazz/32/14477_2.png) [@jupyterjazz](https://forum.rasa.com/u/jupyterjazz)\
**Post date:** [July 26, 2021, 11:25am UTC](https://forum.rasa.com/t/test-scores-for-failed-intents/45907/2 "2021-07-26T11:25:38Z")

</div>

you should be able to see them in **intent\_errors.json** file.

---

<div class="post-metadata">

**Author:** ![laboratory](https://avatars.discourse-cdn.com/v4/letter/l/df705f/32.png) [@laboratory](https://forum.rasa.com/u/laboratory)\
**Post date:** [July 26, 2021, 11:59am UTC](https://forum.rasa.com/t/test-scores-for-failed-intents/45907/3 "2021-07-26T11:59:38Z")

</div>

@jupyterjazz .Thanks. What I see in **intent\_errors.json** is something like

```auto
{
    "text": "cu",
    "intent": "sendoff",
    "intent_prediction": {
      "name": "affirm",
      "confidence": 0.998573899269104
    }

```

However, the texts are not the exact text in my user text in **test\_stories.yml**. so I am wondering where the text in the above is coming from. Is something wrong? This happens for many other cases as shown in **intent\_errors.json**.

---

<div class="post-metadata">

**Author:** ![jupyterjazz](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/jupyterjazz/32/14477_2.png) [@jupyterjazz](https://forum.rasa.com/u/jupyterjazz)\
**Post date:** [July 26, 2021, 12:23pm UTC](https://forum.rasa.com/t/test-scores-for-failed-intents/45907/4 "2021-07-26T12:23:20Z")

</div>

You’re right it won’t show failed intents from the test stories in **intent\_errors.json** , my bad.

I’m not sure if it’s possible to see the probabilities for the failed stories. However, one thing you could do is run [`rasa shell nlu`](https://rasa.com/docs/rasa/command-line-interface/#rasa-shell) which would give you an opportunity to test how your model predicts intents. Since NLU is stateless you can ignore the context and only check what it predicts on “hello there”.

---

<div class="post-metadata">

**Author:** ![laboratory](https://avatars.discourse-cdn.com/v4/letter/l/df705f/32.png) [@laboratory](https://forum.rasa.com/u/laboratory)\
**Post date:** [July 26, 2021, 12:37pm UTC](https://forum.rasa.com/t/test-scores-for-failed-intents/45907/5 "2021-07-26T12:37:41Z")

</div>

@jupyterjazz . Yea I know of `rasa shell nlu` and I use it. It’s only if you have 100 test stories, one cannot try for all of them.

Thanks.