# What is the standard practice for Rasa Testing?

**URL:** <https://forum.rasa.com/t/what-is-the-standard-practice-for-rasa-testing/33804>\
**Category:** Rasa Open Source\
**Created:** [September 16, 2020, 3:00am UTC](https://forum.rasa.com/t/what-is-the-standard-practice-for-rasa-testing/33804 "2020-09-16T03:00:00Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![student99](https://avatars.discourse-cdn.com/v4/letter/s/9e8a1a/32.png) [@student99](https://forum.rasa.com/u/student99)\
**Post date:** [September 16, 2020, 3:00am UTC](https://forum.rasa.com/t/what-is-the-standard-practice-for-rasa-testing/33804/1 "2020-09-16T03:00:00Z")

</div>

Hi All,

By convention how much stories should be tested using ‘Rasa Test’ command, all the data or just a small set. I have about 40 intents. Should I create stories that should fail in the end to end format? can I test fallback policy?

I can make all my stories test pass without changing nlu/core, with careful wording in my text in the end to end format is enough to pass, but surely there are other ways to give a realistic test? My chatbot is far from perfect but passes test with high accuracy, so perhaps i am not using good testing practice?

edit: I have also noticed if we are using a end2end format similar to this: ‘show me ‘[‘chinese’]’(cuisine)’ restaurants’ we already helping the chatbot extract the correct entity, sometime giving the correct entity is enough to predict a good NLU confidence, since most my entities are not reused for diff stories.

---

<div class="post-metadata">

**Author:** ![koaning](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/koaning/32/6626_2.png) [@koaning](https://forum.rasa.com/u/koaning)\
**Post date:** [September 18, 2020, 2:10pm UTC](https://forum.rasa.com/t/what-is-the-standard-practice-for-rasa-testing/33804/2 "2020-09-18T14:10:30Z")

</div>

You can also run `rasa test nlu` with a cross validation command. That way you can have each data point be represented as a test case once. Have you ever done this?

You might appreciate this benchmarking guide that I wrote for [rasa nlu examples](https://rasahq.github.io/rasa-nlu-examples/benchmarking/).

---

<div class="post-metadata">

**Author:** ![student99](https://avatars.discourse-cdn.com/v4/letter/s/9e8a1a/32.png) [@student99](https://forum.rasa.com/u/student99)\
**Post date:** [September 19, 2020, 12:04am UTC](https://forum.rasa.com/t/what-is-the-standard-practice-for-rasa-testing/33804/3 "2020-09-19T00:04:34Z")

</div>

Thanks I have ran the benchmark tests for my NLU. the latter test (cross validation) I get less accuracy compared to Rasa test nlu where it outputs 100%. What does the output result of cross validation suggest.

---

<div class="post-metadata">

**Author:** ![koaning](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/koaning/32/6626_2.png) [@koaning](https://forum.rasa.com/u/koaning)\
**Post date:** [September 22, 2020, 11:28am UTC](https://forum.rasa.com/t/what-is-the-standard-practice-for-rasa-testing/33804/4 "2020-09-22T11:28:10Z")

</div>

Could you share the commands that you ran? There’s settings that might cause such changes.

---

<div class="post-metadata">

**Author:** ![student99](https://avatars.discourse-cdn.com/v4/letter/s/9e8a1a/32.png) [@student99](https://forum.rasa.com/u/student99)\
**Post date:** [September 24, 2020, 1:36am UTC](https://forum.rasa.com/t/what-is-the-standard-practice-for-rasa-testing/33804/5 "2020-09-24T01:36:32Z")

</div>

I ran this following command: rasa test nlu --config basic-config.yml –cross-validation --runs 1 --folds 2 –out gridresults/basic-config