# What really happens during \`rasa test\`

**URL:** <https://forum.rasa.com/t/what-really-happens-during-rasa-test/55293>\
**Category:** Rasa Open Source\
**Tags:** testing\
**Created:** [September 13, 2022, 10:06am UTC](https://forum.rasa.com/t/what-really-happens-during-rasa-test/55293 "2022-09-13T10:06:35Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Rafal](https://avatars.discourse-cdn.com/v4/letter/r/ac91a4/32.png) [@Rafal](https://forum.rasa.com/u/Rafal)\
**Post date:** [September 13, 2022, 10:06am UTC](https://forum.rasa.com/t/what-really-happens-during-rasa-test/55293/1 "2022-09-13T10:06:35Z")

</div>

I have an intent named `nlu_fallback` with examples of trash messages inside the `nlu.yml` file.

I want it to be predicted when there is low intent confidence fallback as well as when users write messages irrelevant to the story flow. The bot works fine, and when I enter some gibberish messages rasa interactive perfectly predicts `nlu_fallback` intent. The problem is when I want to evaluate the bot using `rasa test nlu --cross-validation --folds 3` the bot hardly ever predicts the nlu fallback, instead, it predicts using other intents even if their confidence is ~0.0005. I’ve found in `rasa/nlu/test.py` the following code snippet

```auto
async def get_eval_data(
    processor: MessageProcessor, test_data: TrainingData
) -> Tuple[
    List[IntentEvaluationResult],
    List[ResponseSelectionEvaluationResult],
    List[EntityEvaluationResult],
]:

<some code>

for example in tqdm(test_data.nlu_examples):
        result = await processor.parse_message(
            UserMessage(text=example.get(TEXT)), only_output_properties=False
        )
        _remove_entities_of_extractors(result, PRETRAINED_EXTRACTORS)
        if should_eval_intents:
            if fallback_classifier.is_fallback_classifier_prediction(result):
                # Revert fallback prediction to not shadow
                # the wrongly predicted intent
                # during the test phase.
                result = fallback_classifier.undo_fallback_prediction(result)
            intent_prediction = result.get(INTENT, {})
            intent_results.append(
                IntentEvaluationResult(
                    example.get(INTENT, ""),
                    intent_prediction.get(INTENT_NAME_KEY),
                    result.get(TEXT),
                    intent_prediction.get("confidence"),
                )
            )

<some_code>

```

Could somebody explain to me, why the fallback prediction needs to be undone? I do not really understand this comment. Do I need to change my intent name to sth similar to `out-of-scope`?

Thanks for your help in advance

Cheers