Error when comparing NLU models via evaluation due to retrieval action

Rasa Open Source
1

Hello guys, I am trying to run some simple evaluation tests comparing different pipelines. Running the command as so: rasa test nlu --config pretrained_embeddings_spacy.yml supervised_embeddings.yml --nlu data/nlu.md --runs 3 --percentages 0 25 50 70 90

Gives me the error: No response phase found for faq/ask_data. Check training files for a possible wrong intent name.

Now I am using the new Retrieval Actions feature for my faqs, which is why the intent being flagged has the name as faq/ask_data. The feature is correctly implemented as far as I know, whith the responses being in a seperate file “responses.md”. My guess for this error would be that it doesnt find those responses, because only the nlu.md file is specified as training data?

Anyone got any input on this? Is this feature not usable with retrieval actions yet? By the way doing individual evaluations with the command: rasa test nlu -u data/nlu.md --config config.yml --cross-validation works fine.

Thanks, enjoy your weekend :slight_smile:

2

Yes, this was broken and we have taken a note of it here - NLG responses not found during NLU pipeline comparison · Issue #5201 · RasaHQ/rasa · GitHub We are working on a fix Bugfix nlu comparison persist/load nlg responses by melindaloubser1 · Pull Request #5202 · RasaHQ/rasa · GitHub which should be merged in one of the next couple of patch releases soon.

3

Rasa team

I am facing the exact problem with Rasa 1.10.16. Has the fix been merged? Thank you.