# Response selector Testing **URL:** \ **Category:** Rasa Open Source\ **Created:** [June 28, 2020, 11:40am UTC](https://forum.rasa.com/t/response-selector-testing/30751 "2020-06-28T11:40:30Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![DORpapst](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/dorpapst/32/9707_2.png) [@DORpapst](https://forum.rasa.com/u/DORpapst)\ **Post date:** [June 28, 2020, 11:40am UTC](https://forum.rasa.com/t/response-selector-testing/30751/1 "2020-06-28T11:40:30Z")
Hello Friends, I am into testing my Rasa chatbat and I would like to not just test if a specific code message leads to my response selector but also if it leads to specifically the correct intern response selector intent. Cannot find something about that in the documentation. I know that rasa does it automatically via rasa test for the predefined ones, but I would like to define own tests. If my faq thing looks like that: ## intent:faq/example1 - text1 - text2 - text3 ## intent:faq/example2 - text4 - text5 - text6 ## intent:faq/example3 - text7 - text8 - text9 And I am going to type in “test7b” I would like to test that it really leads to example3. My test file looks similar to that: ## A basic end-to-end test - greetings.greet: hello - utter\_greet - faq: test7b - respond\_faq where respond\_faq is the overall intent of all faq. How can I check it is example3 and not just any faq? Many greetings and thanks, Lukas (Sorry for my stupid example, I obscured the real texts (obviously)) ---
**Author:** ![Tanja](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/tanja/32/4696_2.png) [@Tanja](https://forum.rasa.com/u/Tanja)\ **Post date:** [June 29, 2020, 7:54am UTC](https://forum.rasa.com/t/response-selector-testing/30751/2 "2020-06-29T07:54:33Z")
Not sure if I fully understand the question, but will try my best to answer 🙂 In the latest Rasa Open Source version `rasa test` just produces a report after evaluating responses containing metrics like accuracy or f-score. In order to verify if example3 was predicted incorrect or correct you would need to have a file that contains the successful predictions or a file with the incorrect predictions. Then you could search for the example in the corresponding files. We currently do not generate those files. I recently worked on [Standardize testing output · Issue #5748 · RasaHQ/rasa · GitHub](https://github.com/RasaHQ/rasa/issues/5748) to add more information to the test output. We added the possibility to write down incorrect and correct predictions for response selection and we also generate some plots, such as the confusion matrix. I guess that is exactly what you need. The PR was merged into master. Unfortunately, we don’t have a date yet when the next version from master will be released. Does that help? ---
**Author:** ![DORpapst](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/dorpapst/32/9707_2.png) [@DORpapst](https://forum.rasa.com/u/DORpapst)\ **Post date:** [June 29, 2020, 8:33am UTC](https://forum.rasa.com/t/response-selector-testing/30751/3 "2020-06-29T08:33:09Z")
Hey Tanja, Thank you for your answer! Well, you are right, the question was quite complicated to understand, I am sorry. I think I personally misunderstood the rasa test command. My personal understanding was that I could implement a dictionary like: “text a user types in” - “what rasa should output” to test my response selector manually. This would also include text which never was put into the intention list. But the test command seems to just test those intents who already are inside the official list which one programmed. Which is also fine. I just thought that I could implement own test cases for test. Great that you started to implement some more functions, I am looking forward for the next release. Many greetings, Lukas ---
**Author:** ![Tanja](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/tanja/32/4696_2.png) [@Tanja](https://forum.rasa.com/u/Tanja)\ **Post date:** [June 29, 2020, 9:26am UTC](https://forum.rasa.com/t/response-selector-testing/30751/4 "2020-06-29T09:26:32Z")
If you want to test your response selector manually, you should be able to do so via `rasa shell`. Just chat with the assistant and see what he response. Response selector just knows about those intents that are inside the training data. So if you introduce a complete new intent during testing, it will most likely fail. The test data you provide when executing `rasa test` can and should be different from the training data. You can add your own test cases in there. Ideally, the test data contains examples that are not inside the training data. ---
**Author:** ![DORpapst](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/dorpapst/32/9707_2.png) [@DORpapst](https://forum.rasa.com/u/DORpapst)\ **Post date:** [June 29, 2020, 9:32am UTC](https://forum.rasa.com/t/response-selector-testing/30751/5 "2020-06-29T09:32:26Z")
Ah right, I really forgot about that. Thank you very much for reminding my. I will have a look or that. Have a wonderful day! ---
**Author:** ![lotcz](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/lotcz/32/10758_2.png) [@lotcz](https://forum.rasa.com/u/lotcz)\ **Post date:** [September 20, 2020, 8:36pm UTC](https://forum.rasa.com/t/response-selector-testing/30751/6 "2020-09-20T20:36:38Z")
I am wondering about the same thing. My tests only tell me that **faq** intent was classified as that and **respond\_faq** action followed, but not whether it was correctly classified as specific faq question. I am facing a problem where faqs are incorrectly classified and my tests still pass. Is there really not a solution for that?