[rasa 2][How to] get template confidence for each faq responses?

Hey, there

I have set several responses as below in responses.yml

responses:
  utter_ask_faq/help:
  - text: help
  utter_ask_faq/who_is_builder:
  - text: the hydro center
  utter_ask_faq/who_are_you:
  - text: amy
  utter_ask_faq/bot_challenge:
  - text: yeah, I'm a bot
  utter_ask_faq/human_handoff:
  - text: please call ***-***-*****

After asking ‘who is your builder?’, I get the pre-defined answer. However, if I want to get the confidence of each response template, what should I do?

As below, rasa found the best hit “utter_ask_faq/who_is_builder” correctly. What if I want to print out the top 3 hits for users and let them choose from them, in case the best hit is not what user wants.

2020-10-19 10:14:53 DEBUG    rasa.core.processor  - Action 'utter_ask_faq' ended with events '[BotUttered('the hydro center', {"elements": null, "quick_replies": null, "buttons": null, "attachment": null, "image": null, "custom": null}, {"template_name": "utter_ask_faq/who_is_builder"}, 1603073693.2386665)]'.

The whole log of this query is as below:

2020-10-19 10:14:53 DEBUG    rasa.core.lock_store  - Issuing ticket for conversation '888dbf6acd8d68ccaf5d607e6e886eeb'.
2020-10-19 10:14:53 DEBUG    rasa.core.lock_store  - Acquiring lock for conversation '888dbf6acd8d68ccaf5d607e6e886eeb'.
2020-10-19 10:14:53 DEBUG    rasa.core.lock_store  - Acquired lock for conversation '888dbf6acd8d68ccaf5d607e6e886eeb'.
2020-10-19 10:14:53 DEBUG    rasa.nlu.selectors.response_selector  - Adding following selector key to message property: ask_faq
2020-10-19 10:14:53 DEBUG    rasa.core.processor  - Received user message 'who is your builder?' with intent '{'id': 4570953846918902020, 'name': 'ask_faq', 'confidence': 0.999888002872467}' and entities '[]'
2020-10-19 10:14:53 DEBUG    rasa.core.processor  - Logged UserUtterance - tracker now has 9 events.
2020-10-19 10:14:53 DEBUG    rasa.core.policies.memoization  - Current tracker state [{}, {'user': {'intent': 'ask_faq'}, 'prev_action': {'action_name': 'action_listen'}}]
2020-10-19 10:14:53 DEBUG    rasa.core.policies.memoization  - There is a memorised next action 'utter_ask_faq'
2020-10-19 10:14:53 DEBUG    rasa.core.policies.rule_policy  - Current tracker state: [{}, {'user': {'intent': 'ask_faq'}, 'prev_action': {'action_name': 'action_listen'}}]
2020-10-19 10:14:53 DEBUG    rasa.core.policies.rule_policy  - There is a rule for the next action 'utter_ask_faq'.
2020-10-19 10:14:53 DEBUG    rasa.core.policies.ensemble  - Predicted next action using policy_2_RulePolicy
2020-10-19 10:14:53 DEBUG    rasa.core.processor  - Predicted next action 'utter_ask_faq' with confidence 1.00.
2020-10-19 10:14:53 DEBUG    rasa.core.actions.action  - Picking response from selector of type ask_faq
2020-10-19 10:14:53 DEBUG    rasa.core.processor  - Action 'utter_ask_faq' ended with events '[BotUttered('the hydro center', {"elements": null, "quick_replies": null, "buttons": null, "attachment": null, "image": null, "custom": null}, {"template_name": "utter_ask_faq/who_is_builder"}, 1603073693.2386665)]'.
2020-10-19 10:14:53 DEBUG    rasa.core.policies.memoization  - Current tracker state [{}, {'user': {'intent': 'ask_faq'}, 'prev_action': {'action_name': 'action_listen'}}, {'user': {'intent': 'ask_faq'}, 'prev_action': {'action_name': 'utter_ask_faq'}}]
2020-10-19 10:14:53 DEBUG    rasa.core.policies.memoization  - There is a memorised next action 'action_listen'
2020-10-19 10:14:53 DEBUG    rasa.core.policies.rule_policy  - Current tracker state: [{}, {'user': {'intent': 'ask_faq'}, 'prev_action': {'action_name': 'action_listen'}}, {'user': {'intent': 'ask_faq'}, 'prev_action': {'action_name': 'utter_ask_faq'}}]
2020-10-19 10:14:53 DEBUG    rasa.core.policies.rule_policy  - There is a rule for the next action 'action_listen'.
2020-10-19 10:14:53 DEBUG    rasa.core.policies.ensemble  - Predicted next action using policy_2_RulePolicy
2020-10-19 10:14:53 DEBUG    rasa.core.processor  - Predicted next action 'action_listen' with confidence 1.00.
2020-10-19 10:14:53 DEBUG    rasa.core.processor  - Action 'action_listen' ended with events '[]'.
2020-10-19 10:14:53 DEBUG    rasa.core.lock_store  - Deleted lock for conversation '888dbf6acd8d68ccaf5d607e6e886eeb'.