Determine entity confidence before filling slot

Rasa Open Source
1

Question:

How can I access the confidence of previously detected entities before deciding if I want to fill a slot or not?

Following Use Case:

The user has a problem with a product. The chatbot starts a conversation and after some messages, we decide that we want to report an incident and start a form. Now we want to prefill our form with as much information as possible from the previous conversation. However, we only want automatically fill the slots if we are pretty sure that the item was correctly recognized, otherwise we ask the user to confirm.

Now I have a custom action to validate all form slots, what would be the best way to access the entity confidence for that slot? The tracker.latest_message only returns the values for the last message and tracker.latest_entity_values only returns the values without any confidence values.

I am thinking about something like that in my custom action:

    async def validate_os_version(
            self, slot_value: Any, dispatcher: CollectingDispatcher, tracker: Tracker, domain: DomainDict
    ) -> Dict[Text, Any]:

        # Pseudo Code
        #
        # confidence = tracker.get_entity_confidence("os_version")
        # if confidence < 0.6:
        #    dispatcher.utter_message("Hm, I am not sure about your OS Version. Is slot_value really correct?
        #    return {"os_version": None}

        return {"os_version": slot_value }
2

If you want to save the confidence of an entity extraction from more than one message ago, you probably need to save it in a slot (or change the Rasa source code)

3

Really, interesting… Can you elaborate on how you would save the entity confidence in a slot when it gets extracted? Is there some method or callback I can hook into on my action server?

Otherwise, I am looking at the tracker and its events. I seems possible to iterate over all user events and check in the parse_data field for entities and their confidence. However, it feels a bit hacky as right now I am not sure how to connect the correct event to the filled slot.

Will a slot always be filled from the last detected entity in the conversation or from the first (if the slot is not declared a list)?

I just felt like my use case is not so uncommon and was wondering if there is a recommended approach.