Understanding Fallback Mechanics in Rasa - Need Help with Configuration and Action Handling

Hello,

I have been trying to understand how Rasa works for some time, and one of my major concerns revolves around the use of Fallback. Here is my configuration file:

recipe: default.v1


assistant_id: 20240322-120114-formal-biography

language: fr
pipeline:
  - name: WhitespaceTokenizer
  - name: LexicalSyntacticFeaturizer
  - name: CountVectorsFeaturizer
  - name: CountVectorsFeaturizer
    analyzer: char_wb
    min_ngram: 1
    max_ngram: 4
  - name: DIETClassifier
    epochs: 20
    constrain_similarities: true
  - name: FallbackClassifier
    threshold: 0.4
    fallback_action: "action_handle_intent_and_show_confidences"
    ambiguity_threshold: 0.1

policies:
  - name: MemoizationPolicy
  - name: RulePolicy
    core_fallback_threshold: 0.4
    core_fallback_action_name: action_handle_intent_and_show_confidences
    enable_fallback_prediction: true
  - name: TEDPolicy
    max_history: 5
    epochs: 100
    constrain_similarities: true

And here is my action: action_handle_intent_and_show_confidences

from typing import Any, Text, Dict, List
from rasa_sdk import Action, Tracker
from rasa_sdk.executor import CollectingDispatcher
from rasa_sdk.events import FollowupAction

from typing import Any, Text, Dict, List
from rasa_sdk import Action, Tracker
from rasa_sdk.executor import CollectingDispatcher
from rasa_sdk.events import FollowupAction

class ActionHandleIntentAndShowConfidences(Action):
    def name(self) -> Text:
        return "action_handle_intent_and_show_confidences"

    def run(self, dispatcher: CollectingDispatcher,
            tracker: Tracker,
            domain: Dict[Text, Any]) -> List[Dict[Text, Any]]:

        intent_ranking = tracker.latest_message.get('intent_ranking', [])
        entities = tracker.latest_message.get('entities', [])
        entities_str = ",".join([f'"{entity["entity"]}":"{entity["value"]}"' for entity in entities])
        entities_payload = "{" + entities_str + "}"

        message_confidences = "Voici les scores d'affiliation avec mes intents pour votre dernier message :\n"
        for intent in intent_ranking:
            intent_name = intent['name']
            confidence = intent['confidence']
            message_confidences += f"- {intent_name}: {confidence*100:.2f}%\n"

        dispatcher.utter_message(text=message_confidences)

        if intent_ranking:
            top_intent_confidence = intent_ranking[0]['confidence']
            ambiguity_threshold = top_intent_confidence - 0.1
            ambiguous_intents = [intent for intent in intent_ranking if intent['confidence'] >= ambiguity_threshold]

            if len(ambiguous_intents) > 1:
                buttons = []
                for intent in ambiguous_intents:
                    if intent['name'] != "nlu_fallback":
                        buttons.append({
                            "title": f"Vouliez-vous dire : {intent['name']} ?",
                            "payload": f"/{intent['name']}{entities_payload}"
                        })

                dispatcher.utter_message(text="Je ne suis pas sûr de comprendre votre demande. Vouliez-vous dire :", buttons=buttons)
            else:
                dispatcher.utter_message(text="Je ne suis pas tout à fait sûr de comprendre. Pouvez-vous reformuler ?")

        return [FollowupAction("action_listen")]

I would like to understand why, in some cases, I enter the fallback but my action returns a response indicating that the match is 100%.