# Attempting to handle confidence within a range with conditional rule and customized FallbackClassifier
**URL:** https://forum.rasa.com/t/attempting-to-handle-confidence-within-a-range-with-conditional-rule-and-customized-fallbackclassifier/40070
**Category:** Rasa Open Source
**Created:** [February 11, 2021, 3:06am UTC](https://forum.rasa.com/t/attempting-to-handle-confidence-within-a-range-with-conditional-rule-and-customized-fallbackclassifier/40070 "2021-02-11T03:06:58Z")
**Posts on this page:** 7
**Page:** 1
### Author:  [@zjntmo](https://forum.rasa.com/u/zjntmo)
#### Post date: [February 11, 2021, 3:06am UTC](https://forum.rasa.com/t/attempting-to-handle-confidence-within-a-range-with-conditional-rule-and-customized-fallbackclassifier/40070/1 "2021-02-11T03:06:58Z")
**Goal** : I want a specific action triggered on the first user message if the confidence is within a range (.75 to .85) for only a specific intent.
**Current plan** : Within the existing FallbackClassifier class, I would like to add an additional condition to check the confidence and intent and send back a custom intent to trigger a rule
eg: The model is 76% confident of intent ‘hungry’ (the target for this case). Within FallbackClassifier the new intent will be ‘maybe\_hungry’ and get pushed into the top confidence index (but have the same confidence level). Then a rule will check for ‘conversation\_start: true’ and respond with a custom action.
**Problem** : If it is not the start of the conversation, changing the top intent name will obviously cause problems with story structures.
I’ve considered making another rule that will check for ‘conversation\_start: false’ and then ‘maybe\_hungry’ to trigger a different action that would revert ‘maybe\_hungry’ to ‘hungry’, but I don’t know if that will still cause problems for stories. I saw the “undo fallback prediction” method could potentially be used for this, but I can’t find any usage examples. I assume it would be in a custom action somehow?
Alternatively, would checking within the FallbackClassifier itself for “first message” status work? After digging through the code I found that a generator can be used to make a list of prior trackers/actions, which is checked to see if it’s empty to verify ‘conversation\_start’, but I can’t figure out how to leverage this within the FallbackClassifier.
---
### Author:  [@fkoerner](https://forum.rasa.com/u/fkoerner)
#### Post date: [February 12, 2021, 8:05pm UTC](https://forum.rasa.com/t/attempting-to-handle-confidence-within-a-range-with-conditional-rule-and-customized-fallbackclassifier/40070/2 "2021-02-12T20:05:20Z")
Hi, this seems like an interesting problem!
I’m not quite sure I’ve understood what you’re trying to do, could you confirm or correct my understanding?
If an utterance is predicted to have intent `hungry` with a confidence between 0.75 and 0.85 **and** it’s the first utterance you’d like to assign this utterance the intent `maybe_hungry` instead?
Or would you always like this to happen regardless of when the utterance is made, but you’re thinking it can’t be done for anything but the first message?
---
### Author:  [@zjntmo](https://forum.rasa.com/u/zjntmo)
#### Post date: [February 16, 2021, 11:32pm UTC](https://forum.rasa.com/t/attempting-to-handle-confidence-within-a-range-with-conditional-rule-and-customized-fallbackclassifier/40070/3 "2021-02-16T23:32:59Z")
@fkoernerHi, Felicia!
Your first interpretation is correct, I want to assign `maybe_hungry` only at the start of the conversation. However, my goal is not the intent assignment of `maybe_hungry` but to trigger `maybe_hungry_action`, and my creation of `maybe_hungry` was only so I could trigger a rule (in tandem with `conversation_start: true`) that would execute `maybe_hungry_action`
The way I currently assign `maybe_hungry` is by checking within the FallbackClassifier for intent `hungry` and confidence \>0.75 & \<0.85. Since the Classifier does not have access to the conversation tracker (does it…?) it can’t tell if it’s the start of the conversation, so it ends up also re-labeling `hungry` in the middle of the conversation which breaks storylines.
So the priority here is to simply trigger `maybe_hungry_action` when `hungry` is within a confidence range **and** it is the start of the conversation. I do not necessarily have any reason to use a `maybe_hungry` intent, other than I currently do not know how else to trigger a rule for the desired action.
---
### Author:  [@fkoerner](https://forum.rasa.com/u/fkoerner)
#### Post date: [February 17, 2021, 9:55am UTC](https://forum.rasa.com/t/attempting-to-handle-confidence-within-a-range-with-conditional-rule-and-customized-fallbackclassifier/40070/4 "2021-02-17T09:55:17Z")
Oh, interesting! What if you made a rule to **always** execute the `maybe_hungry_action` whenever the user says they’re hungry at the beginning of the conversation:
```auto
- rule: hungry rule
conversation_start: True
steps:
- intent: hungry
- action: maybe_hungry_action
```
Then inside the `maybe_hungry_action` first check what the confidence is, that information should be inside: `tracker.latest_message`
which will look something like this:
```auto
{'intent': {'id': 1234, 'name': 'hungry', 'confidence': 0.7512345},......}
```
If the confidence isn’t in the right range, you can exit out of the action early and return what you would like the behaviour to be for example: `return [FollowUpAction("action_listen")]`
If the confidence **is** in the right range, continue and do whatever it is you want to do. What do you think?
---
### Author:  [@zjntmo](https://forum.rasa.com/u/zjntmo)
#### Post date: [February 18, 2021, 3:29am UTC](https://forum.rasa.com/t/attempting-to-handle-confidence-within-a-range-with-conditional-rule-and-customized-fallbackclassifier/40070/5 "2021-02-18T03:29:59Z")
It does seem like it could work! However, I have the same concern as with my other approach; would executing a rule break the stories those intents are used in?
I see `action_two_stage_fallback` has a `_message_clarification` that returns the event list `[Listen(), last_message.intent.confidence = 1.0]`; is this meant to simulate the progression of the story uninterrupted? If so, it seems I should add that to the `maybe_hungry_action` instead of just returning early.
\*\*Edit: I just realized - would returning the event list to reset the conversation get the bot stuck in a loop, or since the event list is now longer than 0 would “conversation\_start: true” not trigger?
---
### Author:  [@fkoerner](https://forum.rasa.com/u/fkoerner)
#### Post date: [February 19, 2021, 8:09am UTC](https://forum.rasa.com/t/attempting-to-handle-confidence-within-a-range-with-conditional-rule-and-customized-fallbackclassifier/40070/6 "2021-02-19T08:09:23Z")
Well, it might require some tweaking if you want the prediction to change based on whether the action was executed or not. One thing to think about is: what do you want to happen if the intent doesn’t fit the requirements (i.e. it’s not the start of the story, or the confidence isn’t in the desired range)? If you can share this with me, I can try and give you more specific advice.
There are several ways to encourage the behaviour you’d like to see. If you know that you would like `intent: hungry` to be followed by a particular action you can return this as a `FollowUpAction`. You can send a different `FollowUpAction` for each of the two cases.
Or, you can use a slot to disambiguate whether `maybe_hungry` occurred or not.
For example:
```auto
slots:
maybe_hungry:
type: bool
influence_conversation: true
```
```auto
- story: maybe hungry conditions met
steps:
- intent: hungry
- action: maybe_hungry_action
- slot_was_set:
- maybe_hungry: true
< behaviour you want to see if maybe_hungry is true>
```
```auto
- story: maybe hungry conditions not met
steps:
- intent: hungry
- action: maybe_hungry_action
- slot_was_set:
- maybe_hungry: false
< behaviour you want to see if maybe_hungry is false>
```
---
### Author:  [@fkoerner](https://forum.rasa.com/u/fkoerner)
#### Post date: [February 19, 2021, 8:11am UTC](https://forum.rasa.com/t/attempting-to-handle-confidence-within-a-range-with-conditional-rule-and-customized-fallbackclassifier/40070/7 "2021-02-19T08:11:03Z")
(The stories could also be rules. It all depends on your use case)