After Fallback policy is started Stories stops working

Rasa Open Source
1

Hi,

I have written stories using interactive learning tool , I am training my bot using 3 main policies i.e - Memorization , Keras and have a custom fallback policy with core threshold = 0.3 and nlu threshold = 0.2

I have atleast 40-50 stories which makes my bot working properly. I have been tracking user state when bot goes into a dialogue conversation and also a custom action triggers every time user skips the expected output. i.e Happy path

U - I want to order something

B - what do you want to order?

U - Good morning!

B - Good Morning

B- What do you want to order?

This is a dummy flow for explaining a little bit about my flow!

Now the problem is coming whenever user goes into fallback.

Bot handles it like i wanted but it does not follows the multi-step utterance or the story i wanted it to handle i.e

Not so happy path -

U - I want to order something

B - what do you want to order?

U - I want you to drive the car (fallback)

B - Sorry i cannot order that!

On debugging the http server i am finding that Bot is switching from Memorization policy to Fallback policy and and then it goes to Keras policy and hence it stops following the story. and therefore i did not get response structure i mentioned in first story.

Secondly - On removing the Fallback policy this issue gets resolved but whenever i enter some gibberish input like -

Unhappy path -

U - I want to order something

B - what do you want to order?

U - bshbehbgbbevbtb

Here bot predicts a wrong intent and thus action as well!

How can i control the policies so that i can prevent bot to hinder stories as well as use fallback policy for gibberish content

Thanks!

1 Like
2

hey @Juste / @Ghostvv can you guys help me out with this please?

As i have gone through the fallback policy docs it states that as soon as fallback is encountered bot utters fallback action and reverts back to prev current state. But this is not happening in my case. Bot starts predicting with fallback policy for all the utterances added by the user after the fallback event occurred. Sorry for bothering you guys with tag but this problem is not getting solved by me and i really need to finish this flow ugh deadlines! :sweat_smile:

Here is the debug trace for you guys

image
image1899×1016 54 KB

Thanks!

3

It looks like you have a custom fallback action action_fallback. Please read a note in the docs about fallback: Fallback Actions

4

i have read that and did that way also but same issue is coming

image
image1909×1016 54 KB

5

could you please put your fallback code here

6

class ActionFallback(Action): def name(self): return ‘action_fallback’

	def run(self, dispatcher, tracker, domain):

        
        response = '{intentName:Fallback1 , Response_user: Sorry! I was not able to find an answer for that}'

        dispatcher.utter_message(response)
7

but i am not using this action for fallback my fallback action is being used from domain file

8

I don’t see where, it returns UserUtteranceReverted

9

Could you paste your config for Fallback policy

10

fallback = FallbackPolicy(fallback_action_name=“utter_fallback”, core_threshold=0.2, nlu_threshold=0.3)

agent = Agent(‘domain_dialogue.yml’, policies=[MemoizationPolicy(max_history=3),KerasPolicy(),FormPolicy(),fallback])

11

you set your fallback action to be an utter template action, which doesn’t rewind previous user utterance. Either you should create a custom action, that follows the Note in the docs, or use utter_default and don’t give any fallback_action_name when you initialize FallbackPolicy

12

ok ill follow this and will see if it works!

Thanks for replying :slight_smile:

13

Hey @Ghostvv i tried following the steps mentioned here -

but it didnt work However utter_default worked fine ,

the code -

class ActionFallback(Action): def name(self): return ‘action_fallback’

 def apply_to(self,tracker):
    tracker._reset()
    tracker.replay_events()



 def run(self, dispatcher, tracker, domain):
        
        #intent = tracker.latest_message['intent'].get('name')
        response = '{intentName:Fallback1 , Response_user: Sorry! I was not able to find an answer for that}'

        dispatcher.utter_message(response)
        return []

and thirdly i am still having this issue -

14

the Note says:

You can also create your own custom action to use as a fallback. Be aware that if this action does not return a UserUtteranceReverted event, the next predictions of your bot may become inaccurate, as it very likely that the fallback action is not present in your stories

it should return [UserUtteranceReverted()] !!!

1 Like
15

I don’t understand what you mean with the last one

16

i mean that i have an action which usually triggers like this -

U - i want to order something

B- what would you like to order ?

U - i am thinking of booking a cab

B- where do you want to go?

(Here my bot utters next action whenever something is left unanswered)

B - Do you want to continue to order something?

Now in case of some fallback … my bot responds with fallback action but it is being predicted with fallback policy and thus previous state gets lost i.e the stories i have written gets lost. Now i know form action can work but at last we need stories to make bot remember the actions it should take which bot learns from stories

I hope this makes you clear

17

sorry, still don’t get it :grinning:

the idea of fallback is that it erases the history until before previous user input

18

i dont think so, fallbacks are meant to handle out of scope messages, Agree! but it does not mean that before state should get lost. If a user is in middle of a dialogue and utters an input which goes out of scope or context bot should handle it gracefully but should also remember the previous state.

In docs also for

Here i quote it is written -

action_default_fallback is a default action in Rasa Core, which will send the utter_default template message to the user. Make sure to specify this template in your domain file. It will also revert back to the state of the conversation before the user message that caused the fallback

19

I think there is some kind of misunderstanding, the state before previous utterance do not get lost, but this utterance and everything after it get lost, so what is written in the docs is correct

20

is their any way to prevent that? cz i tried with stories and it does not picks up stories at the time of fallback