Wrong action is getting called in between a story

[yogeshkakde]

Hi, I am facing an issue where in-between a story some other action is getting called. eg. Let’s say there are two stories or paths… 1. create a reservation 2. edit existing reservation. Now when I am in the first story. Ill ask a user for details such as restaurant name, booking date etc. .similar questions will be asked when editing the booking. So when I am in story1 for some reason it’s asking some questions related to editing. Intent and entities are identified properly.

story 2

• greet

• utter_ask_howcanhelp

• activate_lpar

• utter_on_it
• action_edit_reservation
• slot{“requested_slot”: “editcityname”}

• inform{“editcityname”: “newyork”}

• action_edit_reservation
• slot{“editcityname”: “newyork”}
• slot{“requested_slot”: “editrestaurantname”}

• inform{“editrestaurantname”: “newyorkpizza”}

• action_edit_reservation
• slot{“editrestaurantname”: “newyorkpizza”}
• slot{“requested_slot”: “fieldToEdit”}

• inform{“fieldToEdit”: “restaurant”}

• action_edit_reservation
• slot{“fieldToEdit”: “restaurant”}

• thankyou

• utter_goodbye

story 1

• greet

• utter_ask_howcanhelp

• create_reservation

• utter_on_it
• action_create_reservation
• slot{“requested_slot”: “cityname”}

• inform{“cityname”: “newyork”}

• action_create_reservation
• slot{“cityname”: “newyork”}
• slot{“requested_slot”: “restaurantname”}

• inform{“restaurantname”: “newyorkpizza”}

• action_create_reservation
• slot{“restaurantname”: “newyorkpizza”}
• slot{“requested_slot”: “numberofseats”}

• inform{“numberofseats”: 2}

• action_create_reservation
• slot{“numberofseats”: 2}

• thankyou

• utter_goodbye

ActionCreateReservation(FormAction):

RANDOMIZE = False

@staticmethod
def required_fields():
    return [
        CityNameFormField("cityname"),
        RestaurantNameFormField("restaurantname"),
        NumberOfSeatsFormField("numberofseats")
    ]

def name(self):
    return 'action_create_reservation'

def submit(self, dispatcher, tracker, domain):
    //Code to create reservation
    return [AllSlotsReset()]

class ActionEditReservation(FormAction):

RANDOMIZE = False

@staticmethod
def required_fields():
    return [
        CityNameFormField("editcityname"),
        RestaurantNameFormField("editrestaurantname"),
        FieldToEditFormField("fieldToEdit")
    ]

def name(self):
    return 'action_edit_reservation'

def submit(self, dispatcher, tracker, domain):
    //Code to edit reservation
    return [AllSlotsReset()]

[souvikg10 (Souvik Ghosh)]

It seems like your two stories are getting confused. Can you print a confusion matrix to see which actions are getting confused a lot?

there is a evaluate.py script in rasa core to check that

[souvikg10 (Souvik Ghosh)]

Also Interactive learning will help you understand at which point you start to get wrong actions, easy to debug

[yogeshkakde]

It is happening when its in story1 and user gives restaurant name. After that its says “DEBUG:rasa_core.policies.memoization:There is no memorised next action” and goes to story2. It askes for which reservation to edit.

[yogeshkakde]

I am new to rasa so I have a basic question. If there is a story and inputs are given according to the story then whats the reason for it move to other story?

[souvikg10 (Souvik Ghosh)]

Did you check which slots is set in story 1 during your conversation was is editcityname or city name?

[yogeshkakde]

Its city name. In logs I could see “rasa_core.policies.memoization:There is no memorised next action” and then it predicts action from other story.

[souvikg10 (Souvik Ghosh)]

you only have two examples ? or you have more for each of them?

That’s weird, if the slots are different for each story that should matter for the next story’s prediction.

Did you try interactive learning? you can correct this behaviour

[yogeshkakde]

I have more stories but I cannot paste the exact stories. I tried interactive online learning. I got similar stories from that as well. I tried evaludate.py and it says no wrong story and even the matrix is fine. If I train multiple times it resolves the issues sometimes. I am not sure whats wrong.

[souvikg10 (Souvik Ghosh)]

That is weird. You are running 0.11 version or previous?

Are your slots featurised as text or categorical? though if you receive the right slot name as ‘city name’ which is not present in any other story, there shouldn’t be any confusion.

[Abir]

it’s not your fault , don’t expect rasa stories to be accurate . I have been struggling to do the same for sometime but no luck . Tried a lot of permutations but eventually gave up

[Abir]

But does a city name say “NYC” or “LA” in a slot is different ? Both means true right? we can distinguish between two paths on the basis of slot’s emptiness not it’s content . Only if it’s categorical then only different content of slots means different.

@akelad please do respond , it seems this issue is popping out every now and then

[akelad (Akela Drissner)]

which issue precisely? If it’s a text slot, then the value of the slot doesn’t matter. If it’s categorical, then the value does matter

[akelad (Akela Drissner)]

@Abir

it’s not your fault , don’t expect rasa stories to be accurate . I have been struggling to do the same for sometime but no luck . Tried a lot of permutations but eventually gave up

They are accurate. If they’re not, there’s very likely a mistake in them

[Abir]

I am simply using interactive learning , the stories are purely generated by them. I haven’t added a single story on my own apart from the initial stateless stories. Can you explain where could be the mistake ?

Thanks in advance

[akelad (Akela Drissner)]

Could you show me what kind of issues you’re experiencing @Abir? A specific story example would be good

[Abir]

@akelad

• posting_date
  • action_check_intent
  • slot{“conv_id”: “1”}
• inform{“param”: “1”}
  • action_check_reg
  • slot{“regid”: “1”}
• inform{“param”: “1”}
  • action_check_proc
  • slot{“procid”: “1”}
• inform{“param”: “1940”}
  • action_check_iss
  • slot{“issid”: “1940”}
• inform{“param”: “1001”}
  • action_check_com
  • slot{“comid”: “1001”}
  • action_validate
  • slot{“is_valid”: true}
  • utter_file
• file
  • action_check_file
  • slot{“file”: “windows_txt_4567.wmp”}
  • action_post
  • utter_ack2
• affirm
  • utter_ack1
• deny
  • action_deny
  • action_restart

---

• sequence
  • action_check_intent
  • slot{“conv_id”: “1”}
• inform{“param”: “1”}
  • action_check_reg
  • slot{“regid”: “1”}
• inform{“param”: “1”}
  • action_check_proc
  • slot{“procid”: “1”}
• inform{“param”: “1940”}
  • action_check_iss
  • slot{“issid”: “1940”}
• inform{“param”: “1001”}
  • action_check_com
  • slot{“comid”: “1001”}
  • action_validate
  • slot{“is_valid”: true}
  • action_seq
  • utter_ack2
• affirm
  • utter_ack1
• deny
  • action_deny
  • action_restart

You see both the story starts with 2 distinct intents r8 , but it doesn’t follow the expected path . rather it follows one path for a portion and then jumps to another path , hence ruining the overall flow of bot. But only when we make each custom action distinct by name , it works to some extent. If you could address this issue

[akelad (Akela Drissner)]

Ah i know what’s going wrong – it’s because your max_history is not long enough. The stories start to be different at this point right?

@Abir

inform{“param”: “1001”}

• action_check_com
• slot{“comid”: “1001”}
• action_validate
• slot{“is_valid”: true}
• action_seq
• utter_ack2

For this to properly be able to distinguish between the stories you’d need a max_history of something like 17. That’s not really advisable though because it won’t memorise shorter story segments anymore. What you can do is create one action that you use for both, but does different things dependent on what the state of the conversation is.

[Abir]

So @akelad , i have found a way to solve this issue after failing multiple number of times with tuning hyperparameters lol

[akelad (Akela Drissner)]

how did you solve it? with the max_history?