FollowupAction triggers additional intents and actions

[EllisonFord (Ellison)]

Hi,

I’m trying to get an Action from my actions server to run another Action using the return FollowupAction(‘name_action’) method as seen in Events. Unfortunately it starts executing way more actions than I asked it to.

Basic example: I want ‘action_utter_goodbye’ to run ‘action_utter_greet’. Yet this is what I’m getting when I try that.

- Bye bye pal
127.0.0.1 - - [2019-11-02 01:09:42] "POST /webhook HTTP/1.1" 200 234 0.001607
- Hello friend
127.0.0.1 - - [2019-11-02 01:09:42] "POST /webhook HTTP/1.1" 200 169 0.000735
- Bye bye pal
127.0.0.1 - - [2019-11-02 01:09:42] "POST /webhook HTTP/1.1" 200 234 0.000661
- Hello friend
127.0.0.1 - - [2019-11-02 01:09:42] "POST /webhook HTTP/1.1" 200 169 0.000663
- Bye bye pal
127.0.0.1 - - [2019-11-02 01:09:42] "POST /webhook HTTP/1.1" 200 234 0.000656
- Hello friend
127.0.0.1 - - [2019-11-02 01:09:42] "POST /webhook HTTP/1.1" 200 169 0.000661
- Bye bye pal
127.0.0.1 - - [2019-11-02 01:09:42] "POST /webhook HTTP/1.1" 200 234 0.000699
- Hello friend
127.0.0.1 - - [2019-11-02 01:09:42] "POST /webhook HTTP/1.1" 200 169 0.000709
- Bye bye pal
127.0.0.1 - - [2019-11-02 01:09:42] "POST /webhook HTTP/1.1" 200 234 0.001346
- Hello friend

This is what my actions look like:

class UtterGreet(Action):
    def name(self) -> Text:
        return 'action_utter_greet'

    def run(self, dispatcher: CollectingDispatcher, tracker: Tracker, domain: Dict[Text, Any]):
        username = tracker.get_slot('username')
        print(f'\n- Hello {username}\n')
        print('followup_action:', tracker.followup_action)


class UtterGoodbye(Action):
    def name(self) -> Text:
        return 'action_utter_goodbye'

    def run(self, dispatcher: CollectingDispatcher, tracker: Tracker, domain: Dict[Text, Any]):
        print('\n- Bye bye pal\n')
        return [FollowupAction('action_utter_greet')]

And my training policies:

policies:
  - epochs: 50
    max_history: 6
    name: KerasPolicy
  - max_history: 6
    name: AugmentedMemoizationPolicy

And finally the said stories:

## GREETINGS
* greet_intent
  - action_utter_greet

## GOODBYES
* goodbye_intent
  - action_utter_goodbye

I would really appreciate some help with this. Is it a Policies issue? A Rasa Core issue? Something else?

Regards

[btotharye (Brian Hopkins)]

I’m about to test your config but with the initial config you get from rasa init I’m seeing what I believe to be right:

Config - also what is your pipeline you are using just to verify?:

language: en
pipeline: supervised_embeddings
policies:
- name: MemoizationPolicy
- name: KerasPolicy
- name: MappingPolicy

Actions:

class UtterGreet(Action):
    def name(self) -> Text:
        return 'action_utter_greet'

    def run(self, dispatcher: CollectingDispatcher, tracker: Tracker, domain: Dict[Text, Any]):
        print(f'\n- Hello')
        dispatcher.utter_message("Hello")
        print('followup_action:', tracker.followup_action)


class UtterGoodbye(Action):
    def name(self) -> Text:
        return 'action_utter_goodbye'

    def run(self, dispatcher: CollectingDispatcher, tracker: Tracker, domain: Dict[Text, Any]):
        print('\n- Bye bye pal\n')
        dispatcher.utter_message("Bye bye pal")
        return [FollowupAction('action_utter_greet')]

When I talk to it over Rasa X I’m seeing the bye response then the hello utterance so it seems fine:

Is this what you would expect/want to see?

[btotharye (Brian Hopkins)]

From testing your config @EllisonFord it appears to be the config that is doing it, I’m guessing the AugmentedMemoizationPolicy is doing it.

[gonesbuyo (Gonzalo Estrán Buyo)]

And how can this be solved? I’m also using AugmentedMemoizationPolicy and have the same problem with FollowupAction. And utter_template neither works as expected.

[btotharye (Brian Hopkins)]

For testing try out this configuration and let me know what you see:

language: en
pipeline: supervised_embeddings
policies:
- name: MemoizationPolicy
- name: KerasPolicy
- name: MappingPolicy

We have a PR in to document more about the AugmentedMemoizationPolicy but from reading it I believe this to be the issue:

document AugmentedMemoizationPolicy

Info about the policy below here leads me to think that is why this is occurring.

The AugmentedMemoizationPolicy remembers examples from training stories for up to max_history turns.

It has a forgetting mechanism that will forget a certain amount of steps in the conversation history and try to find a match in your stories with the reduced history. It predicts the next action with confidence 1.0 if a match is found, otherwise it predicts None with confidence 0.0.

Note

If it is needed to recall turns from training dialogues where some slots might not be set during prediction time, add relevant stories without such slots to training data, e.g. reminder stories.

Since slots that are set some time in the past are preserved in all future feature vectors until they are set to None, this policy has a capability to recall the turns up to max_history from training stories during prediction even if additional slots were filled in the past for current dialogue.

[EllisonFord (Ellison)]

Hi Brian, thank you for checking this issue. For me it was the Keras policy that I had to remove for the system not to act up with the FollowupAction return. The AugmentedMemoization I am still using and I get the follow-up action as I expect.

On your question as to which NLU Pipeline I’m using, I’m not using any at the moment. Just the RegexInterpreter being passed to the Agent so that I can directly test intents using: “/greet_intent” and so on.

As mentioned, I think there might be an issue with the KerasPolicy and the FollowupAction.

Regards

[btotharye (Brian Hopkins)]

Ok let me look further into the Keras and follow up action stuff

[btotharye (Brian Hopkins)]

Weird also cause I was using this config and it seemed to work fine for me:

I just had assumed it was the difference in the policy on Augmented. Maybe the pipeline is helping out with it as well.

language: en
pipeline: supervised_embeddings
policies:
- name: MemoizationPolicy
- name: KerasPolicy
- name: MappingPolicy

[EllisonFord (Ellison)]

Maybe, I was testing without a pipeline, just pure Rasa Core Intents and Actions. Either way, I removed policies 1 by 1 in my tests, and it was only when the KerasPolicy was gone that the strange behaviour would also stop.

Thanks for your hekp Brian!