Handling unexpected intent received at a checkpoint

Rasa Open Source
1

Hey everyone,

As per current implementation, our rasa bot has the following conversation flow:

  • Bot greets the user and provides the most popular questions in our use case

  • User types/selects from the quick reply buttons.

  • Respective action/utterance is executed for the intent received

After that, the bot follows up with a question “Can I help you with something else?” which is a checkpoint. (The name of that checkpoint is check_require_help)

  • If the user provides intent affirm - the popular questions which were displayed at the start will be displayed again which resets back to starting position and the flow continues (if that makes sense)

  • If the user provides intent deny - the bot starts the feedback process

- story: checkpoint - user requires help
    steps:
      - checkpoint: check_require_help
      - intent: affirm
      - action: utter_popular_questions

  - story: checkpoint - user does not require help
    steps:
      - checkpoint: check_require_help
      - intent: deny
      - action: utter_check_user_satisfaction
      .
      .
      .

The whole approach works perfectly fine except in one case

Failing scenario : While at the checkpoint “check_require_help”, when the user provides one of the popular intents directly without affirming to continue the conversation, it’s resulting in core_fallback as the bot expects either affirm or deny at this point but received another intent.

Things to note:

  • From an end-user perspective, this is a valid scenario and the bot should not go to core_fallback

  • If we consider removing the checkpoint check_require_help, It makes it more difficult to map to feedback process in my situation

  • We have quite a few intents in our bot and each has unique functionality mapped to them. Hence it’s an overhead to add all intents to the checkpoints and in turn, create new stories having respective actions for each intent.

Solution which I tried:

For all the other intents except affirm and deny, one custom action - action_identify_intent is executed to find the previous message’s intent and we create a new UserUttered event where we have the text as /{intent_name} and send the parse data for the event along with it. This adds an user event to the events list inside tracker.

Surprisingly, the next_action_prediction is not considering this user event which we added from a custom action. Hence, the respective action is not getting executed.

Here is the implementation which i described above:

Stories:

- story: checkpoint - user provides the intent directly
    steps:
      - checkpoint: check_require_help
      - or:
        - intent: change_password
        - intent: change_username
        - intent: update_notification
        . 
        . 
        . 
      - action: action_identify_intent
  
  - story: checkpoint - user requires help
    steps:
      - checkpoint: check_require_help
      - intent: affirm
      - action: utter_popular_questions

  - story: checkpoint - user does not require help
    steps:
      - checkpoint: check_require_help
      - intent: deny
      - action: utter_check_user_satisfaction
      .
      .
      .

Custom action:

class ActionIdentityIntent(Action): 
    def name(self) -> Text:
        return "action_identify_intent"

    def run(
        self, dispatcher: CollectingDispatcher, tracker: Tracker, domain: Dict
    ) -> List[EventType]:

        logger.info(
            'action_identify_intent - Execution started')
        latest_intent = "/" + tracker.latest_message['intent_ranking'][0]['name']
        data = \
            {
                "intent": {
                    "name": latest_intent,
                    "confidence": 1.0,
                }
            }
        logger.info(
            'action_identify_intent - Execution ended') 
        return [UserUttered(text=latest_intent, parse_data=data)]

My ultimate goal is to make the bot understand that, even if the user directly gives an intent without affirming that he still wants to continue the conversation, it should perform the corresponding action for the intent received.

Also,

  • Adding more stories for each and every intent is not an optimal solution and it’s costlier as we have to create 2 stories everytime we introduce a new intent in the future
  • Having checkpoint is also essential as we need to split the conversation flow based on yes/no

Rasa version information:

Rasa Version      :         2.8.25
Minimum Compatible Version: 2.8.9
Rasa SDK Version  :         2.8.4
Rasa X Version    :         0.42.4
Python Version    :         3.7.4