Enforcing a 'Yes' or 'No'

Background: There are certain conversations where, at some point, we want to force the user to answer with a simple ‘yes’ or ‘no’. I know it is not ideal, but due to (unavoidable for now) latency, people already typed their next message when our answer to the previous one is still in flight. Or a simple ‘yes’ is accompanied by a lot of unnecessary information, leading to the wrong intent classification. This sends a lot of conversations unnecessarily off the rails.

We have found two methods of solving this. The first solution is in Rasa Core, the other in our action server.

The Rasa Core solution entails a form loop. At the point where the simple ‘yes’ or ‘no’ is required, we start an active loop and exit only when the ‘Yes’ or ‘no’ is given (obviously we also exit after x attempts, but that is not relevant).

What we need for that is the following rules/stories:

- rule: Actually cancel the order when the user says 'yes'
  condition:
  - active_loop: form_enforce_yes_or_no
  steps:
  - intent: confirm
  - action: action_cancel_order
  - active_loop: null

- story: Cancel order
  steps:
  - checkpoint: cancel_order_flow
  - action: action_request_cancel_order
  - active_loop: form_enforce_yes_or_no
  - action: form_enforce_yes_or_no
  - intent: confirm
  - action: action_cancel_order
  - active_loop: null

And an action to handle the yes/no loop, which will emit an action_execution_rejected event if the answer was not ‘yes’ or ‘no’.

This only handles the ‘yes’ part. Obviously for the ‘no’ a similar rule/story pair has to be created. And also for each instance where we want to enforce a yes/no. This gets quite verbose and it is very fragile, you have to do everything just right in terms of actions/loops/events/rules/conditions to make it work. It has taken me a lot of trial and error to get this working.

The second solution is to handle it purely in our action server. The mechanism is pretty simple. In the above example, the ‘action_request_cancel_order’ is what triggers the yes/no requirement. It does this by setting a slot with the two actions corresponding to the ‘yes’ and ‘no’ answer. When the user answers and Rasa Core executes the resolved action, we first check if the slot is set and if the action is in the list of allowed actions. If it is, we execute the action and reset the slot. If not, we emit a rewind event until the user gives one of the optional answers.

This second method has the advantage of having way less code and being more robust. We do not have to change any stories or code a custom solution for each instance where we want to enforce a general yes/no. But I find it less elegant because I feel it sort-of works against Rasa instead of with Rasa.

What do you feel? Is the second solution indeed the most optimal? Or is there maybe another that I overlooked?

How is form_enforce_yes_or_no defined in the domain? Asking because the intent confirm seems to be used to break out of the form, instead of filling a slot that could also be filled by deny, which could reduce verbosity.

The form definition is:

form_enforce_yes_or_no:
  answer:
  - type: from_text
      not_intent:
        - confirm
        - deny

In that case maybe there’s another option, using a loop but waiting on it to be fulfilled, not rejected. The slot is filled by intent. Whether this makes sense may depend what is in the cancel_order_flow preceding checkpoint - can you add that?

slots:
  answer:
     type: bool
     influence_conversation: true
form_enforce_yes_or_no:
  answer:
  - type: from_intent
    intent: confirm
    value: true
  - type: from_intent
    intent: deny
    value: false
- rule: enforce yes or no loop
  steps:
  - action: action_request_cancel_order
  - action: form_enforce_yes_or_no
  - active_loop: form_enforce_yes_or_no

- rule: Actually cancel the order when the user says 'yes'
  condition:
  - active_loop: form_enforce_yes_or_no
  steps:
  - action: form_enforce_yes_or_no
  - slot_was_set: 
       answer: true
  - active_loop: null
  - action: action_cancel_order


- rule: User said no
  condition:
  - active_loop: form_enforce_yes_or_no
  steps:
  - action: form_enforce_yes_or_no
  - slot_was_set: 
       answer: false
  - active_loop: null
  - action: <action when not cancelling order>

The other instances of forcing yes/no, are they also related to cancelling an order or totally different scenarios? I can think of some adjustments to the rules if it needs to be more general.

re.

The second solution is to handle it purely in our action server. The mechanism is pretty simple. In the above example, the ‘action_request_cancel_order’ is what triggers the yes/no requirement. It does this by setting a slot with the two actions corresponding to the ‘yes’ and ‘no’ answer. When the user answers and Rasa Core executes the resolved action, we first check if the slot is set and if the action is in the list of allowed actions. If it is, we execute the action and reset the slot. If not, we emit a rewind event until the user gives one of the optional answers.

I’m not sure I’m understanding this right. Does this entail having a custom action return a forced follow up action?

Thanks for the suggestion, Melinda.

Your solution looks a bit cleaner than mine, mainly because does not seem to be needing the action_execution_rejected event, which makes it cleaner and less fragile.

The checkpoint before the cancel order flow contains two different flows: one where the user is already logged in and we can immediately check preconditions (does the user have a cancellable order?) or ask him to log in and then continue with the precondition check.

- story: Cancel order
  steps:
  - intent: cancel-order
  - checkpoint: cancel_order_flow

- story: Cancel order, with login
  steps:
  - intent: cancel-order
  - action: action_request_cancel_order
  - slot_was_set:
    - action_outcome: login_required
  - intent: logged-in
  - action: action_check_login
  - slot_was_set:
    - action_outcome: authenticated
  - checkpoint: cancel_order_flow

It is not exactly a custom action. We have an action server coded in Kotlin which has a single point of entry and that is where we do the check. At that point it will simply rewind any action other than the allowed actions for the ‘Yes’, resp. ‘No’. As if it never happened.

This can work because all actions (also simple messages) are handled by our action server. We do not use any Rasa messages.

In that case there’d either be two rules or two stories to start the form, since it’ll lead to a contradiciton between rules and stories otherwise e.g.

- rule: enforce yes or no loop
  steps:
  - intent: cancel-order
  - action: action_request_cancel_order
  - slot_was_set:
    - action_outcome: null
  - action: form_enforce_yes_or_no
  - active_loop: form_enforce_yes_or_no

- rule: Cancel order, with login
  steps:
  - intent: logged-in
  - action: action_check_login
  - slot_was_set:
    - action_outcome: authenticated
  - action: form_enforce_yes_or_no
  - active_loop: form_enforce_yes_or_no

Re. the action server approach: I would avoid rewriting history where possible, although fallback is an exception so if you see this as a case of fallback the action server method could also makes sense. It’s a bit opaque though when writing stories.

1 Like