Forms with from_intent failing

I have an issue that a form using from_intent will trigger action fallback after deactivating the form, instead of doing the action at the end of the form rule (action_start_scenario below). In debug, I would see the error There is no applicable rule. and that the fallback is activated. I was able to work around this using from_entity instead, and my guess is that the intents in my button payloads somehow messes up the action prediction after the form closes. Anyone understand why it does like this, and is it documented somewhere?

The form is:

forms:
  scenario_form:
    scenario:
    - type: from_intent
      value: apartment
      intent: play_scenario
    - type: from_intent
      value: play_scenario
      intent: pick_interview

The intention is to let the user select with buttons, and so I assumed that the best mapping would be from_intent with a unique intent for each button. So I have an utterance like this:

  utter_ask_scenario_form_scenario:
  - text: Pick a scenario
    buttons:
    - title: Apartment
      payload: /pick_apartment
    - title: Interview
      payload: /pick_interview

And a rule like this:

- rule: activate scenario form
  steps:
  - intent: play_scenario
  - action: scenario_form
  - active_loop: scenario_form

- rule: submit scenario form
  condition:
  # Condition that form is active.
  - active_loop: scenario_form
  steps:
  # Form is deactivated
  - action: scenario_form
  - active_loop: null
  - slot_was_set:
    - requested_slot: null
  # The actions we want to run when the form is submitted.
  - action: action_start_scenario

Here’s a log of the issue (with only difference that it also added an entity to the button payloads):

Your input ->  /play_scenario                                                                                                                                                                             
2020-12-16 11:58:57 DEBUG    rasa.core.lock_store  - Issuing ticket for conversation '48f4e96d49154477922a82dc6afea672'.
2020-12-16 11:58:57 DEBUG    rasa.core.lock_store  - Acquiring lock for conversation '48f4e96d49154477922a82dc6afea672'.
2020-12-16 11:58:57 DEBUG    rasa.core.lock_store  - Acquired lock for conversation '48f4e96d49154477922a82dc6afea672'.
2020-12-16 11:58:57 DEBUG    rasa.core.tracker_store  - Creating a new tracker for id '48f4e96d49154477922a82dc6afea672'.
2020-12-16 11:58:57 DEBUG    rasa.core.processor  - Starting a new session for conversation ID '48f4e96d49154477922a82dc6afea672'.
2020-12-16 11:58:57 DEBUG    rasa.core.processor  - Action 'action_session_start' ended with events '[<rasa.shared.core.events.SessionStarted object at 0x1456b07c0>, <rasa.shared.core.events.ActionExecuted object at 0x1456b0f40>]'.
2020-12-16 11:58:57 DEBUG    rasa.core.processor  - Current slot values: 
        bot_emotion: None
        requested_slot: None
        scenario: None
        scenario_mode: None
        scenario_set: False
2020-12-16 11:58:57 DEBUG    rasa.core.processor  - Received user message '/play_scenario' with intent '{'name': 'play_scenario', 'confidence': 1.0}' and entities '[]'
2020-12-16 11:58:57 DEBUG    rasa.core.processor  - Logged UserUtterance - tracker now has 4 events.
2020-12-16 11:58:57 DEBUG    rasa.core.policies.rule_policy  - Current tracker state: [{'slots': {'scenario_set': (1.0, 0.0)}}, {'user': {'intent': 'play_scenario'}, 'slots': {'scenario_set': (1.0, 0.0)}, 'prev_action': {'action_name': 'action_listen'}}]
2020-12-16 11:58:57 DEBUG    rasa.core.policies.rule_policy  - There is a rule for the next action 'scenario_form'.
2020-12-16 11:58:57 DEBUG    rasa.core.policies.ensemble  - Predicted next action using policy_0_RulePolicy
2020-12-16 11:58:57 DEBUG    rasa.core.processor  - Predicted next action 'scenario_form' with confidence 1.00.
2020-12-16 11:58:57 DEBUG    rasa.core.actions.forms  - Activated the form 'scenario_form'.
2020-12-16 11:58:57 DEBUG    rasa.core.actions.forms  - No pre-filled required slots to validate.
2020-12-16 11:58:57 DEBUG    rasa.core.actions.forms  - Validating user input 'UserUttered(text: /play_scenario, intent: {'name': 'play_scenario', 'confidence': 1.0}, entities: [])'.
2020-12-16 11:58:57 DEBUG    rasa.core.actions.forms  - Validating extracted slots: {}
2020-12-16 11:58:57 DEBUG    rasa.core.actions.forms  - Request next slot 'scenario'
2020-12-16 11:58:57 DEBUG    rasa.core.processor  - Action 'scenario_form' ended with events '[<rasa.shared.core.events.ActiveLoop object at 0x10f879250>, <rasa.shared.core.events.SlotSet object at 0x1456f05e0>, BotUttered('Pick a scenario', {"elements": null, "quick_replies": null, "buttons": [{"title": "Apartment", "payload": "/pick_apartment{\"scenario_set\": null}"}, {"title": "Interview", "payload": "/pick_interview{\"scenario_set\": null}"}], "attachment": null, "image": null, "custom": null}, {"template_name": "utter_ask_scenario_form_scenario"}, 1608116337.152012)]'.
2020-12-16 11:58:57 DEBUG    rasa.core.processor  - Current slot values: 
        bot_emotion: None
        requested_slot: scenario
        scenario: None
        scenario_mode: None
        scenario_set: False
2020-12-16 11:58:57 DEBUG    rasa.core.policies.rule_policy  - Predicted 'action_listen' after loop 'scenario_form'.
2020-12-16 11:58:57 DEBUG    rasa.core.policies.ensemble  - Predicted next action using policy_0_RulePolicy
2020-12-16 11:58:57 DEBUG    rasa.core.processor  - Predicted next action 'action_listen' with confidence 1.00.
2020-12-16 11:58:57 DEBUG    rasa.core.processor  - Action 'action_listen' ended with events '[]'.
2020-12-16 11:58:57 DEBUG    rasa.core.lock_store  - Deleted lock for conversation '48f4e96d49154477922a82dc6afea672'.
? Pick a scenario  1: Apartment (/pick_apartment{"scenario_set": null})                                                                                                                                   
2020-12-16 11:59:02 DEBUG    rasa.core.lock_store  - Issuing ticket for conversation '48f4e96d49154477922a82dc6afea672'.
2020-12-16 11:59:02 DEBUG    rasa.core.lock_store  - Acquiring lock for conversation '48f4e96d49154477922a82dc6afea672'.
2020-12-16 11:59:02 DEBUG    rasa.core.lock_store  - Acquired lock for conversation '48f4e96d49154477922a82dc6afea672'.
2020-12-16 11:59:02 DEBUG    rasa.core.tracker_store  - Recreating tracker for id '48f4e96d49154477922a82dc6afea672'
2020-12-16 11:59:02 DEBUG    rasa.core.processor  - Received user message '/pick_apartment{"scenario_set": null}' with intent '{'name': 'pick_apartment', 'confidence': 1.0}' and entities '[{'entity': 'scenario_set', 'start': 15, 'end': 37, 'value': None}]'
2020-12-16 11:59:02 DEBUG    rasa.core.processor  - Current slot values: 
        bot_emotion: None
        requested_slot: scenario
        scenario: None
        scenario_mode: None
        scenario_set: None
2020-12-16 11:59:02 DEBUG    rasa.core.processor  - Logged UserUtterance - tracker now has 11 events.
2020-12-16 11:59:02 DEBUG    rasa.core.policies.rule_policy  - Predicted loop 'scenario_form'.
2020-12-16 11:59:02 DEBUG    rasa.core.policies.ensemble  - Predicted next action using policy_0_RulePolicy
2020-12-16 11:59:02 DEBUG    rasa.core.processor  - Predicted next action 'scenario_form' with confidence 1.00.
2020-12-16 11:59:02 DEBUG    rasa.core.actions.forms  - Validating user input 'UserUttered(text: /pick_apartment{"scenario_set": null}, intent: {'name': 'pick_apartment', 'confidence': 1.0}, entities: [{'entity': 'scenario_set', 'start': 15, 'end': 37, 'value': None}])'.
2020-12-16 11:59:02 DEBUG    rasa.core.actions.forms  - Trying to extract requested slot 'scenario' ...
2020-12-16 11:59:02 DEBUG    rasa.core.actions.forms  - Got mapping '{'type': 'from_intent', 'value': 'apartment', 'intent': 'pick_apartment'}'
2020-12-16 11:59:02 DEBUG    rasa.core.actions.forms  - Successfully extracted 'apartment' for requested slot 'scenario'
2020-12-16 11:59:02 DEBUG    rasa.core.actions.forms  - Validating extracted slots: {'scenario': 'apartment'}
2020-12-16 11:59:02 DEBUG    rasa.core.actions.forms  - Deactivating the form 'scenario_form'
2020-12-16 11:59:02 DEBUG    rasa.core.processor  - Action 'scenario_form' ended with events '[<rasa.shared.core.events.SlotSet object at 0x144d3f340>, <rasa.shared.core.events.SlotSet object at 0x144c74b50>, <rasa.shared.core.events.ActiveLoop object at 0x144d3f3a0>]'.
/Users/martin/dev/src/rasa2/venv/lib/python3.8/site-packages/rasa/shared/utils/io.py:93: UserWarning: Action 'scenario_form' set a slot type 'scenario' which it never set during the training. This can throw off the prediction. Make sure to include training examples in your stories for the different types of slots this action can return. Remember: you need to set the slots manually in the stories by adding '- slot{"scenario": apartment}' after the action.
2020-12-16 11:59:02 DEBUG    rasa.core.processor  - Current slot values: 
        bot_emotion: None
        requested_slot: None
        scenario: apartment
        scenario_mode: None
        scenario_set: None
2020-12-16 11:59:02 DEBUG    rasa.core.policies.rule_policy  - Current tracker state: [{'slots': {'scenario_set': (1.0, 0.0)}}, {'user': {'intent': 'play_scenario'}, 'slots': {'scenario_set': (1.0, 0.0)}, 'prev_action': {'action_name': 'action_listen'}}, {'user': {'intent': 'play_scenario'}, 'slots': {'scenario': (1.0, 0.0, 0.0)}, 'prev_action': {'action_name': 'scenario_form'}}]
2020-12-16 11:59:02 DEBUG    rasa.core.policies.rule_policy  - There is no applicable rule.
2020-12-16 11:59:02 DEBUG    rasa.core.policies.ensemble  - Predicted next action using policy_0_RulePolicy
2020-12-16 11:59:02 DEBUG    rasa.core.processor  - Predicted next action 'action_default_fallback' with confidence 0.30.
2020-12-16 11:59:02 DEBUG    rasa.core.processor  - Action 'action_default_fallback' ended with events '[BotUttered('Sorry, not sure what to do. Can you rephrase?', {"elements": null, "quick_replies": null, "buttons": null, "attachment": null, "image": null, "custom": null}, {"template_name": "utter_default"}, 1608116342.171509), <rasa.shared.core.events.UserUtteranceReverted object at 0x144c6e1f0>]'.
2020-12-16 11:59:02 DEBUG    rasa.core.processor  - Current slot values: 
        bot_emotion: None
        requested_slot: None
        scenario: None
        scenario_mode: None
        scenario_set: False
2020-12-16 11:59:02 DEBUG    rasa.core.processor  - Predicted next action 'action_listen' with confidence 1.00.
2020-12-16 11:59:02 DEBUG    rasa.core.processor  - Action 'action_listen' ended with events '[]'.
2020-12-16 11:59:02 DEBUG    rasa.core.lock_store  - Deleted lock for conversation '48f4e96d49154477922a82dc6afea672'.

Hi What rasa version are you running?