Doubt on why TED predicted action is not executing

Rasa Open Source
1

My story basically cancels the form like this:

Story

  - story: Cancel comprar_insumo form
    steps:
      - or:
          - intent: comprar_insumo
          - intent: consultar_precio
      - action: pedido_insumo_form
      - active_loop: pedido_insumo_form
      - or:
          - intent: cancelar
          - intent: deny
      - action: action_reset_forms_except_datos_generales
      - action: action_deactivate_loop
      - active_loop: null

Custom Action

class ActionResetFormsExceptDatosGenerales(Action):
    def name(self) -> Text:
        return 'action_reset_forms_except_datos_generales'

    def run(self, dispatcher: CollectingDispatcher,
            tracker: Tracker,
            domain: Dict[Text, Any]) -> List[Dict[Text, Any]]:
        dispatcher.utter_message('Ok, te cancelo la orden!')
        nombre_cliente = tracker.get_slot('nombre_cliente')
        telefono_cliente = tracker.get_slot('telefono_cliente')
        direccion_cliente = tracker.get_slot('direccion_cliente')
        email_cliente = tracker.get_slot('email_cliente')
        return [
            AllSlotsReset(),
            SlotSet('nombre_cliente', nombre_cliente),
            SlotSet('telefono_cliente', telefono_cliente),
            SlotSet('direccion_cliente', direccion_cliente),
            SlotSet('email_cliente', email_cliente)
        ]

Problem

When I run with rasa shell --debug I get the following output:

2021-10-12 16:39:13 DEBUG    rasa.core.processor  - Received user message 'ninguno' with intent '{'id': -1936173825261923767, 'name': 'cancelar', 'confidence': 0.9982604384422302}' and entities '[]'
2021-10-12 16:39:13 DEBUG    rasa.core.processor  - Logged UserUtterance - tracker now has 21 events.
2021-10-12 16:39:13 DEBUG    rasa.core.policies.memoization  - Current tracker state:
[state 1] user intent: comprar_insumo | user entities: ('insumo', 'modelo_maquina') | previous action name: action_listen
[state 2] user intent: comprar_insumo | user entities: ('insumo', 'modelo_maquina') | previous action name: pedido_insumo_form | active loop: {'name': 'pedido_insumo_form'}
[state 3] user intent: cancelar | previous action name: action_listen | active loop: {'name': 'pedido_insumo_form'}
2021-10-12 16:39:13 DEBUG    rasa.core.policies.memoization  - There is no memorised next action
2021-10-12 16:39:13 DEBUG    rasa.core.policies.rule_policy  - Current tracker state:
[state 1] user intent: comprar_insumo | user entities: ('insumo', 'modelo_maquina') | previous action name: action_listen
[state 2] user intent: comprar_insumo | user entities: ('insumo', 'modelo_maquina') | previous action name: pedido_insumo_form | active loop: {'name': 'pedido_insumo_form'}
[state 3] user text: ninguno | previous action name: action_listen | active loop: {'name': 'pedido_insumo_form'}
2021-10-12 16:39:13 DEBUG    rasa.core.policies.rule_policy  - There is no applicable rule.
2021-10-12 16:39:13 DEBUG    rasa.core.policies.rule_policy  - Predicted loop 'pedido_insumo_form'.
2021-10-12 16:39:13 DEBUG    rasa.core.policies.unexpected_intent_policy  - Querying for intent `cancelar`.
2021-10-12 16:39:13 DEBUG    rasa.core.policies.unexpected_intent_policy  - Score for intent `cancelar` is `1.6664292812347412`, while threshold is `-0.7296526432037354`.
2021-10-12 16:39:13 DEBUG    rasa.core.policies.unexpected_intent_policy  - Top 5 intents (in ascending order) that are likely here are: `[('restart', -1.3735716), ('session_start', -1.3735716), ('stop', -1.3735716), ('deny', 1.3235723), ('cancelar', 1.6664293)]`.
2021-10-12 16:39:13 DEBUG    rasa.core.policies.ted_policy  - TED predicted 'action_reset_forms_except_datos_generales' based on user intent.
2021-10-12 16:39:13 DEBUG    rasa.core.policies.ensemble  - Made prediction using user intent.
2021-10-12 16:39:13 DEBUG    rasa.core.policies.ensemble  - Added `DefinePrevUserUtteredFeaturization(False)` event.
2021-10-12 16:39:13 DEBUG    rasa.core.policies.ensemble  - Predicted next action using policy_1_RulePolicy.
2021-10-12 16:39:13 DEBUG    rasa.core.processor  - Predicted next action 'pedido_insumo_form' with confidence 1.00.
2021-10-12 16:39:13 DEBUG    rasa.core.actions.forms  - Validating user input 'UserUttered(text: ninguno, intent: cancelar, use_text_for_featurization: False)'.
2021-10-12 16:39:13 DEBUG    rasa.core.actions.forms  - Trying to extract requested slot 'eleccion_rollo_menu' ...
2021-10-12 16:39:13 DEBUG    rasa.core.actions.forms  - Got mapping '{'type': 'from_entity', 'entity': 'eleccion_rollo_menu', 'intent': [], 'not_intent': [], 'role': None, 'group': None}'
2021-10-12 16:39:13 DEBUG    rasa.core.actions.forms  - Failed to extract requested slot 'eleccion_rollo_menu'
2021-10-12 16:39:13 DEBUG    rasa.core.actions.forms  - Validating extracted slots: {}
2021-10-12 16:39:13 DEBUG    rasa.core.actions.action  - Calling action endpoint to run action 'validate_pedido_insumo_form'.
2021-10-12 16:39:15 DEBUG    rasa.core.actions.forms  - Request next slot 'eleccion_rollo_menu'
2021-10-12 16:39:15 DEBUG    rasa.core.actions.action  - Calling action endpoint to run action 'action_ask_eleccion_rollo_menu'.

Specifically my doubt is the following, why in above log first is shown

TED predicted 'action_reset_forms_except_datos_generales' based on user intent. (what I actually want)

but then form is executed, and my custom action is ignored:

Predicted next action 'pedido_insumo_form' with confidence 1.00.

How can I fix this?

2

Im still stuck. This limits the rasa power for me to use rules than stories :frowning:

3

Hello @andrejankas, thanks for your question. It seems that the prediction of the rule below overrides TED prediction (confidence 1.0):

2021-10-12 16:39:13 DEBUG rasa.core.policies.ensemble - Predicted next action using policy_1_RulePolicy.

2021-10-12 16:39:13 DEBUG rasa.core.processor - Predicted next action ‘pedido_insumo_form’ with confidence 1.00.

Would you mind showing other training data files, especially the rules that you defined?

4

Yes! Sorry for my late reply:

Rules

- rule: Activate Pedido Insumos form
  steps:
    - or:
        - intent: comprar_insumo
        - intent: consultar_precio
    - action: pedido_insumo_form
    - active_loop: pedido_insumo_form

- rule: Submit form
  condition:
    # Condition that form is active.
    - active_loop: pedido_insumo_form
  steps:
    # Form is deactivated
    - action: pedido_insumo_form
    - active_loop: null
    - slot_was_set:
        - requested_slot: null
    # The actions we want to run when the form is submitted.
    - action: datos_generales_form
    - action: action_validate_email
    - action: otp_auth_form
    - action: action_mostrar_pedido_insumo_form
    - action: custom_utter_algo_mas
5
all my rules 
version: "2.0"

rules:
- rule: Activate Datos Generales form
  steps:
    - action: datos_generales_form
    - active_loop: datos_generales_form

- rule: Submit Datos Generales form
  condition:
    - active_loop: datos_generales_form
  steps:
    - action: datos_generales_form
    - active_loop: null
    - slot_was_set:
        - requested_slot: null
    - action: custom_utter_saludar_from_datos_generales

- rule: Activate Pedido Insumos form
  steps:
    - or:
        - intent: comprar_insumo
        - intent: consultar_precio
    - action: pedido_insumo_form
    - active_loop: pedido_insumo_form

- rule: Submit form
  condition:
    # Condition that form is active.
    - active_loop: pedido_insumo_form
  steps:
    # Form is deactivated
    - action: pedido_insumo_form
    - active_loop: null
    - slot_was_set:
        - requested_slot: null
    # The actions we want to run when the form is submitted.
    - action: datos_generales_form
    - action: action_validate_email
    - action: otp_auth_form
    - action: action_mostrar_pedido_insumo_form
    - action: custom_utter_algo_mas

- rule: Activate OTP form
  steps:
    - action: otp_auth_form
    - active_loop: otp_auth_form

- rule: Submit OTP form
  condition:
    - active_loop: otp_auth_form
  steps:
    - action: otp_auth_form
    - active_loop: null
    - slot_was_set:
        - requested_slot: null

- rule: Activate Email form
  steps:
    - intent: informacion_pedido
    - action: email_form
    - active_loop: email_form

- rule: Submit Email form
  condition:
    # Condition that form is active.
    - active_loop: email_form
  steps:
    # Form is deactivated
    - action: email_form
    - active_loop: null
    - slot_was_set:
        - requested_slot: null
    - action: action_mostrar_ultimos_pedidos