How current tracker is predicting next action depends on memorization and keras policy

surya1994 · July 26, 2019, 8:28am

Hi

I am new to rasa and i am trying to build a poc for my client but i am not able to understand how policies(memeorization and keras ) predict next action depends on current tracker and also i am not able to understand how current tracker works (ie how to check action flow in current tracker i am bit confused while i am checking current tracker).

when i train model with augmentation 0 during flow somes times it is going to fallback action but if i train with out augmentation 0 it is predicting next actions correctly .can you please let me know why it is behaving like this when augmentation 0 .

below is a story,config,debug details attcahed for reference.

This is a sample story i have another 4 stories for different flows STORY:

* user_payment_message 
 - utter_message_login_credentials
 - utter_yes_no_buttons
* affirm
 - utter_user_id
* user_name{"user_id":"kondapur"}
 - action_user_id_slot_set
 - slot{"user_id":"kondapur"}
 - utter_password
* user_password{"password":"action123"}
 - action_password_slot_set
 - slot{"password":"action123"}
 - action_outstanding_amount_preprocess
 - slot{"group_num":"910098"}
 - slot{"user_name":"SuryaMurthi"}
 - slot{"Email_address":"suryanarayanamurthy.gadepalli@anthem.com"}
 - slot{"group_name":"DAVID_HOUSE_17"}
 - action_outstanding_amount
 - slot{"state_code":"CA"}
 - slot{"brandCode":"VIRGINIA"}
 - slot{"businessunitcode":"SMGRP"}
 - slot{"groupsyskey":"15565"}
 - slot{"billingentitysyskey":"37366"}
 - slot{"lockboxnumber":"116789"}
 - slot{"billingentityname":"BULB_TESTER"}
 - slot{"paidamount":"12300"}
 - slot{"operationname":"SAVE_PAYMENTS"}
 - slot{"systemcode":"STARWARS"}
 - slot{"err_reason":"bill empty"}
 - action_due_amount_check
 - slot{"due_amount_check":"False"}
 - action_due_amount_check_validated
 - slot{"due_amount_flag": 0}
 - utter_payment_message
 - utter_yes_no_buttons
* affirm
 - action_bank_account_detais
 - slot{"accounttype":"CHECKINGTYPE"}
 - slot{"accountno":"5555551111"}
 - slot{"routingnumber":"12312312314567"}
 - utter_payment_account_deny_affirm
 - utter_yes_no_buttons
* affirm
 - action_bank_payment_success
 - utter_ask_rating
 - utter_rating_buttons
* user_rating{"rate":"1"}
 - utter_thanks
 - action_user_restart

CONFIG:

# Configuration for Rasa NLU.
# https://rasa.com/docs/rasa/nlu/components/
language: en
pipeline: supervised_embeddings

  

# Configuration for Rasa Core.
# https://rasa.com/docs/rasa/core/policies/
policies:
  - name: MemoizationPolicy
    max_history: 19
  - name: KerasPolicy
    max_history: 19
  - name: MappingPolicy
  - name: FallbackPolicy
    nlu_threshold: 0.4
    core_threshold: 0.4
    fallback_action_name: "action_custom_fallback"

Custom fall back predict when augmentation 0:

2019-07-26 03:59:31 DEBUG    rasa.core.processor  - Predicted next action 'action_outstanding_amount' with confidence 1.00.
2019-07-26 03:59:31 DEBUG    rasa.core.actions.action  - Calling action endpoint to run action 'action_outstanding_amount'.
2019-07-26 03:59:37 DEBUG    rasa.core.processor  - Action 'action_outstanding_amount' ended with events '['SlotSet(key: err_reason, value: null or zero)', 'SlotSet(key: state_code, value: CA)']'
2019-07-26 03:59:37 DEBUG    rasa.core.processor  - Current slot values:
        Email_address: suryanarayanamurthy.gadepalli@anthem.com
        accountno: None
        accounttype: None
        billingentityname: None
        billingentitysyskey: None
        brandCode: None
        businessunitcode: None
        due_amount_check: False
        due_amount_flag: 0
        err_reason: null or zero
        group_name: DAVID_HOUSE
        group_num: 300700
        groupsyskey: None
        lockboxnumber: None
        operationname: None
        paidamount: None
        password: action123
        routingnumber: None
        state_code: CA
        systemcode: None
        user_id: surya2019
        user_name: SuryaMurthi
2019-07-26 03:59:37 DEBUG    rasa.core.policies.memoization  - Current tracker state [None, None, None, None, None, None, {'slot_err_reason_0': 1.0, 'slot_due_amount_check_0': 1.0, 'slot_due_amount_flag_0': 1.0}, {'slot_due_amount_check_0': 1.0, 'slot_due_amount_flag_0': 1.0, 'slot_err_reason_0': 1.0, 'prev_action_listen': 1.0, 'intent_user_payment_message': 1.0}, {'prev_utter_message_login_credentials': 1.0, 'slot_due_amount_check_0': 1.0, 'slot_due_amount_flag_0': 1.0, 'slot_err_reason_0': 1.0, 'intent_user_payment_message': 1.0}, {'slot_due_amount_check_0': 1.0, 'prev_utter_yes_no_buttons': 1.0, 'slot_due_amount_flag_0': 1.0, 'slot_err_reason_0': 1.0, 'intent_user_payment_message': 1.0}, {'slot_due_amount_check_0': 1.0, 'intent_affirm': 1.0, 'slot_due_amount_flag_0': 1.0, 'slot_err_reason_0': 1.0, 'prev_action_listen': 1.0}, {'slot_due_amount_check_0': 1.0, 'intent_affirm': 1.0, 'prev_utter_user_id': 1.0, 'slot_due_amount_flag_0': 1.0, 'slot_err_reason_0': 1.0}, {'slot_due_amount_check_0': 1.0, 'intent_user_name': 1.0, 'slot_due_amount_flag_0': 1.0, 'slot_err_reason_0': 1.0, 'prev_action_listen': 1.0, 'entity_user_id': 1.0}, {'slot_due_amount_check_0': 1.0, 'intent_user_name': 1.0, 'slot_due_amount_flag_0': 1.0, 'prev_action_user_id_slot_set': 1.0, 'slot_err_reason_0': 1.0, 'entity_user_id': 1.0}, {'slot_due_amount_check_0': 1.0, 'intent_user_name': 1.0, 'slot_due_amount_flag_0': 1.0, 'prev_utter_password': 1.0, 'slot_err_reason_0': 1.0, 'entity_user_id': 1.0}, {'slot_due_amount_check_0': 1.0, 'slot_due_amount_flag_0': 1.0, 'entity_password': 1.0, 'slot_err_reason_0': 1.0, 'intent_user_password': 1.0, 'prev_action_listen': 1.0}, {'slot_due_amount_check_0': 1.0, 'slot_due_amount_flag_0': 1.0, 'entity_password': 1.0, 'prev_action_password_slot_set': 1.0, 'slot_err_reason_0': 1.0, 'intent_user_password': 1.0}, {'slot_due_amount_check_0': 1.0, 'slot_user_name_0': 1.0, 'prev_action_outstanding_amount_preprocess': 1.0, 'slot_due_amount_flag_0': 1.0, 'slot_group_num_0': 1.0, 'slot_group_name_0': 1.0, 'entity_password': 1.0, 'slot_err_reason_0': 1.0, 'intent_user_password': 1.0, 'slot_Email_address_0': 1.0}, {'slot_due_amount_check_0': 1.0, 'slot_user_name_0': 1.0, 'slot_due_amount_flag_0': 1.0, 'slot_group_num_0': 1.0, 'slot_group_name_0': 1.0, 'entity_password': 1.0, 'slot_state_code_0': 1.0, 'slot_err_reason_0': 1.0, 'intent_user_password': 1.0, 'prev_action_outstanding_amount': 1.0, 'slot_Email_address_0': 1.0}]
2019-07-26 03:59:37 DEBUG    rasa.core.policies.memoization  - There is no memorised next action
2019-07-26 03:59:37 DEBUG    rasa.core.policies.mapping_policy  - There is no mapped action for the predicted intent, 'user_password'.
2019-07-26 03:59:37 DEBUG    rasa.core.policies.fallback  - NLU confidence threshold met, confidence of fallback action set to core threshold (0.4).
2019-07-26 03:59:37 DEBUG    rasa.core.policies.ensemble  - Predicted next action using policy_3_FallbackPolicy
2019-07-26 03:59:37 DEBUG    rasa.core.processor  - Predicted next action 'action_custom_fallback' with confidence 0.40.
2019-07-26 03:59:37 DEBUG    rasa.core.actions.action  - Calling action endpoint to run action 'action_custom_fallback'.
2019-07-26 03:59:37 DEBUG    rasa.core.processor  - Action 'action_custom_fallback' ended with events '['BotUttered(text: Please reach out to our dedicated customer support team for help.\n (855) 854-1429 or Email General Support - CALGEmployerAccess@anthem.com, data: {"elements": null, "quick_replies": null, "buttons": null, "attachment": null, "image": null, "custom": null}, metadata: {})']'
2019-07-26 03:59:37 DEBUG    rasa.core.processor  - Current slot values:
        Email_address: suryanarayanamurthy.gadepalli@anthem.com
        accountno: None
        accounttype: None
        billingentityname: None
        billingentitysyskey: None
        brandCode: None
        businessunitcode: None
        due_amount_check: False
        due_amount_flag: 0
        err_reason: null or zero
        group_name: DAVID_HOUSE
        group_num: 300700
        groupsyskey: None
        lockboxnumber: None
        operationname: None
        paidamount: None
        password:action123
        routingnumber: None
        state_code: CA
        systemcode: None
        user_id: surya2019
        user_name: SuryaMurthi
2019-07-26 03:59:37 DEBUG    rasa.core.policies.memoization  - Current tracker state [None, None, None, None, None, {'slot_err_reason_0': 1.0, 'slot_due_amount_check_0': 1.0, 'slot_due_amount_flag_0': 1.0}, {'slot_due_amount_check_0': 1.0, 'slot_due_amount_flag_0': 1.0, 'slot_err_reason_0': 1.0, 'prev_action_listen': 1.0, 'intent_user_payment_message': 1.0}, {'prev_utter_message_login_credentials': 1.0, 'slot_due_amount_check_0': 1.0, 'slot_due_amount_flag_0': 1.0, 'slot_err_reason_0': 1.0, 'intent_user_payment_message': 1.0}, {'slot_due_amount_check_0': 1.0, 'prev_utter_yes_no_buttons': 1.0, 'slot_due_amount_flag_0': 1.0, 'slot_err_reason_0': 1.0, 'intent_user_payment_message': 1.0}, {'slot_due_amount_check_0': 1.0, 'intent_affirm': 1.0, 'slot_due_amount_flag_0': 1.0, 'slot_err_reason_0': 1.0, 'prev_action_listen': 1.0}, {'slot_due_amount_check_0': 1.0, 'intent_affirm': 1.0, 'prev_utter_user_id': 1.0, 'slot_due_amount_flag_0': 1.0, 'slot_err_reason_0': 1.0}, {'slot_due_amount_check_0': 1.0, 'intent_user_name': 1.0, 'slot_due_amount_flag_0': 1.0, 'slot_err_reason_0': 1.0, 'prev_action_listen': 1.0, 'entity_user_id': 1.0}, {'slot_due_amount_check_0': 1.0, 'intent_user_name': 1.0, 'slot_due_amount_flag_0': 1.0, 'prev_action_user_id_slot_set': 1.0, 'slot_err_reason_0': 1.0, 'entity_user_id': 1.0}, {'slot_due_amount_check_0': 1.0, 'intent_user_name': 1.0, 'slot_due_amount_flag_0': 1.0, 'prev_utter_password': 1.0, 'slot_err_reason_0': 1.0, 'entity_user_id': 1.0}, {'slot_due_amount_check_0': 1.0, 'slot_due_amount_flag_0': 1.0, 'entity_password': 1.0, 'slot_err_reason_0': 1.0, 'intent_user_password': 1.0, 'prev_action_listen': 1.0}, {'slot_due_amount_check_0': 1.0, 'slot_due_amount_flag_0': 1.0, 'entity_password': 1.0, 'prev_action_password_slot_set': 1.0, 'slot_err_reason_0': 1.0, 'intent_user_password': 1.0}, {'slot_due_amount_check_0': 1.0, 'slot_user_name_0': 1.0, 'prev_action_outstanding_amount_preprocess': 1.0, 'slot_due_amount_flag_0': 1.0, 'slot_group_num_0': 1.0, 'slot_group_name_0': 1.0, 'entity_password': 1.0, 'slot_err_reason_0': 1.0, 'intent_user_password': 1.0, 'slot_Email_address_0': 1.0}, {'slot_due_amount_check_0': 1.0, 'slot_user_name_0': 1.0, 'slot_due_amount_flag_0': 1.0, 'slot_group_num_0': 1.0, 'slot_group_name_0': 1.0, 'entity_password': 1.0, 'slot_state_code_0': 1.0, 'slot_err_reason_0': 1.0, 'intent_user_password': 1.0, 'prev_action_outstanding_amount': 1.0, 'slot_Email_address_0': 1.0}, {'slot_due_amount_check_0': 1.0, 'slot_user_name_0': 1.0, 'prev_action_custom_fallback': 1.0, 'slot_due_amount_flag_0': 1.0, 'slot_group_num_0': 1.0, 'slot_group_name_0': 1.0, 'entity_password': 1.0, 'slot_state_code_0': 1.0, 'slot_err_reason_0': 1.0, 'intent_user_password': 1.0, 'slot_Email_address_0': 1.0}]
2019-07-26 03:59:37 DEBUG    rasa.core.policies.memoization  - There is no memorised next action
2019-07-26 03:59:37 DEBUG    rasa.core.policies.mapping_policy  - There is no mapped action for the predicted intent, 'user_password'.
2019-07-26 03:59:37 DEBUG    rasa.core.policies.ensemble  - Predicted next action using policy_3_FallbackPolicy
2019-07-26 03:59:37 DEBUG    rasa.core.processor  - Predicted next action 'action_listen' with confidence 1.00.
2019-07-26 03:59:37 DEBUG    rasa.core.processor  - Action 'action_listen' ended with events '[]'
2019-07-26 03:59:37 DEBUG    rasa.core.agent  - Deleted lock for conversation '76d8e19c-37ec-42a3-a4b2-bd3007526f86' (unused)

DEBUG DETAILS HOW IT IS PREDICTING NEXT ACTION THROUGH MEMORIZATION AND KERAS POLICY:

2019-07-26 03:59:30 DEBUG    rasa.core.policies.memoization  - Current tracker state [None, None, None, None, None, None, None, None, {'slot_err_reason_0': 1.0, 'slot_due_amount_check_0': 1.0, 'slot_due_amount_flag_0': 1.0}, {'slot_due_amount_check_0': 1.0, 'slot_due_amount_flag_0': 1.0, 'slot_err_reason_0': 1.0, 'prev_action_listen': 1.0, 'intent_user_payment_message': 1.0}, {'prev_utter_message_login_credentials': 1.0, 'slot_due_amount_check_0': 1.0, 'slot_due_amount_flag_0': 1.0, 'slot_err_reason_0': 1.0, 'intent_user_payment_message': 1.0}, {'slot_due_amount_check_0': 1.0, 'prev_utter_yes_no_buttons': 1.0, 'slot_due_amount_flag_0': 1.0, 'slot_err_reason_0': 1.0, 'intent_user_payment_message': 1.0}, {'slot_due_amount_check_0': 1.0, 'intent_affirm': 1.0, 'slot_due_amount_flag_0': 1.0, 'slot_err_reason_0': 1.0, 'prev_action_listen': 1.0}, {'slot_due_amount_check_0': 1.0, 'intent_affirm': 1.0, 'prev_utter_user_id': 1.0, 'slot_due_amount_flag_0': 1.0, 'slot_err_reason_0': 1.0}, {'slot_due_amount_check_0': 1.0, 'intent_user_name': 1.0, 'slot_due_amount_flag_0': 1.0, 'slot_err_reason_0': 1.0, 'prev_action_listen': 1.0, 'entity_user_id': 1.0}, {'slot_due_amount_check_0': 1.0, 'intent_user_name': 1.0, 'slot_due_amount_flag_0': 1.0, 'prev_action_user_id_slot_set': 1.0, 'slot_err_reason_0': 1.0, 'entity_user_id': 1.0}, {'slot_due_amount_check_0': 1.0, 'intent_user_name': 1.0, 'slot_due_amount_flag_0': 1.0, 'prev_utter_password': 1.0, 'slot_err_reason_0': 1.0, 'entity_user_id': 1.0}, {'slot_due_amount_check_0': 1.0, 'slot_due_amount_flag_0': 1.0, 'entity_password': 1.0, 'slot_err_reason_0': 1.0, 'intent_user_password': 1.0, 'prev_action_listen': 1.0}, {'slot_due_amount_check_0': 1.0, 'slot_due_amount_flag_0': 1.0, 'entity_password': 1.0, 'prev_action_password_slot_set': 1.0, 'slot_err_reason_0': 1.0, 'intent_user_password': 1.0}]
2019-07-26 03:59:30 DEBUG    rasa.core.policies.memoization  - There is a memorised next action '14'
2019-07-26 03:59:30 DEBUG    rasa.core.policies.mapping_policy  - There is no mapped action for the predicted intent, 'user_password'.
2019-07-26 03:59:30 DEBUG    rasa.core.policies.fallback  - NLU confidence threshold met, confidence of fallback action set to core threshold (0.4).
2019-07-26 03:59:30 DEBUG    rasa.core.policies.ensemble  - Predicted next action using policy_0_MemoizationPolicy
2019-07-26 03:59:30 DEBUG    rasa.core.processor  - Predicted next action 'action_outstanding_amount_preprocess' with confidence 1.00.
2019-07-26 03:59:30 DEBUG    rasa.core.actions.action  - Calling action endpoint to run action 'action_outstanding_amount_preprocess'.
2019-07-26 03:59:31 DEBUG    rasa.core.processor  - Action 'action_outstanding_amount_preprocess' ended with events '['SlotSet(key: group_num, value: 315858)', 'SlotSet(key: user_name, value: SuryaMurthi)', 'SlotSet(key: Email_address, value: suryanarayanamurthy.gadepalli@anthem.com)', 'SlotSet(key: group_name, value: DAVID_CAB_17)']'
2019-07-26 03:59:31 DEBUG    rasa.core.processor  - Current slot values:
        Email_address: suryanarayanamurthy.gadepalli@anthem.com
        accountno: None
        accounttype: None
        billingentityname: None
        billingentitysyskey: None
        brandCode: None
        businessunitcode: None
        due_amount_check: False
        due_amount_flag: 0
        err_reason: no error
        group_name: DAVID_house
        group_num: 300700
        groupsyskey: None
        lockboxnumber: None
        operationname: None
        paidamount: None
        password: action123
        routingnumber: None
        state_code: None
        systemcode: None
        user_id: surya2019
        user_name: SuryaMurthi
2019-07-26 03:59:31 DEBUG    rasa.core.policies.memoization  - Current tracker state [None, None, None, None, None, None, None, {'slot_err_reason_0': 1.0, 'slot_due_amount_check_0': 1.0, 'slot_due_amount_flag_0': 1.0}, {'slot_due_amount_check_0': 1.0, 'slot_due_amount_flag_0': 1.0, 'slot_err_reason_0': 1.0, 'prev_action_listen': 1.0, 'intent_user_payment_message': 1.0}, {'prev_utter_message_login_credentials': 1.0, 'slot_due_amount_check_0': 1.0, 'slot_due_amount_flag_0': 1.0, 'slot_err_reason_0': 1.0, 'intent_user_payment_message': 1.0}, {'slot_due_amount_check_0': 1.0, 'prev_utter_yes_no_buttons': 1.0, 'slot_due_amount_flag_0': 1.0, 'slot_err_reason_0': 1.0, 'intent_user_payment_message': 1.0}, {'slot_due_amount_check_0': 1.0, 'intent_affirm': 1.0, 'slot_due_amount_flag_0': 1.0, 'slot_err_reason_0': 1.0, 'prev_action_listen': 1.0}, {'slot_due_amount_check_0': 1.0, 'intent_affirm': 1.0, 'prev_utter_user_id': 1.0, 'slot_due_amount_flag_0': 1.0, 'slot_err_reason_0': 1.0}, {'slot_due_amount_check_0': 1.0, 'intent_user_name': 1.0, 'slot_due_amount_flag_0': 1.0, 'slot_err_reason_0': 1.0, 'prev_action_listen': 1.0, 'entity_user_id': 1.0}, {'slot_due_amount_check_0': 1.0, 'intent_user_name': 1.0, 'slot_due_amount_flag_0': 1.0, 'prev_action_user_id_slot_set': 1.0, 'slot_err_reason_0': 1.0, 'entity_user_id': 1.0}, {'slot_due_amount_check_0': 1.0, 'intent_user_name': 1.0, 'slot_due_amount_flag_0': 1.0, 'prev_utter_password': 1.0, 'slot_err_reason_0': 1.0, 'entity_user_id': 1.0}, {'slot_due_amount_check_0': 1.0, 'slot_due_amount_flag_0': 1.0, 'entity_password': 1.0, 'slot_err_reason_0': 1.0, 'intent_user_password': 1.0, 'prev_action_listen': 1.0}, {'slot_due_amount_check_0': 1.0, 'slot_due_amount_flag_0': 1.0, 'entity_password': 1.0, 'prev_action_password_slot_set': 1.0, 'slot_err_reason_0': 1.0, 'intent_user_password': 1.0}, {'slot_due_amount_check_0': 1.0, 'slot_user_name_0': 1.0, 'prev_action_outstanding_amount_preprocess': 1.0, 'slot_due_amount_flag_0': 1.0, 'slot_group_num_0': 1.0, 'slot_group_name_0': 1.0, 'entity_password': 1.0, 'slot_err_reason_0': 1.0, 'intent_user_password': 1.0, 'slot_Email_address_0': 1.0}]
2019-07-26 03:59:31 DEBUG    rasa.core.policies.memoization  - There is a memorised next action '13'
2019-07-26 03:59:31 DEBUG    rasa.core.policies.mapping_policy  - There is no mapped action for the predicted intent, 'user_password'.
2019-07-26 03:59:31 DEBUG    rasa.core.policies.fallback  - NLU confidence threshold met, confidence of fallback action set to core threshold (0.4).
2019-07-26 03:59:31 DEBUG    rasa.core.policies.ensemble  - Predicted next action using policy_0_MemoizationPolicy
2019-07-26 03:59:31 DEBUG    rasa.core.processor  - Predicted next action 'action_outstanding_amount' with confidence 1.00.
2019-07-26 03:59:31 DEBUG    rasa.core.actions.action  - Calling action endpoint to run action 'action_outstanding_amount'.
2019-07-26 03:59:37 DEBUG    rasa.core.processor  - Action 'action_outstanding_amount' ended with events '['SlotSet(key: err_reason, value: null or zero)', 'SlotSet(key: state_code, value: CA)']'
2019-07-26 03:59:37 DEBUG    rasa.core.processor  - Current slot values:
        Email_address: suryanarayanamurthy.gadepalli@anthem.com
        accountno: None
        accounttype: None
        billingentityname: None
        billingentitysyskey: None
        brandCode: None
        businessunitcode: None
        due_amount_check: False
        due_amount_flag: 0
        err_reason: null or zero
        group_name: DAVID_HOUSE
        group_num: 300700
        groupsyskey: None
        lockboxnumber: None
        operationname: None
        paidamount: None
        password: action123
        routingnumber: None
        state_code: CA
        systemcode: None
        user_id: surya2019
        user_name: SuryaMurthi

Sorry for the long mail.

Thanks, Surya.

erohmensing · July 30, 2019, 6:29pm

Hi Surya, have you read the docs on augmentation here? I think what is happening is that the augmentation is improving the performance of your KerasPolicy’s predictions. However, augmentation doesn’t affect the MemoizationPolicy, so I think something might be going wrong in your first story, which might be something wrong in the back end here: action 'action_outstanding_amount' ended with events '['SlotSet(key: err_reason, value: null or zero)

In general these logs you’ve provided aren’t directly comparable since the first actions, and therefore the tracker state, are different, so it’s hard to tell exactly what’s going wrong.

surya1994 · August 10, 2019, 7:23pm

Hi @erohmensing

I am facing following issue when i train the model .I have a case in which i have to go to different story flows so i defined one common logic to go through different stories but some times intent that start flow for this common story is not predicting corresponding action but goes to fallback and here the prediction is done by keras policy not by memorization policy i tried different ways but i am not able to fix it so can you please let me know how to do this.

NLU Data:

intent:common_intent

Please let me know how can i help you today

This one is the intent i will get from buttons.

##Story1

common_intent

action_get_details_again
action_logic_set
slot{“setfalg”:1}
action_enter_detais

Notes: Here for some stories when ever i call this intent( common_intent) instead of going to action_get_details_again keras policy is predicting fallback_action i am not able to understand why only for some story it is behaving like this.

domain:

utter_key:

text:" Please enter yes or no" - buttons: - title"Yes" payload: ‘- Please let me know how can i help you today’
- title: “No” payload: ‘no’

policy:

Configuration for Rasa NLU.

Components

language: en pipeline: supervised_embeddings

Configuration for Rasa Core.

Policies

policies:

name: MemoizationPolicy max_history: 25
name: KerasPolicy max_history: 25
name: MappingPolicy
name: FallbackPolicy nlu_threshold: 0.6 core_threshold: 0.6 fallback_action_name: “action_custom_fallback”

surya1994 · August 12, 2019, 8:41am

can any one please answer my question

erohmensing · August 12, 2019, 8:55am

Are you saying that sometimes when you have the buttons it isn’t predicting with the memoization policy? Do you know you can call the intent directly in the payload with payload: "/common_intent"?

Are you making sure to start your story over with /restart before testing? Your tracker looks really long for a story this short.

surya1994 · August 13, 2019, 3:12pm

Thanks for reply Yes some times it is using keras policy instead of memorization policy and also keras is not predicting action after intent called instead it goes to default fall back action i am not able to understand how to improve accuracy of keras policy and i know that we can call intent( "/common_intent" ) directly by using payload but it does not look good in UI so that the reason i am calling intent using example(Please let me know how can i help). Here i cannot use restart for story because here i have set variables which i have to use for other cases also and also it is just a small story i have atleast 20 stories which have atleast 15 actions in it.

Can you please let me know how can i make keras policy predicting next action correctly and also why next action is predicted using keras policy instead of memorization and what is diffrence between keras and memorization policy

surya1994 · August 14, 2019, 6:40am

can any one please reply

erohmensing · August 14, 2019, 11:50am

i know that we can call intent( "/common_intent" ) directly by using payload but it does not look good in UI so that the reason i am calling intent using example(Please let me know how can i help).

Hm okay that makes sense I guess. Seems like that’s more of a UI issue, it should be displaying the title to the user, not the payload.

Here i cannot use restart for story because here i have set variables which i have to use for other cases also and also it is just a small story i have atleast 20 stories which have atleast 15 actions in it.

For this I just meant the examples you showed me – the stories aren’t comparable with the logs you showed me because there was so much other history in the tracker. If you have very long stories of 15 actions and lots of slots, the reason keras policy is kicking in is most likely because you’ve missed some slot events in your stories, or something like that. For a use case like this i’d recommend writing your stories with interactive learning.

surya1994 · August 14, 2019, 6:34pm

Thanks @erohmensing

Hm okay that makes sense I guess. Seems like that’s more of a UI issue, it should be displaying the title to the user, not the payload.

Here my requirement is my client has asked me to generate other forms by using same story before resetting all the slots(before restarting story) so i defined one common story so that flow can flow from one story to another story depends on user input for that here i defined a intent named common_intent with actions.common_intent story will get activated when ever user click “Yes” button and its corresponding payload is “Please let me know how can i help you today” and also this example gets populated on chat screen and also here text for buttons are showing correctly only but still i did not go with payload(/common_intent) directly because when user click Yes in that case “\common_intent” gets populated on the chat box which is not good in user interface prospective so that why i am going with example.so can you please let me know how can i call intent directly with out populating that payload on chat bot.

For this I just meant the examples you showed me – the stories aren’t comparable with the logs you showed me because there was so much other history in the tracker. If you have very long stories of 15 actions and lots of slots, the reason keras policy is kicking in is most likely because you’ve missed some slot events in your stories, or something like that. For a use case like this i’d recommend writing your stories with interactive learning.

Here the logs i have provided are nothing to do with my current problem these logs are of my above stories but still can you please let me know how to check my tracker if it is predicting according to story or not i am bit confused because there are lots of slots. As per my understanding both memorization and keras policies predict the next action but finally only with high score is selected among both.For all the stories the next action is predicted through memorization only but when i call common story the next action is predicted through keras and also for common story i set all the slots but i don’t know why still it is predicting next action through keras. Will we get keras policy if we define stories not correctly or if that is not the case can you please let me know the following: How can i check tracker what action keras policy predicted next (if tracker is present) How keras policy predicts next action Generally how much max history i have to consider if i have 15 actions in some stories

Can you also please let me know how can i integrate voice with rasa and what are the different voice assistants supported.

surya1994 · August 15, 2019, 5:00pm

Can any one please explain the above

erohmensing · August 22, 2019, 4:41pm

i did not go with payload(/common_intent) directly because when user click Yes in that case “\common_intent” gets populated on the chat box which is not good in user interface prospective so that why i am going with example.so can you please let me know how can i call intent directly with out populating that payload on chat bot.

Yeah that’s completely on your front-end. You have to use the /intent style in order to call an intent directly. Your front end should take the "title" of the button to populate the text field instead of the "payload" but should still send the "payload" to the rasa server. Rasa just takes whatever you send it.

As per my understanding both memorization and keras policies predict the next action but finally only with high score is selected among both.

This is correct. If memoization has a memorized action, it will predict it with a score of 1, so it will always beat keras. If it doesn’t have a memorized action, it will not predict an action, so keras will win.

For all the stories the next action is predicted through memorization only but when i call common story the next action is predicted through keras and also for common story i set all the slots but i don’t know why still it is predicting next action through keras.

this is probably because the intent is not being predicted correctly, or it is picking up entities where they are not expected, or setting an unexpected slot, etc.

How can i check tracker what action keras policy predicted next (if tracker is present)

You can only check which action was predicted and which policy it was predicted by. E.g. if memoization wins, you can’t see what keras predicted. these logs are in the debug:

2019-07-26 03:59:31 DEBUG    rasa.core.policies.ensemble  - Predicted next action using policy_0_MemoizationPolicy
2019-07-26 03:59:31 DEBUG    rasa.core.processor  - Predicted next action 'action_outstanding_amount' with confidence 1.00.