[HELP] two stage fallback policy removes slot which was set in a custom action

Rasa Open Source
1

Hello

rasa 1.7.4

Short problem: Two stage fallback policy removes a slot set in a custom action when it detects low confidence intent.

Story looks like this:

* request_flight_info
  - action_prefill_flight_info_form
  - flight_info_form
  - form{"name": "flight_info_form"}
  - slot{"trip_type": "one way"}        // categorical type
  - slot{"cabin_type": "economy"}   // categorical type
  - form{"name": null}
  - utter_ask_form_confirmed    // asks user to confirm slot values
* deny
  - utter_form_not_confirmed    // tells the user he can make changes now
* change_origin_destination OR change_dates OR change_trip_type OR change_cabin_type OR change_contact_method OR change_passengers
  - action_change_form_slots        // custom action to handle change intent and extract slots
  - utter_ask_form_confirmed
* affirm
  - utter_form_confirmed
  - action_search_flights

When user writes change trip type to round trip, intent change_trip_type is being detected with a low confidence. Even if it is detected with low confidence, custom action action_change_form_slots is being executed and slot is being filled with value round trip.

After that, two stage fallback gets triggered. User is being asked to confirm. When he confirms his intent, the slot is reset to it’s previous value and the flow continues.

Expected result: slot value should be round trip after user confirms intent

Current result: slot contains previous value after user confirms intent

Is this intended behavior or am I missing something?

Upon digging into this, I found maybe relevant logs that might point to issue

2020-03-05 11:08:24 DEBUG    rasa.core.processor  - Received user message 'change trip type to round trip' with intent '{'name': 'change_trip_type', 'confidence': 0.546561580266845}' and entities '[{'start': 20, 'end': 30, 'value': 'round trip', 'entity': 'trip_type', 'confidence': 0.9938186175533266, 'extractor': 'CRFEntityExtractor'}]'
2020-03-05 11:08:24 DEBUG    rasa.core.processor  - Current slot values:
prints new slot values extracted by custom action
...
2020-03-05 11:08:24 DEBUG    rasa.core.policies.fallback  - NLU confidence 0.546561580266845 is lower than NLU threshold 0.60.
2020-03-05 11:08:24 DEBUG    rasa.core.policies.two_stage_fallback  - User 'default' has to affirm intent 'change_trip_type'.
...
Did you mean 'change_trip_type'?  1: Yes (/change_trip_type)
2020-03-05 11:08:31 DEBUG    rasa.core.tracker_store  - Recreating tracker for id 'default'
2020-03-05 11:08:31 DEBUG    rasa.core.processor  - Received user message '/change_trip_type' with intent '{'name': 'change_trip_type', 'confidence': 1.0}' and entities '[]'
...
2020-03-05 11:08:31 DEBUG    rasa.core.policies.two_stage_fallback  - User 'default' affirmed intent 'change_trip_type'
...
2020-03-05 11:08:31 DEBUG    rasa.core.processor  - Predicted next action 'action_revert_fallback_events' with confidence 1.00.
2020-03-05 11:08:31 DEBUG    rasa.core.processor  - Action 'action_revert_fallback_events' ended with events '[<rasa.core.events.UserUtteranceReverted object at 0x7f61623d1f98>, <rasa.core.events.UserUtteranceReverted object at 0x7f61623d1b70>, <rasa.core.events.ActionExecuted object at 0x7f6162431128>, <rasa.core.events.UserUttered object at 0x7f61623fcc50>]'.
2020-03-05 11:08:31 DEBUG    rasa.core.processor  - Current slot values:

here is prints slot values and I see that slot value is wrong
2

Anyone?

I forgot to mention that I was testing model from rasa shell. Is there any problem with that? Like shell buttons do not properly pass on entities, etc

Thanks

3

Hmm, well the custom action action_change_form_slots should be executed after confirming the intent with the user (by the two-stage FallBack policy). Can you please post the complete debug log of the server?

4

Here are the full logs

Your input ->  change trip type to one way
2020-03-06 06:32:26 DEBUG    rasa.core.tracker_store  - Recreating tracker for id 'default'
2020-03-06 06:32:26 WARNING  root  - Could not parse timestamp c48f94f19ced46718dfdb1cb77cf9ef5. Instead current UTC time will be passed to duckling. Error: invalid literal for int() with base 10: 'c48f94f19ced46718dfdb1cb77cf9ef5'
2020-03-06 06:32:26 DEBUG    rasa.core.processor  - Received user message 'change trip type to one way' with intent '{'name': 'change_trip_type', 'confidence': 0.24059756913673983}' and entities '[{'start': 20, 'end': 27, 'value': 'one way', 'entity': 'trip_type', 'confidence': 0.9736422292523272, 'extractor': 'CRFEntityExtractor'}, {'start': 20, 'end': 23, 'text': 'one', 'value': 1, 'confidence': 1.0, 'additional_info': {'value': 1, 'type': 'value'}, 'entity': 'number', 'extractor': 'DucklingHTTPExtractor'}]'
/home/eugen/venv-rasa-1.7.4/lib/python3.6/site-packages/rasa/utils/common.py:351: UserWarning: Interpreter parsed an entity 'number' which is not defined in the domain. Please make sure all entities are listed in the domain.
  More info at https://rasa.com/docs/rasa/core/domains/
2020-03-06 06:32:26 DEBUG    rasa.core.processor  - Current slot values:
	adults: 1
	cabin_type: economy
	children: 0
	contact_method: test@gmail.com
	departure_date: 2020-03-09T00:00:00.000-07:00
	destination: Paris
	infants: 0
	origin: Moldova
	requested_slot: None
	return_date: 2020-04-05T00:00:00.000-07:00
	total_passengers: None
	trip_type: one way
2020-03-06 06:32:26 DEBUG    rasa.core.processor  - Logged UserUtterance - tracker now has 57 events.
2020-03-06 06:32:26 WARNING  rasa.core.featurizers  - Feature 'entity_GPE' could not be found in feature map.
2020-03-06 06:32:26 WARNING  rasa.core.featurizers  - Feature 'entity_GPE' could not be found in feature map.
2020-03-06 06:32:26 WARNING  rasa.core.featurizers  - Feature 'entity_GPE' could not be found in feature map.
2020-03-06 06:32:26 WARNING  rasa.core.featurizers  - Feature 'entity_GPE' could not be found in feature map.
2020-03-06 06:32:26 WARNING  rasa.core.featurizers  - Feature 'entity_number' could not be found in feature map.
2020-03-06 06:32:26 DEBUG    rasa.core.policies.memoization  - Current tracker state [{'slot_cabin_type_0': 1.0, 'slot_trip_type_1': 1.0}, {'slot_trip_type_1': 1.0, 'entity_direction_from': 1.0, 'intent_request_flight_info': 1.0, 'prev_action_listen': 1.0, 'slot_cabin_type_0': 1.0, 'entity_GPE': 1.0}, {'slot_trip_type_1': 1.0, 'prev_action_prefill_flight_info_form': 1.0, 'entity_direction_from': 1.0, 'intent_request_flight_info': 1.0, 'slot_cabin_type_0': 1.0, 'entity_GPE': 1.0}, {'slot_trip_type_1': 1.0, 'entity_direction_from': 1.0, 'intent_request_flight_info': 1.0, 'prev_flight_info_form': 1.0, 'slot_cabin_type_0': 1.0, 'entity_GPE': 1.0}, {'slot_trip_type_1': 1.0, 'entity_direction_from': 1.0, 'intent_request_flight_info': 1.0, 'prev_utter_ask_form_confirmed': 1.0, 'slot_cabin_type_0': 1.0, 'entity_GPE': 1.0}, {'slot_trip_type_0': 1.0, 'intent_change_trip_type': 1.0, 'prev_action_listen': 1.0, 'slot_cabin_type_0': 1.0, 'entity_trip_type': 1.0, 'entity_number': 1.0}]
2020-03-06 06:32:26 DEBUG    rasa.core.policies.memoization  - Launch DeLorean...
2020-03-06 06:32:26 DEBUG    rasa.core.policies.memoization  - Current tracker state [None, None, None, None, {'slot_cabin_type_0': 1.0, 'slot_trip_type_1': 1.0}, {'slot_trip_type_0': 1.0, 'intent_change_trip_type': 1.0, 'prev_action_listen': 1.0, 'slot_cabin_type_0': 1.0, 'entity_trip_type': 1.0, 'entity_number': 1.0}]
2020-03-06 06:32:26 DEBUG    rasa.core.policies.memoization  - There is no memorised next action
2020-03-06 06:32:26 DEBUG    rasa.core.policies.fallback  - NLU confidence 0.24059756913673983 is lower than NLU threshold 0.60.
2020-03-06 06:32:26 DEBUG    rasa.core.policies.two_stage_fallback  - User 'default' has to affirm intent 'change_trip_type'.
2020-03-06 06:32:26 DEBUG    rasa.core.policies.form_policy  - There is no active form
2020-03-06 06:32:26 DEBUG    rasa.core.policies.ensemble  - Predicted next action using policy_2_TwoStageFallbackPolicy
2020-03-06 06:32:26 DEBUG    rasa.core.processor  - Predicted next action 'action_default_ask_affirmation' with confidence 1.00.
2020-03-06 06:32:26 DEBUG    rasa.core.processor  - Action 'action_default_ask_affirmation' ended with events '[BotUttered('Did you mean 'change_trip_type'?', {"elements": null, "quick_replies": null, "buttons": [{"title": "Yes", "payload": "/change_trip_type"}, {"title": "No", "payload": "/out_of_scope"}], "attachment": null, "image": null, "custom": null}, {}, 1583476346.4229012)]'.
2020-03-06 06:32:26 DEBUG    rasa.core.processor  - Current slot values:
	adults: 1
	cabin_type: economy
	children: 0
	contact_method: test@gmail.com
	departure_date: 2020-03-09T00:00:00.000-07:00
	destination: Paris
	infants: 0
	origin: Moldova
	requested_slot: None
	return_date: 2020-04-05T00:00:00.000-07:00
	total_passengers: None
	trip_type: one way
2020-03-06 06:32:26 WARNING  rasa.core.featurizers  - Feature 'entity_GPE' could not be found in feature map.
2020-03-06 06:32:26 WARNING  rasa.core.featurizers  - Feature 'entity_GPE' could not be found in feature map.
2020-03-06 06:32:26 WARNING  rasa.core.featurizers  - Feature 'entity_GPE' could not be found in feature map.
2020-03-06 06:32:26 WARNING  rasa.core.featurizers  - Feature 'entity_GPE' could not be found in feature map.
2020-03-06 06:32:26 WARNING  rasa.core.featurizers  - Feature 'entity_number' could not be found in feature map.
2020-03-06 06:32:26 WARNING  rasa.core.featurizers  - Feature 'entity_number' could not be found in feature map.
2020-03-06 06:32:26 DEBUG    rasa.core.policies.memoization  - Current tracker state [{'slot_trip_type_1': 1.0, 'entity_direction_from': 1.0, 'intent_request_flight_info': 1.0, 'prev_action_listen': 1.0, 'slot_cabin_type_0': 1.0, 'entity_GPE': 1.0}, {'slot_trip_type_1': 1.0, 'prev_action_prefill_flight_info_form': 1.0, 'entity_direction_from': 1.0, 'intent_request_flight_info': 1.0, 'slot_cabin_type_0': 1.0, 'entity_GPE': 1.0}, {'slot_trip_type_1': 1.0, 'entity_direction_from': 1.0, 'intent_request_flight_info': 1.0, 'prev_flight_info_form': 1.0, 'slot_cabin_type_0': 1.0, 'entity_GPE': 1.0}, {'slot_trip_type_1': 1.0, 'entity_direction_from': 1.0, 'intent_request_flight_info': 1.0, 'prev_utter_ask_form_confirmed': 1.0, 'slot_cabin_type_0': 1.0, 'entity_GPE': 1.0}, {'slot_trip_type_0': 1.0, 'intent_change_trip_type': 1.0, 'prev_action_listen': 1.0, 'slot_cabin_type_0': 1.0, 'entity_trip_type': 1.0, 'entity_number': 1.0}, {'prev_action_default_ask_affirmation': 1.0, 'slot_trip_type_0': 1.0, 'intent_change_trip_type': 1.0, 'slot_cabin_type_0': 1.0, 'entity_trip_type': 1.0, 'entity_number': 1.0}]
2020-03-06 06:32:26 DEBUG    rasa.core.policies.memoization  - Launch DeLorean...
2020-03-06 06:32:26 DEBUG    rasa.core.policies.memoization  - Current tracker state [None, None, None, None, {'slot_cabin_type_0': 1.0, 'slot_trip_type_1': 1.0}, {'slot_cabin_type_0': 1.0, 'slot_trip_type_1': 1.0, 'prev_action_default_ask_affirmation': 1.0}]
2020-03-06 06:32:26 DEBUG    rasa.core.policies.memoization  - There is no memorised next action
2020-03-06 06:32:26 DEBUG    rasa.core.policies.form_policy  - There is no active form
2020-03-06 06:32:26 DEBUG    rasa.core.policies.mapping_policy  - There is no mapped action for the predicted intent, 'change_trip_type'.
2020-03-06 06:32:26 DEBUG    rasa.core.policies.ensemble  - Predicted next action using policy_2_TwoStageFallbackPolicy
2020-03-06 06:32:26 DEBUG    rasa.core.processor  - Predicted next action 'action_listen' with confidence 1.00.
2020-03-06 06:32:26 DEBUG    rasa.core.processor  - Action 'action_listen' ended with events '[]'.
2020-03-06 06:32:26 DEBUG    rasa.core.lock_store  - Deleted lock for conversation 'default'.
? Did you mean 'change_trip_type'?  1: Yes (/change_trip_type)
2020-03-06 06:32:34 DEBUG    rasa.core.tracker_store  - Recreating tracker for id 'default'
2020-03-06 06:32:34 DEBUG    rasa.core.processor  - Received user message '/change_trip_type' with intent '{'name': 'change_trip_type', 'confidence': 1.0}' and entities '[]'
2020-03-06 06:32:34 DEBUG    rasa.core.processor  - Logged UserUtterance - tracker now has 61 events.
2020-03-06 06:32:34 WARNING  rasa.core.featurizers  - Feature 'entity_GPE' could not be found in feature map.
2020-03-06 06:32:34 WARNING  rasa.core.featurizers  - Feature 'entity_GPE' could not be found in feature map.
2020-03-06 06:32:34 WARNING  rasa.core.featurizers  - Feature 'entity_GPE' could not be found in feature map.
2020-03-06 06:32:34 WARNING  rasa.core.featurizers  - Feature 'entity_GPE' could not be found in feature map.
2020-03-06 06:32:34 WARNING  rasa.core.featurizers  - Feature 'entity_number' could not be found in feature map.
2020-03-06 06:32:34 WARNING  rasa.core.featurizers  - Feature 'entity_number' could not be found in feature map.
2020-03-06 06:32:34 DEBUG    rasa.core.policies.memoization  - Current tracker state [{'slot_trip_type_1': 1.0, 'prev_action_prefill_flight_info_form': 1.0, 'entity_direction_from': 1.0, 'intent_request_flight_info': 1.0, 'slot_cabin_type_0': 1.0, 'entity_GPE': 1.0}, {'slot_trip_type_1': 1.0, 'entity_direction_from': 1.0, 'intent_request_flight_info': 1.0, 'prev_flight_info_form': 1.0, 'slot_cabin_type_0': 1.0, 'entity_GPE': 1.0}, {'slot_trip_type_1': 1.0, 'entity_direction_from': 1.0, 'intent_request_flight_info': 1.0, 'prev_utter_ask_form_confirmed': 1.0, 'slot_cabin_type_0': 1.0, 'entity_GPE': 1.0}, {'slot_trip_type_0': 1.0, 'intent_change_trip_type': 1.0, 'prev_action_listen': 1.0, 'slot_cabin_type_0': 1.0, 'entity_trip_type': 1.0, 'entity_number': 1.0}, {'prev_action_default_ask_affirmation': 1.0, 'slot_trip_type_0': 1.0, 'intent_change_trip_type': 1.0, 'slot_cabin_type_0': 1.0, 'entity_trip_type': 1.0, 'entity_number': 1.0}, {'slot_cabin_type_0': 1.0, 'prev_action_listen': 1.0, 'slot_trip_type_0': 1.0, 'intent_change_trip_type': 1.0}]
2020-03-06 06:32:34 DEBUG    rasa.core.policies.memoization  - Launch DeLorean...
2020-03-06 06:32:34 DEBUG    rasa.core.policies.memoization  - Current tracker state [None, None, None, None, {'slot_cabin_type_0': 1.0, 'slot_trip_type_1': 1.0}, {'slot_cabin_type_0': 1.0, 'prev_action_listen': 1.0, 'slot_trip_type_1': 1.0, 'intent_change_trip_type': 1.0}]
2020-03-06 06:32:34 DEBUG    rasa.core.policies.memoization  - There is no memorised next action
2020-03-06 06:32:34 DEBUG    rasa.core.policies.two_stage_fallback  - User 'default' affirmed intent 'change_trip_type'
2020-03-06 06:32:34 DEBUG    rasa.core.policies.form_policy  - There is no active form
2020-03-06 06:32:34 DEBUG    rasa.core.policies.ensemble  - Predicted next action using policy_2_TwoStageFallbackPolicy
2020-03-06 06:32:34 DEBUG    rasa.core.processor  - Predicted next action 'action_revert_fallback_events' with confidence 1.00.
2020-03-06 06:32:34 DEBUG    rasa.core.processor  - Action 'action_revert_fallback_events' ended with events '[<rasa.core.events.UserUtteranceReverted object at 0x7f9fdc0a2da0>, <rasa.core.events.UserUtteranceReverted object at 0x7f9fdc0d87f0>, <rasa.core.events.ActionExecuted object at 0x7f9fdc083390>, <rasa.core.events.UserUttered object at 0x7f9fdc0a4cf8>]'.
2020-03-06 06:32:34 DEBUG    rasa.core.processor  - Current slot values:
	adults: 1
	cabin_type: economy
	children: 0
	contact_method: test@gmail.com
	departure_date: 2020-03-09T00:00:00.000-07:00
	destination: Paris
	infants: 0
	origin: Moldova
	requested_slot: None
	return_date: 2020-04-05T00:00:00.000-07:00
	total_passengers: None
	trip_type: round trip
2020-03-06 06:32:34 WARNING  rasa.core.featurizers  - Feature 'entity_GPE' could not be found in feature map.
2020-03-06 06:32:34 WARNING  rasa.core.featurizers  - Feature 'entity_GPE' could not be found in feature map.
2020-03-06 06:32:34 WARNING  rasa.core.featurizers  - Feature 'entity_GPE' could not be found in feature map.
2020-03-06 06:32:34 WARNING  rasa.core.featurizers  - Feature 'entity_GPE' could not be found in feature map.
2020-03-06 06:32:34 DEBUG    rasa.core.policies.memoization  - Current tracker state [{'slot_cabin_type_0': 1.0, 'slot_trip_type_1': 1.0}, {'slot_trip_type_1': 1.0, 'entity_direction_from': 1.0, 'intent_request_flight_info': 1.0, 'prev_action_listen': 1.0, 'slot_cabin_type_0': 1.0, 'entity_GPE': 1.0}, {'slot_trip_type_1': 1.0, 'prev_action_prefill_flight_info_form': 1.0, 'entity_direction_from': 1.0, 'intent_request_flight_info': 1.0, 'slot_cabin_type_0': 1.0, 'entity_GPE': 1.0}, {'slot_trip_type_1': 1.0, 'entity_direction_from': 1.0, 'intent_request_flight_info': 1.0, 'prev_flight_info_form': 1.0, 'slot_cabin_type_0': 1.0, 'entity_GPE': 1.0}, {'slot_trip_type_1': 1.0, 'entity_direction_from': 1.0, 'intent_request_flight_info': 1.0, 'prev_utter_ask_form_confirmed': 1.0, 'slot_cabin_type_0': 1.0, 'entity_GPE': 1.0}, {'slot_cabin_type_0': 1.0, 'prev_action_listen': 1.0, 'slot_trip_type_1': 1.0, 'intent_change_trip_type': 1.0}]
2020-03-06 06:32:34 DEBUG    rasa.core.policies.memoization  - Launch DeLorean...
2020-03-06 06:32:34 DEBUG    rasa.core.policies.memoization  - Current tracker state [None, None, None, None, {'slot_cabin_type_0': 1.0, 'slot_trip_type_1': 1.0}, {'slot_cabin_type_0': 1.0, 'prev_action_listen': 1.0, 'slot_trip_type_1': 1.0, 'intent_change_trip_type': 1.0}]
2020-03-06 06:32:34 DEBUG    rasa.core.policies.memoization  - There is no memorised next action
2020-03-06 06:32:34 DEBUG    rasa.core.policies.two_stage_fallback  - NLU confidence threshold met, confidence of fallback action set to core threshold (0.3).
2020-03-06 06:32:34 DEBUG    rasa.core.policies.form_policy  - There is no active form
2020-03-06 06:32:34 DEBUG    rasa.core.policies.ensemble  - Predicted next action using policy_0_EmbeddingPolicy
2020-03-06 06:32:34 DEBUG    rasa.core.processor  - Predicted next action 'action_change_form_slots' with confidence 0.98.
2020-03-06 06:32:34 DEBUG    rasa.core.actions.action  - Calling action endpoint to run action 'action_change_form_slots'.
2020-03-06 06:32:34 DEBUG    rasa.core.processor  - Action 'action_change_form_slots' ended with events '[]'.
2020-03-06 06:32:34 DEBUG    rasa.core.processor  - Current slot values:
	adults: 1
	cabin_type: economy
	children: 0
	contact_method: test@gmail.com
	departure_date: 2020-03-09T00:00:00.000-07:00
	destination: Paris
	infants: 0
	origin: Moldova
	requested_slot: None
	return_date: 2020-04-05T00:00:00.000-07:00
	total_passengers: None
	trip_type: round trip
2020-03-06 06:32:34 WARNING  rasa.core.featurizers  - Feature 'entity_GPE' could not be found in feature map.
2020-03-06 06:32:34 WARNING  rasa.core.featurizers  - Feature 'entity_GPE' could not be found in feature map.
2020-03-06 06:32:34 WARNING  rasa.core.featurizers  - Feature 'entity_GPE' could not be found in feature map.
2020-03-06 06:32:34 WARNING  rasa.core.featurizers  - Feature 'entity_GPE' could not be found in feature map.
2020-03-06 06:32:34 DEBUG    rasa.core.policies.memoization  - Current tracker state [{'slot_trip_type_1': 1.0, 'entity_direction_from': 1.0, 'intent_request_flight_info': 1.0, 'prev_action_listen': 1.0, 'slot_cabin_type_0': 1.0, 'entity_GPE': 1.0}, {'slot_trip_type_1': 1.0, 'prev_action_prefill_flight_info_form': 1.0, 'entity_direction_from': 1.0, 'intent_request_flight_info': 1.0, 'slot_cabin_type_0': 1.0, 'entity_GPE': 1.0}, {'slot_trip_type_1': 1.0, 'entity_direction_from': 1.0, 'intent_request_flight_info': 1.0, 'prev_flight_info_form': 1.0, 'slot_cabin_type_0': 1.0, 'entity_GPE': 1.0}, {'slot_trip_type_1': 1.0, 'entity_direction_from': 1.0, 'intent_request_flight_info': 1.0, 'prev_utter_ask_form_confirmed': 1.0, 'slot_cabin_type_0': 1.0, 'entity_GPE': 1.0}, {'slot_cabin_type_0': 1.0, 'prev_action_listen': 1.0, 'slot_trip_type_1': 1.0, 'intent_change_trip_type': 1.0}, {'slot_cabin_type_0': 1.0, 'prev_action_change_form_slots': 1.0, 'slot_trip_type_1': 1.0, 'intent_change_trip_type': 1.0}]
2020-03-06 06:32:34 DEBUG    rasa.core.policies.memoization  - Launch DeLorean...
2020-03-06 06:32:34 DEBUG    rasa.core.policies.memoization  - Current tracker state [None, None, None, None, {'slot_cabin_type_0': 1.0, 'slot_trip_type_1': 1.0}, {'slot_cabin_type_0': 1.0, 'slot_trip_type_1': 1.0, 'prev_action_change_form_slots': 1.0}]
2020-03-06 06:32:34 DEBUG    rasa.core.policies.memoization  - There is no memorised next action
2020-03-06 06:32:34 DEBUG    rasa.core.policies.two_stage_fallback  - NLU confidence threshold met, confidence of fallback action set to core threshold (0.3).
2020-03-06 06:32:34 DEBUG    rasa.core.policies.form_policy  - There is no active form
2020-03-06 06:32:34 DEBUG    rasa.core.policies.mapping_policy  - There is no mapped action for the predicted intent, 'change_trip_type'.
2020-03-06 06:32:34 DEBUG    rasa.core.policies.ensemble  - Predicted next action using policy_0_EmbeddingPolicy
2020-03-06 06:32:34 DEBUG    rasa.core.processor  - Predicted next action 'utter_ask_form_confirmed' with confidence 0.97.
2020-03-06 06:32:34 DEBUG    rasa.core.processor  - Action 'utter_ask_form_confirmed' ended with events '[BotUttered('Is this right?
 You requested a round trip search from Moldova to Paris in economy class
 Departure date: 2020-03-09T00:00:00.000-07:00; Return date: 2020-04-05T00:00:00.000-07:00
 Adults: 1; Children: 0; Infants: 0
 You will be contacted at test@gmail.com', {"elements": null, "quick_replies": null, "buttons": null, "attachment": null, "image": null, "custom": null}, {}, 1583476354.2795367)]'.
2020-03-06 06:32:34 WARNING  rasa.core.featurizers  - Feature 'entity_GPE' could not be found in feature map.
2020-03-06 06:32:34 WARNING  rasa.core.featurizers  - Feature 'entity_GPE' could not be found in feature map.
2020-03-06 06:32:34 WARNING  rasa.core.featurizers  - Feature 'entity_GPE' could not be found in feature map.
2020-03-06 06:32:34 WARNING  rasa.core.featurizers  - Feature 'entity_GPE' could not be found in feature map.
2020-03-06 06:32:34 DEBUG    rasa.core.policies.memoization  - Current tracker state [{'slot_trip_type_1': 1.0, 'prev_action_prefill_flight_info_form': 1.0, 'entity_direction_from': 1.0, 'intent_request_flight_info': 1.0, 'slot_cabin_type_0': 1.0, 'entity_GPE': 1.0}, {'slot_trip_type_1': 1.0, 'entity_direction_from': 1.0, 'intent_request_flight_info': 1.0, 'prev_flight_info_form': 1.0, 'slot_cabin_type_0': 1.0, 'entity_GPE': 1.0}, {'slot_trip_type_1': 1.0, 'entity_direction_from': 1.0, 'intent_request_flight_info': 1.0, 'prev_utter_ask_form_confirmed': 1.0, 'slot_cabin_type_0': 1.0, 'entity_GPE': 1.0}, {'slot_cabin_type_0': 1.0, 'prev_action_listen': 1.0, 'slot_trip_type_1': 1.0, 'intent_change_trip_type': 1.0}, {'slot_cabin_type_0': 1.0, 'prev_action_change_form_slots': 1.0, 'slot_trip_type_1': 1.0, 'intent_change_trip_type': 1.0}, {'slot_cabin_type_0': 1.0, 'slot_trip_type_1': 1.0, 'prev_utter_ask_form_confirmed': 1.0, 'intent_change_trip_type': 1.0}]
2020-03-06 06:32:34 DEBUG    rasa.core.policies.memoization  - Launch DeLorean...
2020-03-06 06:32:34 DEBUG    rasa.core.policies.memoization  - Current tracker state [None, None, None, None, {'slot_cabin_type_0': 1.0, 'slot_trip_type_1': 1.0}, {'slot_cabin_type_0': 1.0, 'slot_trip_type_1': 1.0, 'prev_utter_ask_form_confirmed': 1.0}]
2020-03-06 06:32:34 DEBUG    rasa.core.policies.memoization  - There is no memorised next action
2020-03-06 06:32:34 DEBUG    rasa.core.policies.two_stage_fallback  - NLU confidence threshold met, confidence of fallback action set to core threshold (0.3).
2020-03-06 06:32:34 DEBUG    rasa.core.policies.form_policy  - There is no active form
2020-03-06 06:32:34 DEBUG    rasa.core.policies.mapping_policy  - There is no mapped action for the predicted intent, 'change_trip_type'.
2020-03-06 06:32:34 DEBUG    rasa.core.policies.ensemble  - Predicted next action using policy_0_EmbeddingPolicy
2020-03-06 06:32:34 DEBUG    rasa.core.processor  - Predicted next action 'action_listen' with confidence 0.99.
2020-03-06 06:32:34 DEBUG    rasa.core.processor  - Action 'action_listen' ended with events '[]'.
2020-03-06 06:32:34 DEBUG    rasa.core.lock_store  - Deleted lock for conversation 'default'.
Is this right?
 You requested a round trip search from Moldova to Paris in economy class
 Departure date: 2020-03-09T00:00:00.000-07:00; Return date: 2020-04-05T00:00:00.000-07:00
 Adults: 1; Children: 0; Infants: 0
 You will be contacted at test@gmail.com
Your input ->
5

Now that I am looking into it more…:

  1. The slot is being filled before two stage policy kicks in because slot autofilling is set to true in domains file and the slot name and entity name is the same. I guess this is intended behavior? But I am not sure that it is right for the slot to be filled before it asks user for affirmation
  2. I put a log for tracker.latest_message inside run method action_change_form_slots and got this:

{'text': '/change_trip_type', 'intent': {'name': 'change_trip_type', 'confidence': 1.0}, 'intent_ranking': [{'name': 'change_trip_type', 'confidence': 1.0}], 'entities': []}

You can see that no entities are getting passed into this action after user confirms intent