How do I slot fill multiple slots from a predicted “multi-intent” (e.g. intent_X + intent_Y) from within a Form?
Intent_X and and Intent_Y are registered in a FormAction, and each fill a corresponding slot and entity when they are predicted as separate intents. However, I would like to allow a user to update multiple slots at once. How would I change the bot to fill both corresponding slots from an intent that is a combination of two intents?
Does anyone know anything about the status of this feature in Rasa? Currently my form breaks when this situation is present. However, this seems like it would be a basic feature. Any advice or information about incorporation of this feature in Rasa would be most appreciated.
Hi @argideritzalpea - you will be able to do with by writing a custom from_multi_intent extractor, see here
If you find this useful I think it could be a great contribution to make it a native feature!
Hi @amn41 , thanks for your response! I was taking a look at the code - wouldn’t writing a custom from_multi_intent() extractor necessitate the user to define custom mappings for all desired combinations of intents with different multi-intent mappings? It is also unclear to me how the returned objects in the extractor methods would be able to handle multiple intents. I wondered if there might be a more general approach to split any intent with a “+” and slot fill according to each of the sub-intents with the from_intent() extractors for each of those?
can you maybe give a more concrete example of what you’re trying to achieve? maybe if you give one or two example multi-intents you’d like to parse it’ll be easier to make a recommendation
Hi Alan, I will outline a full example below:
Here are a few of my entities and intents, and my form, “count_episode”.
- returnStatus
- number
- timeAbs
- timeAgo
- source
- yearBorn
- zipCode
- time
intents:
- affirm
- deny
- hello
- goodbye
- why
- repeat_please
- skip_please
- inform_countTime
- inform_peopleCount
- inform_vehicleCount
- inform_partyVehics
- inform_returnStatus
- inform_trailVisits
- inform_zipCode
- inform_source
- inform_yearBorn
- inform_howLong
- inform_seenBefore
- inform_seenPastDay
- inform_ambigVehic
- inform_ambigNumber
forms:
- count_episode
Here is my “count_episode” action. The first part is a simple logic that adds more questions depending on prior answers in the form. However, our interest is in the slot map that is defined in slot_mappings():
class ActionCountEpisode(FormAction):
RANDOMIZE = False
@staticmethod
def required_slots(tracker):
base_set = ['seenBefore', "vehicleCount", "countTime", "peopleCount",
"partyVehics", "trailVisits", "source", "returnStatus"]
final_set = base_set
#print("return status: ", tracker.get_slot('returnStatus'))
if tracker.get_slot('returnStatus') == 'return':
final_set += ["howLong"]
if tracker.get_slot('seenBefore') == True:
final_set.insert(1, "seenPastDay")
if tracker.get_slot('seenBefore') == False:
final_set += ["zipCode"]
final_set += ["yearBorn"]
#print(final_set)
return final_set
def name(self):
return 'count_episode'
def slot_mappings(self):
return {"vehicleCount": [self.from_entity(entity="number", intent="inform_vehicleCount"),
self.from_entity(entity="number", intent="inform_ambigVehic"),
self.from_entity(entity="number", intent="inform_ambigNumber")],
"countTime": [self.from_entity(entity="timeAbs", intent="inform_countTime"),
self.from_entity(entity="timeAbs", intent="inform_ambigNumber"),
self.from_entity(entity="timeAgo", intent="inform_countTime"),
self.from_entity(entity="timeAgo", intent="inform_ambigNumber"),
self.from_entity(entity="time", intent="inform_countTime"),
self.from_entity(entity="time", intent="inform_ambigNumber"),
self.from_entity(entity="number", intent="inform_countTime"),
self.from_entity(entity="number", intent="inform_ambigNumber")],
"peopleCount": [self.from_entity(entity="number", intent="inform_peopleCount"),
self.from_entity(entity="number", intent="inform_ambigNumber")],
"partyVehics": [self.from_entity(entity="number", intent="inform_partyVehics"),
self.from_entity(entity="number", intent="inform_ambigVehic"),
self.from_entity(entity="number", intent="inform_ambigNumber")],
"source": self.from_entity(entity="source", intent="inform_source"),
"trailVisits": [self.from_entity(entity="number", intent="inform_trailVisits"),
self.from_entity(entity="number", intent="inform_ambigNumber")],
"returnStatus": self.from_entity(entity="returnStatus", intent="inform_returnStatus"),
"zipCode": self.from_entity(entity="zipCode", intent="inform_zipCode"),
"yearBorn": self.from_entity(entity="yearBorn", intent="inform_yearBorn"),
"seenBefore": [self.from_intent(intent='affirm', value=True),
self.from_intent(intent='deny',value=False)],
"howLong": self.from_entity(entity="timeAgo", intent="inform_howLong"),
"seenPastDay": [self.from_intent(intent='affirm', value=True),
self.from_intent(intent='deny',value=False)]
}
def submit(self, dispatcher, tracker, domain):
return []
My desire is that there might be a switch or parameter that, when a multi-intent is detected, (e.g. inform_countTime+inform_peopleCount), the utterance would search the utterance to fill both potential slots defined in the mapping. However, I don’t want to have to manually code all all combinations of these potential differences. If the intent is returned as inform_countTime+inform_peopleCount, I want Rasa to do everything it can based on the slot mappings I have already defined for each of these component intents, and update the slots according to the entities it finds in the utterance that would be mapped separately to each of these intents.
The whole idea behind this is in case someone supplies extra information in the form, or perhaps answers the form out of turn, the slots associated with the entities in a particular intent can still be filled, even several in the same utterance. This would reduce duplication in the conversation and increase apparent fluency in the dialogue.
thanks, it’s much clearer what you want to achieve now. Is it an option to replace all the inform_x intents with a single inform intent and use entities instead to extract peopleCount etc? If those entities are extracted the corresponding slots will automatically get set
Unfortunately, I don’t think so. The entities are very similar in form and context, and that is one of the reasons I was using the Form Action class to begin with. If I just used entities the recognition would be confused.
may I ask what the form currently does if let’s say two intents are triggered:
inform_yearBorn+inform_vehicleCount
with: “I was born in 1982 and I need 2 vehicles” (just for an example)? As of your action, the following should happen:
"yearBorn": self.from_entity(entity="yearBorn", intent="inform_yearBorn"),
"vehicleCount": self.from_entity(entity="number", intent="inform_vehicleCount")
If I get it right, you want both slots filled at once. Could you please post the verbose output of rasa at this point?
Regards
Here is an example of the debug output. It detects two entities but fills / validates only the “requested slot”:
Can you let us know how many vehicles are in the parking lot?
2019-07-10 04:44:28 DEBUG rasa_core.policies.ensemble - Predicted next action using policy_0_FormPolicy
2019-07-10 04:44:28 DEBUG rasa_core.processor - Predicted next action 'action_listen' with prob 1.00.
2019-07-10 04:44:28 DEBUG rasa_core.processor - Action 'action_listen' ended with events '[]'
127.0.0.1 - - [2019-07-10 04:44:28] "POST /webhooks/rest/webhook?stream=true&token= HTTP/1.1" 200 261 0.055524
There are 5 vehicles at 5pm
2019-07-10 04:44:44 DEBUG rasa_core.tracker_store - Recreating tracker for id 'default'
2019-07-10 04:44:44 DEBUG rasa_core.processor - Received user message 'There are 5 vehicles at 5pm' with intent '{'name': 'inform_vehicleCount+inform_countTime', 'confidence': 0.8991318941116333}' and entities '[{'start': 10, 'end': 11, 'text': '5', 'value': 5, 'confidence': 1.0, 'additional_info': {'value': 5, 'type': 'value'}, 'entity': 'number', 'extractor': 'ner_duckling_http'}, {'start': 21, 'end': 27, 'text': 'at 5pm', 'value': '2019-07-10T17:00:00.000-07:00', 'confidence': 1.0, 'additional_info': {'values': [{'value': '2019-07-10T17:00:00.000-07:00', 'grain': 'hour', 'type': 'value'}, {'value': '2019-07-11T17:00:00.000-07:00', 'grain': 'hour', 'type': 'value'}, {'value': '2019-07-12T17:00:00.000-07:00', 'grain': 'hour', 'type': 'value'}], 'value': '2019-07-10T17:00:00.000-07:00', 'grain': 'hour', 'type': 'value'}, 'entity': 'time', 'extractor': 'ner_duckling_http'}, {'start': 10, 'end': 11, 'value': '5', 'entity': 'number', 'confidence': 0.979454478104968, 'extractor': 'ner_crf'}, {'start': 24, 'end': 27, 'value': '5 pm', 'entity': 'timeAbs', 'confidence': 0.9990286885606756, 'extractor': 'ner_crf'}]'
2019-07-10 04:44:44 DEBUG rasa_core.processor - Logged UserUtterance - tracker now has 18 events
2019-07-10 04:44:44 DEBUG rasa_core.processor - Current slot values:
age: None
can_use_spacy: None
countTime: None
current_api: None
data_stored: None
entity_extractor: None
feedback_message: None
feedback_value: None
howLong: None
language: None
name: None
nlu_part: None
onboarding: None
package_manager: None
partyVehics: None
peopleCount: None
problem_description: None
product: None
requested_slot: vehicleCount
returnStatus: None
search_results: None
seenBefore: False
seenPastDay: None
shown_privacy: None
source: None
step: None
suggestion: None
trailVisits: None
unknown_nlu_part: None
unknown_product: None
vehicleCount: None
yearBorn: None
zipCode: None
2019-07-10 04:44:44 DEBUG rasa_core.policies.form_policy - There is an active form 'count_episode'
2019-07-10 04:44:44 DEBUG rasa_core.policies.memoization - Current tracker state [{'prev_action_listen': 1.0, 'intent_hello': 1.0}, {'prev_action_setDefaultSlotValues': 1.0, 'intent_hello': 1.0}, {'prev_utter_hello': 1.0, 'intent_hello': 1.0}]
2019-07-10 04:44:44 DEBUG rasa_core.policies.memoization - There is a memorised next action '40'
2019-07-10 04:44:44 DEBUG rasa_core.policies.ensemble - Predicted next action using policy_0_FormPolicy
2019-07-10 04:44:44 DEBUG rasa_core.processor - Predicted next action 'count_episode' with prob 1.00.
2019-07-10 04:44:44 DEBUG rasa_core.actions.action - Calling action endpoint to run action 'count_episode'.
127.0.0.1 - - [2019-07-10 04:44:44] "POST /webhook HTTP/1.1" 200 987 0.003345
2019-07-10 04:44:44 DEBUG rasa_core.processor - Action 'count_episode' ended with events '["SlotSet(key: vehicleCount, value: [5, '5'])", 'SlotSet(key: requested_slot, value: countTime)']'
2019-07-10 04:44:44 DEBUG rasa_core.processor - Bot utterance 'BotUttered(text: What time did you count vehicles?, data: {
"elements": null,
"buttons": null,
"attachment": null
})'
2019-07-10 04:44:44 DEBUG rasa_core.policies.form_policy - There is an active form 'count_episode'
2019-07-10 04:44:44 DEBUG rasa_core.policies.memoization - Current tracker state [{'prev_action_listen': 1.0, 'intent_hello': 1.0}, {'prev_action_setDefaultSlotValues': 1.0, 'intent_hello': 1.0}, {'prev_utter_hello': 1.0, 'intent_hello': 1.0}]
2019-07-10 04:44:44 DEBUG rasa_core.policies.memoization - There is a memorised next action '40'
What time did you count vehicles?
2019-07-10 04:44:44 DEBUG rasa_core.policies.ensemble - Predicted next action using policy_0_FormPolicy
2019-07-10 04:44:44 DEBUG rasa_core.processor - Predicted next action 'action_listen' with prob 1.00.
2019-07-10 04:44:44 DEBUG rasa_core.processor - Action 'action_listen' ended with events '[]'
127.0.0.1 - - [2019-07-10 04:44:44] "POST /webhooks/rest/webhook?stream=true&token= HTTP/1.1" 200 233 0.178769
what?
2019-07-10 04:45:47 DEBUG rasa_core.tracker_store - Recreating tracker for id 'default'
2019-07-10 04:45:47 DEBUG rasa_core.processor - Received user message 'what?' with intent '{'name': 'repeat_please', 'confidence': 0.9081987738609314}' and entities '[]'
2019-07-10 04:45:47 DEBUG rasa_core.processor - Logged UserUtterance - tracker now has 24 events
2019-07-10 04:45:47 DEBUG rasa_core.processor - Current slot values:
age: None
can_use_spacy: None
countTime: None
current_api: None
data_stored: None
entity_extractor: None
feedback_message: None
feedback_value: None
howLong: None
language: None
name: None
nlu_part: None
onboarding: None
package_manager: None
partyVehics: None
peopleCount: None
problem_description: None
product: None
requested_slot: countTime
returnStatus: None
search_results: None
seenBefore: False
seenPastDay: None
shown_privacy: None
source: None
step: None
suggestion: None
trailVisits: None
unknown_nlu_part: None
unknown_product: None
vehicleCount: [5, '5']
yearBorn: None
zipCode: None
2019-07-10 04:45:47 DEBUG rasa_core.policies.form_policy - There is an active form 'count_episode'
2019-07-10 04:45:47 DEBUG rasa_core.policies.memoization - Current tracker state [{'prev_action_listen': 1.0, 'i
ntent_hello': 1.0}, {'prev_action_setDefaultSlotValues': 1.0, 'intent_hello': 1.0}, {'prev_utter_hello': 1.0, 'inten
t_hello': 1.0}]
2019-07-10 04:45:47 DEBUG rasa_core.policies.memoization - There is a memorised next action '40'
2019-07-10 04:45:47 DEBUG rasa_core.policies.ensemble - Predicted next action using policy_0_FormPolicy
2019-07-10 04:45:47 DEBUG rasa_core.processor - Predicted next action 'count_episode' with prob 1.00.
2019-07-10 04:45:47 DEBUG rasa_core.actions.action - Calling action endpoint to run action 'count_episode'.
that’s interesting. Have you tried to change the slot mapping of countTime to:
"countTime": [self.from_entity(entity="time", intent="inform_vehicleCount+inform_countTime")
Because timeAbs doesn’t seem to be extracted rather than time and the slot mapping can’t work if you only allow the extraction with the corresponding single intent.
@amn41 Am I wrong here? Naming an intent in slot_mapping alongside an entity is a hard binding, isn’t it?
Regards Julian
My intent is that all combinations of these intents need not be hardcoded in manually. I hope that Rasa recognizes that this is a logical feature to introduce to FormActions after having released multi-intents. If I had a bot or form with many questions, the solution you suggest @JulianGerhard would be intractable. Indeed, it would be a pain to do so even with the slots I currently have, and my bot isn’t too complicated.
@argideritzalpea the forms are definitely defined with this use case in mind. Typically though the multi intent feature is used for saying two functionally different things. When you have two slot values provided in a single utterance, that’s usually covered by having one intent with multiple entities. That is handled natively by the form because the slots get set automatically provided they have the same name as the entity.
I’m not sure what the exact difficulty is with this solution that makes it necessary to use multi-intents, but that’s not an edge case we ever had in mind, especially in the case of NxN different multi-intents
@amn41 Based on my understanding of your comment, you might recommend a single “inform_data” intent as opposed to multiple intent-based mappings to fill different slots. However, in my bot, several slots are filled by entities that are identical in form. In these situations, it is not the entity identification but the intent that is useful for determining the slot fillings. If I put all entities in a single intent (inform_data) intent with many possible mappings to slots according to entities, there will not be enough context to disambiguate which slot to fill.
Keeping the inform intents separated also provides an advantage in introducing custom logic according to the type of information provided, whether or not the slot requested was matching what the interlocutor offered.
hi @argideritzalpea - I see where you are coming from. From what I understand though, what you want to do is already possible with a form, it’s just cumbersome to write all of thee required functions. Perhaps you could use a macro or something to generate that code programatically, or create your own subclass of the Form which allows this behaviour. If there is a neat solution I’d be interested in seeing it as a possible contribution to Rasa!
It also appears that multi-intents are returned as “intent_1+intent2”. Since they need to be hardcoded in the slot mappings, it is necessary to know the order in which they are returned. Is the order they are listed in the multi-intent name determined by order of appearance in the domain.py file?
@argideritzalpea - for a multi-intent A+B to be identified, there has to be at least one example of that multi-intent in your training data. Otherwise, we would have to generate and check all N*(N-1)/2 possible multi-intents at prediction time, which would be very slow. So the order will be given as it is in your training data (and domain file): A+B. You should not have both A+B and B+A in your training data, because they will get identical scores from the model and so there is no guarantee which one you’ll get at prediction time
I just want to flag the case as still being a potential improvement - that there be an option built in to allow multi-intents to trigger the actions of each of their constituent intents.
A follow up to my question above - even if I defined slot mappings to accept entities for every dual intent explicitly as you suggested, wont the form still only fill the “requested” slot? There is only one requested slot in the form. Since there is only one requested slot, is there no way to allow multiple slot fillings at once in a form?
i also have an issue with that…you did the multi selection of intents and entities in custom action but now slot mapping is deprecated to open source and done inside domain.yaml file so how can we do a multiple intent selection for one slot inside a formaction…assume i have a question to ask a user with button of yes or no…