Define stories using form with a categorical Slot

naamtokyam · December 22, 2021, 3:40pm

Hi all, I am new to Rasa.

I have encountered an issue where I have a slot (and an entity) called contact_type which has a categorical type and a form contact_form which requires contact_type to be fulfilled.

What I eventually want to achieve is that:

Ask a user if they want the email address or phone number if the user didn’t specify either one initially.
To reprompt if the user doesn’t give a valid category value when asked
Repeat contact_library intent as many time as user want.

I am using rasa version 2.8.16, rasa-sdk version 2.8.3 and python version 3.8.12.

My settings are following:

config.yml

language: en

pipeline:
  - name: WhitespaceTokenizer
  - name: RegexFeaturizer
  - name: LexicalSyntacticFeaturizer
  - name: CountVectorsFeaturizer
  - name: CountVectorsFeaturizer
    analyzer: char_wb
    min_ngram: 1
    max_ngram: 4
  - name: DIETClassifier
    BILOU_flag: true
    epochs: 100
    constrain_similarities: true
  - name: EntitySynonymMapper
  - name: ResponseSelector
    epochs: 100
    constrain_similarities: true
  - name: FallbackClassifier
    threshold: 0.3
    ambiguity_threshold: 0.1
  - name: "DucklingEntityExtractor"
    url: "http://localhost:8000"
    dimensions: ["time"]

policies:
  - name: MemoizationPolicy
  - name: RulePolicy
  - name: UnexpecTEDIntentPolicy
    max_history: 5
    epochs: 100
  - name: TEDPolicy
    max_history: 5
    epochs: 100

domain.yml

intents:
- contact_library

entities:
- contact_type

slots:
  contact_type:
    type: categorical 
    influence_conversation: true 
    values: 
      - email 
      - phone

responses:
  utter_library_phone:
  - text: The library phone number is 00 11 2233
  utter_library_email:
  - text: The library email address is aabbcc@gmail.cam
  utter_ask_contact_type:
  - text: You can contact us by email or call. Which information would you like to know?


forms:
  contact_form:
    required_slots:
      contact_type:
        - type: from_entity
          entity: contact_type

nlu.yml

- intent: contact_library
  examples: | 
    - how can I contact the library?
    - gimme some contact info.
    - i wanna get the [number](contact_type) of the library.
    - what is the [number](contact_type) of the library?
    - how can i contact you?
    - how can i reach you?
    - contact info pleaze
    - i want to [call](contact_type) the library.
    - give me the [phone number](contact_type) of the library.
    - i want to [email](contact_type) the library 
    - i want to contact the library 
    - what's the contact [number](contact_type)
    - what is the available contact information?
    - give me the [email address](contact_type) of the library please.
    - gimme the [phone number](contact_type).
    - what's [telephone number](contact_type) i can call to the library?
    - i need a contact information of the library.
    - i would like to get your [electric mail address](contact_type).
    - what's ur [email](contact_type)?
    - would you be nice enough to tell me the library's [email](contact_type)?
    - do you know the [number](contact_type) of the library i can call?

- synonym: email
  examples: | 
    - email address
    - electric mail address
    - email account

- synonym: phone 
  examples: |
    - phone number
    - number
    - telephone
    - telephone number 
    - call
    - tell
    - tell number

So i have defined following simple stories which activate and loop the form until slot is correctly filled and depends on the category value, generate the response.

stories.yml

stories: 
- story: ask how to contact library - email 
  steps:
    - intent: contact_library
    - action: contact_form
    - active_loop: contact_form
    - slot_was_set:
      - contact_type: email
    - slot_was_set:
      - requested_slot: null
    - active_loop: null
    - action: utter_library_email 

- story: ask how to contack library - phone number
  steps: 
    - intent: contact_library
    - action: contact_form
    - active_loop: contact_form
    - slot_was_set:
      - requested_slot: null
    - slot_was_set:
      - contact_type: phone 
    - active_loop: null
    - action: utter_library_phone

However, when I ran rasa train, and test it with rasa shell/rasa interactive, it works for the first try if the user asks for contact information, it triggers and gets correct response, but after that, it keeps failing, triggering the default_action_fallback.

The debugging output is following for failed turn:

Your input ->  how can i contact the library?                                                                                                     
2021-12-22 16:48:13 DEBUG    rasa.core.lock_store  - Issuing ticket for conversation 'bb0fadcbea5a42afaa18113c627ef7ac'.
2021-12-22 16:48:13 DEBUG    rasa.core.lock_store  - Acquiring lock for conversation 'bb0fadcbea5a42afaa18113c627ef7ac'.
2021-12-22 16:48:13 DEBUG    rasa.core.lock_store  - Acquired lock for conversation 'bb0fadcbea5a42afaa18113c627ef7ac'.
2021-12-22 16:48:13 DEBUG    rasa.core.tracker_store  - Recreating tracker for id 'bb0fadcbea5a42afaa18113c627ef7ac'
2021-12-22 16:48:13 DEBUG    rasa.nlu.classifiers.diet_classifier  - There is no trained model for 'ResponseSelector': The component is either not trained or didn't receive enough training data.
2021-12-22 16:48:13 DEBUG    rasa.nlu.selectors.response_selector  - Adding following selector key to message property: default
2021-12-22 16:48:13 DEBUG    urllib3.connectionpool  - Starting new HTTP connection (1): localhost:8000
2021-12-22 16:48:13 DEBUG    urllib3.connectionpool  - http://localhost:8000 "POST /parse HTTP/1.1" 200 None
2021-12-22 16:48:13 DEBUG    rasa.core.processor  - Received user message 'how can i contact the library?' with intent '{'id': 2734815225362141272, 'name': 'contact_library', 'confidence': 0.9997653365135193}' and entities '[]'
2021-12-22 16:48:13 DEBUG    rasa.core.processor  - Logged UserUtterance - tracker now has 26 events.
2021-12-22 16:48:13 DEBUG    rasa.core.policies.memoization  - Current tracker state:
[state 1] user intent: contact_library | previous action name: action_listen
[state 2] user intent: contact_library | previous action name: contact_form | slots: {'contact_type': (1.0, 0.0, 0.0)}
[state 3] user intent: contact_library | previous action name: utter_library_email | slots: {'contact_type': (1.0, 0.0, 0.0)}
[state 4] user intent: contact_library | previous action name: action_listen | slots: {'contact_type': (1.0, 0.0, 0.0)}
2021-12-22 16:48:13 DEBUG    rasa.core.policies.memoization  - There is no memorised next action
2021-12-22 16:48:13 DEBUG    rasa.core.policies.rule_policy  - Current tracker state:
[state 1] user intent: contact_library | previous action name: action_listen
[state 2] user intent: contact_library | previous action name: contact_form | slots: {'contact_type': (1.0, 0.0, 0.0)}
[state 3] user intent: contact_library | previous action name: utter_library_email | slots: {'contact_type': (1.0, 0.0, 0.0)}
[state 4] user intent: thanks | previous action name: action_listen | slots: {'contact_type': (1.0, 0.0, 0.0)}
[state 5] user intent: thanks | previous action name: utter_yourwelcome | slots: {'contact_type': (1.0, 0.0, 0.0)}
[state 6] user text: how can i contact the library? | previous action name: action_listen | slots: {'contact_type': (1.0, 0.0, 0.0)}
2021-12-22 16:48:13 DEBUG    rasa.core.policies.rule_policy  - There is no applicable rule.
2021-12-22 16:48:13 DEBUG    rasa.core.policies.rule_policy  - Current tracker state:
[state 1] user intent: contact_library | previous action name: action_listen
[state 2] user intent: contact_library | previous action name: contact_form | slots: {'contact_type': (1.0, 0.0, 0.0)}
[state 3] user intent: contact_library | previous action name: utter_library_email | slots: {'contact_type': (1.0, 0.0, 0.0)}
[state 4] user intent: thanks | previous action name: action_listen | slots: {'contact_type': (1.0, 0.0, 0.0)}
[state 5] user intent: thanks | previous action name: utter_yourwelcome | slots: {'contact_type': (1.0, 0.0, 0.0)}
[state 6] user intent: contact_library | previous action name: action_listen | slots: {'contact_type': (1.0, 0.0, 0.0)}
2021-12-22 16:48:13 DEBUG    rasa.core.policies.rule_policy  - There is no applicable rule.
2021-12-22 16:48:13 DEBUG    rasa.core.policies.ensemble  - Made prediction using user intent.
2021-12-22 16:48:13 DEBUG    rasa.core.policies.ensemble  - Added `DefinePrevUserUtteredFeaturization(False)` event.
2021-12-22 16:48:13 DEBUG    rasa.core.policies.ensemble  - Predicted next action using policy_1_RulePolicy.
2021-12-22 16:48:13 DEBUG    rasa.core.processor  - Predicted next action 'action_default_fallback' with confidence 0.30.
2021-12-22 16:48:13 DEBUG    rasa.core.processor  - Policy prediction ended with events '[<rasa.shared.core.events.DefinePrevUserUtteredFeaturization object at 0x7fbaddafba60>]'.
2021-12-22 16:48:13 DEBUG    rasa.core.processor  - Action 'action_default_fallback' ended with events '[BotUttered('Sorry, I can't help you.', {"elements": null, "quick_replies": null, "buttons": null, "attachment": null, "image": null, "custom": null}, {"utter_action": "utter_default"}, 1640188093.4870102), <rasa.shared.core.events.UserUtteranceReverted object at 0x7fbadda8a730>]'.
2021-12-22 16:48:13 DEBUG    rasa.core.processor  - Current slot values: 
	time: None
	contact_type: email
	requested_slot: None
	session_started_metadata: None
2021-12-22 16:48:13 DEBUG    rasa.core.policies.memoization  - Current tracker state:
[state 1] user intent: contact_library | previous action name: action_listen
[state 2] user intent: contact_library | previous action name: contact_form | slots: {'contact_type': (1.0, 0.0, 0.0)}
[state 3] user intent: contact_library | previous action name: utter_library_email | slots: {'contact_type': (1.0, 0.0, 0.0)}
[state 4] user intent: contact_library | previous action name: action_listen | slots: {'contact_type': (1.0, 0.0, 0.0)}
2021-12-22 16:48:13 DEBUG    rasa.core.policies.memoization  - There is no memorised next action
2021-12-22 16:48:13 DEBUG    rasa.core.policies.rule_policy  - Current tracker state:
[state 1] user intent: contact_library | previous action name: action_listen
[state 2] user intent: contact_library | previous action name: contact_form | slots: {'contact_type': (1.0, 0.0, 0.0)}
[state 3] user intent: contact_library | previous action name: utter_library_email | slots: {'contact_type': (1.0, 0.0, 0.0)}
[state 4] user intent: thanks | previous action name: action_listen | slots: {'contact_type': (1.0, 0.0, 0.0)}
[state 5] user intent: thanks | previous action name: utter_yourwelcome | slots: {'contact_type': (1.0, 0.0, 0.0)}
2021-12-22 16:48:13 DEBUG    rasa.core.policies.rule_policy  - There is a rule for the next action 'action_listen'.
2021-12-22 16:48:13 DEBUG    rasa.core.policies.ensemble  - Predicted next action using policy_1_RulePolicy.
2021-12-22 16:48:13 DEBUG    rasa.core.processor  - Predicted next action 'action_listen' with confidence 1.00.
2021-12-22 16:48:13 DEBUG    rasa.core.processor  - Policy prediction ended with events '[]'.
2021-12-22 16:48:13 DEBUG    rasa.core.processor  - Action 'action_listen' ended with events '[]'.
2021-12-22 16:48:13 DEBUG    rasa.core.lock_store  - Deleted lock for conversation 'bb0fadcbea5a42afaa18113c627ef7ac'.
Sorry, I can't help you.

I also tried defining the rules that activate&deactivate form (according to Form), but then it gave me a following error:

rules.yml

- rule: Activate contact_form
  steps:
  - intent: contact_library
  - action: contact_form
  - active_loop: contact_form

- rule: Deactivate contact_form - phone
  condition:
  - active_loop: contact_form
  steps:
  - action: contact_form
  - active_loop: null
  - slot_was_set:
    - contact_type: phone
  - action: utter_library_phone

- rule: Deactivate contact_form - email
  condition:
  - active_loop: contact_form
  steps:
  - action: contact_form
  - active_loop: null
  - slot_was_set:
    - contact_type: email
  - action: utter_library_email

InvalidRule: 
Incomplete rules found🚨

- the action 'contact_form' in rule 'Activate contact_form' does not set some of the slots that it sets in other rules. Slots not set in rule 'Activate contact_form': 'contact_type'. Please update the rule with an appropriate slot or if it is the last action add 'wait_for_user_input: false' after this action.
Please note that if some slots or active loops should not be set during prediction you need to explicitly set them to 'null' in the rules.
You can find more information about the usage of rules at https://rasa.com/docs/rasa/rules.

which I am not sure how to fix.

Is the usage of the form appropriate? Or should I not use form and handle the non-correct values with custom utterances?

What should be the correct way to implement this to achieve what I want?

Thank you!

nik202 · December 22, 2021, 4:24pm

@naamtokyam You can use FormValidation for the same use case. For the first screenshot of the conversation, try to delete the older models and re-train them. As you have mentioned the fallback classifier in the config, it’s triggering right.

naamtokyam · December 22, 2021, 10:47pm

Thank you for your reply and suggestions @nik202!

Following your advice, I have tried to add validation form ValidateContactForm and also a custom action ActionTellContactInfo since I want to reset the slot after the story ends as I no longer need to keep the slot value from this state.

class ActionTellContactInfo(Action):
    def name(self):
        return "action_tell_contact_info"
    
    def run(self, dispatcher: CollectingDispatcher,
            tracker: Tracker,
            domain: Dict[Text, Any]) -> List[Dict[Text, Any]]:
        slot_value = tracker.get_slot("contact_type")

        logger.debug(f"naamtokyam: action_tell_contact_info is called.")

        if slot_value == "phone":
            dispatcher.utter_message(response="utter_library_phone")
        else:
            dispatcher.utter_message(response="utter_library_email")

        return [SlotSet("contact_type", None)]

class ValidateContactForm(FormValidationAction):
    def name(self) -> Text:
        return "validate_contact_form"

    def validate_contact_type(
        self,
        slot_value: Any,
        dispatcher: CollectingDispatcher,
        tracker: Tracker,
        domain: DomainDict,
    ) -> Dict[Text, Any]:
        logger.debug(f"naamtokyam: ValidateContactForm is called. slot_value is {slot_value}")

        ans = slot_value
        if ans not in ["email", "phone"]:
            dispatcher.utter_message(text=f"Sorry, we only have information of email or phone number. Which one would you like to know?")
            ans = None
        return {"contact_type": ans}

I then erased old models, retrained a new one, yet some problems still persist.

For example, when I tested with an invalid value for the contact_type slot (e.g. typo, random words, etc.), although the ValidateContactForm should be run, it seems like it’s not called because my validation form utter_message of "Sorry, we only have information of email or phone number. Which one would you like to know?" was not asked as well as my log was not printed. Instead, it presents and triggers other intents (likely fallback), then maybe re-ask the utter_ask_contact_type until a valid category value is inputted.
rasa31110×548 62.1 KB

the output of action debug when invalid input is used

2021-12-22 23:28:54 DEBUG    rasa_sdk.executor  - Received request to run 'validate_contact_form'
2021-12-22 23:28:54 DEBUG    rasa_sdk.executor  - Finished running 'validate_contact_form'

the output of action debug when valid input (this case email) is used

2021-12-22 23:38:39 DEBUG    rasa_sdk.executor  - Received request to run 'validate_contact_form'
2021-12-22 23:38:39 DEBUG    actions.actions  - naamtokyam: ValidateContactForm is called. slot_value is email
2021-12-22 23:38:39 DEBUG    rasa_sdk.executor  - Finished running 'validate_contact_form'
2021-12-22 23:38:39 DEBUG    rasa_sdk.executor  - Received request to run 'action_tell_contact_info'
2021-12-22 23:38:39 DEBUG    actions.actions  - naamtokyam: action_tell_contact_info is called.
2021-12-22 23:38:39 DEBUG    rasa_sdk.executor  - Finished running 'action_tell_contact_info'

Why is validation not working in this case?

When the contact_form is triggered, the user reply with “call”, however this utter is not recognized and triggered utter_default_fallback even though listed in synonym for the category value for phone. Is there a reason for this behavior?

Thanks a lot for taking a look at this issue!

nik202 · December 22, 2021, 10:55pm

@naamtokyam Hope you provided good training examples in the nlu ? can you tell how many examples currently you had for the each intents for your code? I guess code is working but its entering into fallback as he get low confidence value sometime. Even try investigate using rasa interactive that will also give you clear idea what is happening.

naamtokyam · December 22, 2021, 11:10pm

Assuming you are referring to the second question in my previous reply, I have relatively small (21 for example contact_library intent as listed in my first post of this question). So you are suggesting I should increase the example size and that should help?

Also any ideas with the first question with the validation form?

Thanks!

nik202 · December 22, 2021, 11:17pm

@naamtokyam Its good amount of nlu but do make same for other intent too, is that you also set the default fallback in your config.yml file? Try use rasa interactive.

naamtokyam · December 23, 2021, 12:32pm

Thanks @nik202! Will use rasa interactive.

Topic		Replies	Views
How to use categorical slots in rasa 3.1 version? (Syntax) Rasa Open Source	2	355	July 26, 2023
Categorical Slot Validation Rasa Open Source	5	317	December 5, 2022
Slot_was_set event with categorical slots Rasa Open Source	1	531	December 11, 2020
Categorical slots and stories Feedback on Rasa Open Source	4	669	October 24, 2021
Categorical Slots with slot_was_set stories Rasa Open Source	7	2030	July 7, 2021

Define stories using form with a categorical Slot

Related Topics