Querying for intent `None`

My bot starts to crash if the user inputs an empty string or simply a space character into the rasa form.

The name of the form is authenticate_form. The form asks for the first and last name of the user. If the user responses with his name, everything works fine:

However, if the user replies with a space character, we run into trouble:

The final error ist:

ValueError: None is not in list

My interpretation is that Rasa tried to find an intent None but couldn’t find anything. It is similar to this post.

Any ideas what’s going wrong here?

@Chatbot_Ra interesting issue, I will dig this issue more, never got this type of issue. Can you share the action.py file of this authentication code? and also the config.yml

Thank you very much.

The config-file:

# Configuration for Rasa NLU.
# https://rasa.com/docs/rasa/nlu/components/
language: en

pipeline:
# # No configuration for the NLU pipeline was provided. The following default pipeline was used to train your model.
# # If you'd like to customize it, uncomment and adjust the pipeline.
# # See https://rasa.com/docs/rasa/tuning-your-model for more information.
   - name: WhitespaceTokenizer
   - name: RegexFeaturizer
   - name: LexicalSyntacticFeaturizer
   - name: CountVectorsFeaturizer
   - name: CountVectorsFeaturizer
     analyzer: char_wb
     min_ngram: 1
     max_ngram: 4
   - name: DIETClassifier
     epochs: 100
     constrain_similarities: true
   - name: EntitySynonymMapper
   - name: ResponseSelector
     epochs: 200
     constrain_similarities: true
   - name: FallbackClassifier
     threshold: 0.2
     ambiguity_threshold: 0.1

# Configuration for Rasa Core.
# https://rasa.com/docs/rasa/core/policies/
policies:
# # No configuration for policies was provided. The following default policies were used to train your model.
# # If you'd like to customize them, uncomment and adjust the policies.
# # See https://rasa.com/docs/rasa/policies for more information.
   - name: MemoizationPolicy
   - name: RulePolicy
   - name: UnexpecTEDIntentPolicy
     max_history: 5
     epochs: 100
   - name: TEDPolicy
     max_history: 5 #5
     epochs: 100 #100
     constrain_similarities: true

The form asks multiple questions. I have removed some of them in order keep it short but without loosing the inner structure and logic:

class ValidateNameForm(FormValidationAction):
    """
    Class to simplify the process of validating extracted slots.
    In this case, you need to write functions named validate_<slot_name> for every extracted slot.
    """

    #  each regex follows name convention: regex_<slot_name>
    regex_name = r"^(?=.{2,100}$)[^\W\d_]+(?:[-' ][^\W\d_]+)*[.?!]?$"  # https://regex101.com/r/GSciL5/1
    regex_name_last = regex_name  # same as above
    regex_street_name = r"^(?:[A-Z] \d|[^\W\d_]{2,}\.?)(?:[- '’][^\W\d_]+\.?)*$"  # https://regex101.com/r/WRd1jx/7
    regex_contract_no = r"[1-9A-Za-z]{7,10}[.?!]?"  # https://regex101.com/r/TJWHSb/1
    regex_account_id = r"\d{10,10}"  # https://regex101.com/r/gvcaqs/1
    regex_stop = r"[sS]+[tT]+[oO]+[pP]+!+"  # https://regex101.com/r/iy1X4Q/1

    def __init__(self):
        self.form_slots = ["name", "name_last", "street_name", "contract_no", "account_id"]  # name of all slots
        self.misspelling_threshold = 2
        self.deactivated = False
        self.end_reason = None
        self.misspelling_counter = {slot: 0 for slot in self.form_slots}

    def name(self) -> Text:
        return "validate_authenticate_form"

    async def required_slots(
        self,
        slots_mapped_in_domain,
        dispatcher,
        tracker,
        domain
    ):
        """
        Method sets required slots depending on value of slot user_topic
        Parameters
        ----------
        slots_mapped_in_domain : List[str]
            slots that are mapped in domain file
        dispatcher : CollectingDispatcher
        tracker : Tracker
        domain : DomainDict

        Returns
        -------
        required_slots: List[str]
            list of required slots
        """
        user_topic = tracker.get_slot("user_topic")
        if tracker.slots.get("deactivated"):
            required_slots = []
            self.misspelling_counter = {slot: 0 for slot in slots_mapped_in_domain + ['contract_no', 'account_id']}
            self.deactivated = False
        else:
            if user_topic == "Frage stellen":
                required_slots = slots_mapped_in_domain + ["account_id"]
            else:
                required_slots = slots_mapped_in_domain + ["contract_no"]
        return required_slots

    def check_deactivation(self, slot_name, slot_value):
        """
        Method to check whether the form was deactivated either by too many misspellings, or by stop word
        Parameters
        ----------
        slot_name : str
            name of the slot of interest
        slot_value : str
            value of the slot of interest

        Returns
        -------
        ret_val : Dict[str, bool]
            dictionary with slot "deactivated" and its value (True or False)
        """
        ret_val = {"deactivated": False}
        if self.misspelling_counter[slot_name] > self.misspelling_threshold:
            self.deactivated = True
            self.end_reason = END_REASON_MISSPELLING
            ret_val = {"deactivated": True}

        if re.search(self.regex_stop, slot_value):
            self.deactivated = True
            self.end_reason = END_REASON_STOPWORD
            ret_val = {"deactivated": True}
        return ret_val

    def _validate_slot(self, slot_name, slot_value, dispatcher, tracker):
        """
        Generic method to validate a slot and it's corresponding value
        Parameters
        ----------
        slot_name : str
            name of the slot
        slot_value : str
            value of the slot
        dispatcher : CollectingDispatcher
            rasa dispatcher
        tracker : Tracker
            rasa tracker

        Returns
        -------
        Dict
            dictionary with validation result
        """
        print(f"validate_slot method: slot {slot_name} with value {slot_value}")

        misspelling_counter = 0
        if tracker.get_slot("misspelling_counter"): # misspelling_counter is stored in oracle db
            LOG.debug(f"misspelling counter found in db with value {tracker.get_slot('misspelling_counter')}")
            misspelling_counter = tracker.get_slot("misspelling_counter")
            self.misspelling_counter[slot_name] = misspelling_counter
        else: # no value for misspelling counter is found in oracle db
            self.misspelling_counter[slot_name] = misspelling_counter
            LOG.debug(f"no misspelling counter found in db starting with class attribute value {self.misspelling_counter[slot_name]}")

        # validate slot value with corresponding regex AND check if slot value is not a stop-word
        if (re.fullmatch(getattr(self, f"regex_{slot_name}"), slot_value)) and not \
                (re.search(self.regex_stop, slot_value)):
            self.misspelling_counter[slot_name] = 0

            if (slot_name == "name_last") and (tracker.slots.get("correct_slot") != "street_name") and (
                    tracker.slots.get("street_name") is None):
                dispatcher.utter_message(response="utter_thank_user_name")

            return {
                slot_name: slot_value,
                "deactivated": False,
                "misspelling_counter": self.misspelling_counter[slot_name]
            }

        else:  # slot value is invalid or user deactivated form (with stopword or 3 misspellings)
            self.misspelling_counter[slot_name] += 1

            requested_slot = slot_name
            deactivation_status = self.check_deactivation(slot_name, slot_value)

            if deactivation_status["deactivated"]:  # request was deactivated
                requested_slot = None
                self.deactivated = True
            else:  # Input of user doesn't match requirements
                if slot_name != "birthday":
                    # standart responses if user input is not matching validation
                    dispatcher.utter_message(response=f"utter_not_valid_{slot_name}", slot_value=slot_value)
                    dispatcher.utter_message(response='utter_tryAgain')

            return {
                slot_name: None,
                "deactivated": self.deactivated,
                "end_reason": self.end_reason,
                # to end the form action, set requested_slot to none if the max allowed fails are exceeded.
                "requested_slot": requested_slot,
                "misspelling_counter": self.misspelling_counter[slot_name]
            }

    async def extract_contract_no(
            self, dispatcher: CollectingDispatcher, tracker: Tracker, domain: Dict
    ) -> Dict[Text, Any]:
        if not tracker.slots['requested_slot'] == "contract_no":
            return {}
        text_of_last_user_message = tracker.latest_message.get("text")
        return {"contract_no": text_of_last_user_message}

    async def extract_account_id(
            self, dispatcher: CollectingDispatcher, tracker: Tracker, domain: Dict
    ) -> Dict[Text, Any]:
        if not tracker.slots['requested_slot'] == "account_id":
            return {}
        text_of_last_user_message = tracker.latest_message.get("text")
        return {"account_id": text_of_last_user_message}

    def validate_name(
        self,
        slot_value: Any,
        dispatcher: CollectingDispatcher,
        tracker: Tracker,
        domain: DomainDict,
    ) -> Dict[Text, Any]:
        """
        Method to validate slot 'name' --> rasa requires methods validate_<slot_name> for each slot in
        FormValidationAction
        """
        slot_name = "name"
        return self._validate_slot(slot_name, slot_value, dispatcher, tracker)

    def validate_name_last(
        self,
        slot_value: Any,
        dispatcher: CollectingDispatcher,
        tracker: Tracker,
        domain: DomainDict,
    ) -> Dict[Text, Any]:
        """
        Method to validate slot 'name_last' --> rasa requires methods validate_<slot_name> for each slot in
        FormValidationAction
        """
        slot_name = "name_last"
        return self._validate_slot(slot_name, slot_value, dispatcher, tracker)

    def validate_street_name(
        self,
        slot_value: Any,
        dispatcher: CollectingDispatcher,
        tracker: Tracker,
        domain: DomainDict,
    ) -> Dict[Text, Any]:
        slot_name = "street_name"
        return self._validate_slot(slot_name, slot_value, dispatcher, tracker)

    def validate_contract_no(
        self,
        slot_value: Any,
        dispatcher: CollectingDispatcher,
        tracker: Tracker,
        domain: DomainDict,
    ) -> Dict[Text, Any]:
        slot_name = "contract_no"
        return self._validate_slot(slot_name, slot_value, dispatcher, tracker)

    def validate_account_id(
        self,
        slot_value: Any,
        dispatcher: CollectingDispatcher,
        tracker: Tracker,
        domain: DomainDict,
    ) -> Dict[Text, Any]:
        slot_name = "account_id"
        return self._validate_slot(slot_name, slot_value, dispatcher, tracker)

The form action is rather complex and long. Not sure if you can pin down the problem.

I am working with rules and stories. Each intent has multiple examples and they are all more or less in balance.

Apart from this behaviour the bot works absolutly fine. It’s just that he doesn’t like empty strings.

@Chatbot_Ra main issue when user input the space it showing the error right in forms?

Not sure what you mean. The error will be triggered when the users responds to a question in the form. Here a fill picture of the error:

I mean what user input in the conversation? Did you try rasa interactive?

The user input is basically nothing. The bot asks for the name and the user just inputs nothing or just a space character.

It perfectly works if the user says sth. like Peter. But it crashes if the users responds with a space character.

@Chatbot_Ra have you mentioned some length of user input to validate if it’s not then the bot will return the fallback message.

@Chatbot_Ra How does your NLU model classify the empty input vs special character input?

@nik202 : Each slot will be validated with a corresponding regex. For example the regex for the first name and last name, respectively, are the same. The regex (shown here) doesn’t match if the user inputs a space character. I have not set any minimum length for the user input.

One important fact I did not mention yet: When running the bot in the shell (rasa shell) everything works perfectly fine!

@Juste: Not sure what you mean. I would say the the NLU doesn’t classify the empty input at all. I think this is the reason why it breaks.

@Chatbot_Ra I can see your regex is fine.

can you tell how you are running the bot if not using rasa shell? I’m confused now.