Rasa 2.0, how are forms working?

petasis · October 3, 2020, 7:59pm

Hi all,

I am trying to port my bot from rasa 1.x to 2.x, but it has turned out a difficult process. The documentation is sparse, or I cannot understand it.

I have a form:

forms:
  flight_departure_form:
    sl_frm_flight_departure_info_city:
    - type: from_entity
      entity: city
      intent:
        - flight_departure_info
        - inform

And I am asking: “I am flying to Berlin”:

2020-10-03 22:51:07 DEBUG    rasa.core.processor  - Logged UserUtterance - tracker now has 10 events.
2020-10-03 22:51:07 DEBUG    rasa.core.policies.memoization  - Current tracker state [{}, {'user': {'intent': 'flight_departure_info', 'entities': ('flying', 'city', 'language')}, 'slots': {'language': (1.0, 0.0, 0.0)}, 'prev_action': {'action_name': 'action_listen'}}]
2020-10-03 22:51:07 DEBUG    rasa.core.policies.memoization  - There is no memorised next action
2020-10-03 22:51:07 DEBUG    rasa.core.policies.rule_policy  - Current tracker state: [{}, {'user': {'intent': 'flight_departure_info', 'entities': ('flying', 'city', 'language')}, 'slots': {'language': (1.0, 0.0, 0.0)}, 'prev_action': {'action_name': 'action_listen'}}]
2020-10-03 22:51:07 DEBUG    rasa.core.policies.rule_policy  - There is a rule for the next action 'flight_departure_form'.
2020-10-03 22:51:07 DEBUG    rasa.core.policies.ensemble  - Predicted next action using policy_2_RulePolicy
2020-10-03 22:51:07 DEBUG    rasa.core.processor  - Predicted next action 'flight_departure_form' with confidence 1.00.
2020-10-03 22:51:07 DEBUG    rasa.core.actions.forms  - Activated the form 'flight_departure_form'.
2020-10-03 22:51:07 DEBUG    rasa.core.actions.forms  - No pre-filled required slots to validate.
2020-10-03 22:51:07 DEBUG    rasa.core.actions.forms  - Validating user input 'UserUttered(text: ?????? ????????????????, intent: {'id': 6883722366486273833, 'name': 'flight_departure_info', 'confidence': 0.9999984502792358}, entities: [{'start': 0, 'end': 12, 'value': 'el', 'entity': 'language', 'extractor': 'LanguageDetection', 'confidence': 0.8, 'processors': []}, {'entity': 'city', 'start': 4, 'end': 12, 'value': '????????????????', 'extractor': 'RegexEntityExtractor'}, {'entity': 'flying', 'start': 0, 'end': 3, 'value': '??????', 'extractor': 'RegexEntityExtractor'}, {'entity': 'flying', 'start': 0, 'end': 3, 'confidence_entity': 0.9989129304885864, 'value': '??????', 'extractor': 'DIETClassifier'}, {'entity': 'city', 'start': 4, 'end': 12, 'confidence_entity': 0.9877170920372009, 'value': '????????????????', 'extractor': 'DIETClassifier'}])'.
2020-10-03 22:51:07 DEBUG    rasa.core.actions.forms  - Validating extracted slots: {}
2020-10-03 22:51:07 DEBUG    rasa.core.actions.action  - Calling action endpoint to run action 'validate_flight_departure_form'.
2020-10-03 22:51:07 DEBUG    rasa.core.actions.forms  - Request next slot 'sl_frm_flight_departure_info_city'
2020-10-03 22:51:07 DEBUG    rasa.core.actions.action  - Calling action endpoint to run action 'action_ask_sl_frm_flight_departure_info_city'.
2020-10-03 22:51:07 DEBUG    rasa.core.processor  - Action 'flight_departure_form' ended with events '[<rasa.shared.core.events.ActiveLoop object at 0x7fd06006dfd0>, <rasa.shared.core.events.SlotSet object at 0x7fd06006dbe0>]'.

I cannot understand. There is a city detected in the user input, why the slot is not filled? And why the form has to ask again for filling the slot?

Let alone the fact, that despite the form wants to ask for a city, the message is never asked.

petasis · October 3, 2020, 9:56pm

After spending a couple of hours, I understood why it does not work:

In FormAction() class, there is the extract_other_slots() method, run when the form activates. This decides what slots will be filled. It uses method entity_is_desired(), which does:

if (
            requested_slot_mapping.get("role") is None
            and requested_slot_mapping.get("group") is None
        ) or entity_type_of_slot_to_fill != requested_slot_mapping.get("entity"):
            slot_fulfils_entity_mapping = False

Which simply means, that the slot will be filled only if the role/group attributes have been defined, or if the slot has the same name as the entrity. Else, it will fail.

So, a simple slot definition like this:

sl_frm_flight_departure_info_city:
    - type: from_entity
      entity: city

will NEVER be initialised from the user input. And this is not in the documentation!

petasis · October 3, 2020, 11:58pm

I have updated to rc3, and now I get an error:

  File "rasa/core/actions/forms.py", line 142, in _create_unique_entity_mappings
    for slot_mapping in slot_mappings:
TypeError: 'NoneType' object is not iterable

This is caused if a slot mapping does not have a type…

petasis · October 4, 2020, 12:18am

After removing the slots causing problems from the form, now validation does not work:

2020-10-04 03:11:20 DEBUG    rasa.core.actions.forms  - Validating extracted slots: {'sl_frm_flight_departure_info_city': '????????????????'}
2020-10-04 03:11:20 DEBUG    rasa.core.actions.action  - Calling action endpoint to run action 'validate_flight_departure_form'.

This is the code in the validation action:

def run(
        self, dispatcher: CollectingDispatcher, tracker: Tracker, domain: Dict
    ) -> List[EventType]:
        extracted_slots: Dict[Text, Any] = tracker.form_slots_to_validate()

        validation_events = []
        print("FlightForm::run()===================================================", flush=True)
        print(self.name())
        print("extracted_slots:", extracted_slots, flush=True)

The output is:

FlightForm::run()===================================================
validate_flight_departure_form
extracted_slots: {}

tracker.form_slots_to_validate() returns an empty list!

petasis · October 4, 2020, 11:37pm

github.com/RasaHQ/rasa-sdk

`tracker.form_slots_to_validate` returns no slots on activation

opened 06:53PM - 15 Sep 20 UTC

closed 10:28AM - 05 Oct 20 UTC

NetBKarine

type:bug

area:rasa-sdk

**Rasa version**: 2.0.0a4 **Rasa SDK version**: 2.0.0a4 **Python version**: 3.7.8 **Operating system** (windows, osx, ...): Ubuntu 18.04 **Issue**: tracker.form_slots_to_validate() ignores slots to validate when called on form activation. I didn't try it with pre-filled slots, but the problem will probably be the same as with a slot filled with a `from_trigger_intent` mapping (I think it's the check to see if there's an active loop that causes this). Here's a simple example to test it: **Command or request that led to error**: ``` rasa shell --debug Your input -> /test_form Asking other slot Your input -> /affirm Form completed. Value is value_to_correct ``` **Content of configuration file (config.yml)** (if relevant): ```yml ... policies: - name: RulePolicy ``` **Content of domain file (domain.yml)** (if relevant): ```yml intents: - affirm - test_form actions: - validate_test_trigger_intent_form forms: - test_trigger_intent_form: filled_with_trigger_intent: - type: from_trigger_intent intent: test_form value: value_to_correct other_slot: # Just to verify if value is validated later - type: from_intent intent: affirm value: True slots: filled_with_trigger_intent: type: unfeaturized other_slot: type: unfeaturized templates: utter_ask_other_slot: - text: Asking other slot utter_form_completed: - text: Form completed. Value is {filled_with_trigger_intent} ``` **Content of rules file (rules.yml)** (if relevant): ```yml ... rules: - rule: test trigger intent validation steps: - intent: test_form - action: test_trigger_intent_form - active_loop: test_trigger_intent_form - active_loop: null - action: utter_form_completed ``` **Contents of actions.py** (if relevant): ```python class ValidateTestTriggerIntentForm(Action): def name(self) -> Text: return "validate_test_trigger_intent_form" def run( self, dispatcher: CollectingDispatcher, tracker: Tracker, domain: Dict ) -> List[EventType]: extracted_slots: Dict[Text, Any] = tracker.form_slots_to_validate() logger.info(f"Validating extracted slots: {extracted_slots}") logger.info(tracker.events) validation_events: List[EventType] = [] for slot_name, slot_value in extracted_slots.items(): if slot_name == "filled_with_trigger_intent" and slot_value == "value_to_correct": validation_events.append(SlotSet("filled_with_trigger_intent", "corrected_value")) return validation_events ``` Logs show that the correct events are there and core calls the validation action with the slot: **Action server logs for first interaction**: ``` ... action-server_1 | 2020-09-15 18:16:03 INFO covidflow.actions.test_trigger_intent_form - [info ] Validating extracted slots: {} action-server_1 | 2020-09-15 18:16:03 INFO covidflow.actions.test_trigger_intent_form - [info ] [{'event': 'action', 'timestamp': 1600193759.547206, 'name': 'action_listen', 'policy': None, 'confidence': 1.0}, {'event': 'user', 'timestamp': 1600193763.3620577, 'text': '/test_form', 'parse_data': {'intent': {'name': 'test_form', 'confidence': 1.0}, 'entities': [], 'text': '/test_form', 'message_id': 'b3562360e39546a685aea2ff0afa0b29', 'metadata': {}, 'intent_ranking': [{'name': 'test_form', 'confidence': 1.0}]}, 'input_channel': 'cmdline', 'message_id': 'b3562360e39546a685aea2ff0afa0b29', 'metadata': {}}, {'event': 'action', 'timestamp': 1600193763.3705964, 'name': 'test_trigger_intent_form', 'policy': None, 'confidence': None}, {'event': 'slot', 'timestamp': 1600193763.370542, 'name': 'filled_with_trigger_intent', 'value': 'value_to_correct'}] ... ``` **Core logs for first interaction**: ``` ... 2020-09-15 18:42:58 DEBUG rasa.core.actions.forms - Activated the form 'test_trigger_intent_form'. 2020-09-15 18:42:58 DEBUG rasa.core.actions.forms - No pre-filled required slots to validate. 2020-09-15 18:42:58 DEBUG rasa.core.actions.forms - Validating user input 'UserUttered(text: /test_form, intent: {'name': 'test_form', 'confidence': 1.0}, entities: [])'. 2020-09-15 18:42:58 DEBUG rasa.core.actions.forms - Extracted 'value_to_correct' for extra slot 'filled_with_trigger_intent'. 2020-09-15 18:42:58 DEBUG rasa.core.actions.forms - Validating extracted slots: {'filled_with_trigger_intent': 'value_to_correct'} 2020-09-15 18:42:58 DEBUG rasa.core.actions.action - Calling action endpoint to run action 'validate_test_trigger_intent_form'. ... ```

So, this is a bug. Which means validation for forms in rasa 2.x is currently broken.

akelad · October 5, 2020, 8:08am

Hey @petasis thanks for bringing that up, it’s being fixed in this PR

petasis · October 5, 2020, 8:37am

Is there a timeplan when a new rasa 2.x version will be released with the fix? Is there a workaround for now?

Right now, what is do is that when “extracted_slots” is None, I collect all slots that are not None and must be validated, and validate them. It happens only when slots have been filled from the 1st user utterance.

akelad · October 5, 2020, 8:50am

The final release of version 2.0 is planned for this week (of course this isn’t a guarantee, there’s always a chance something goes wrong and it’s delayed). The version you’re using right now is still a release candidate, and so not suited for production yet. You could use the version of the SDK on that branch for now, and it’ll also be merged to master soon.

petasis · October 9, 2020, 11:15am

@akelad Hi again. I just updated to rasa 2.0, and I am back at my original question: are forms working?

After so much effort to make my forms work in 2.0.0rc3, 2.0.0rc4, my forms do not work with rasa 2.0. And the documentation confusing. What is written in Forms does not work for me.

akelad · October 9, 2020, 11:29am

Sounds like you’ve asked your question in a different thread already - we’ll get back to you there.

Topic		Replies	Views
Rasa 2.x does not follow story Rasa Open Source	5	470	December 13, 2021
Weird behaviour stories Rasa Open Source	1	353	February 7, 2020
Rasa does not learn from stories Rasa Open Source	1	290	March 1, 2021
RASA 2.0 Stories do not continue after form is handled Rasa Open Source	4	997	October 29, 2020
Trouble with 'There is no memorized next action' message Rasa Open Source	10	5702	March 19, 2021

Rasa 2.0, how are forms working?

Related Topics