Peculiar validation failure

Rasa Open Source
1

High level description.

A validation function in a form action seemingly assigns the slot value as expected. However, when the slot is checked later on in another validation function, the value is not updated. Kind of similar to this one but no actual solution is mentioned there. The issue seems to be that the tracker is not updated ‘on the run’. I wonder what the recommended solution would be.

Detailed description:

Let say I have a FormAction with 3 required slots X, Y, Z (in that order).

I have validation functions for all 3 of them.

All 3 validation functions behave as expected if I walk the chatbot through the form action one by one, in other words - if the conversation looks something like this:

Start my form action
Question for X
Answer for X
Question for Y
Answer for Y
Question Z
Answer for Z

However, if I do something like this:

Start my form actions with values X, Y, Z

The bot fails in a strange way.

From debugging I can tell you the following:

X, Y, Z are extracted and assigned to their appropriate slots.

The validation functions are called in the correct order (X. Y. Z) for all 3 of them.

If I issue a print statement in the function for X(the 1st one called) it looks like the validation function is executed and the value for slot X is reassigned appropriately to the tracker’s current slot values. I have even done this manually by getting and setting the value in the validation function.

However, when validation function for Y is called(after the validation for X was called), the slot for X is still the value that was inputted by the user i.e. not the one assigned by the validation function. Since I depend on a chain logic, as in, validation of Y depends on the validation of X, this causes an application bug.

The issue persists regardless of whether there is a tracker db.

Suggestion Solutions

A not so good solution I can think of is to hold a copycat (of the tracker) object in the form action and access the values from there on the run.

A much nicer solution would be if I could manually call an update_slots(slot_value=value) method on the tracker and force update it during validation. Unless I am missing something, I can’t see it here.

Unless there is some good reason for it, it would be even nicer if the tracker got automatically updated on the fly without the user having to call anything on it.

2

There should always be a tracker db. Rasa Core requires this. Under what circumstances are you deleting the tracker db?

Can you provide the code for one of your validate functions?

3

Hi Greg,

By tracker db I mean a tracker store or an external database (in my case mongodb) that logs the conversation. I do not believe this is necessary for Rasa Core as the application functions without it and does not come with it out of the box. I can add/remove support for it from endpoints.yml and my point is that this doesn’t affect the behavior of the app.

I am not sure what you mean by ‘deleting the tracker db’. If you mean deleting the tracker object which keeps track of the slots and other data, then I am not not deleting it in any circumstances.

This would be a mock, slightly adjusted sample of a validation function I have written:

    def validate_entity2(self, value, dispatcher, tracker, domain):
        entity1 = tracker.current_slot_values()['entity1'] # this one is already inputted at this point
        value = self.assign_value_from_db(value, entity_x) # reassign value of entity2  based on the value of entity1
        if value in self.combinations_db['entity2'] # check if the assigned value is valid
            return {'entity2': value}
        else:
            dispatcher.utter_message(text='Sorry for Entity2 we only support values X, Y and Z at this point')
            return {'entity2': None}

Please also bear in mind my point that whether my validation functions work is tied to how I lead the conversation with the bot.

4

Rasa Core always creates a tracker store. This is integral to how it keeps track of user conversations, intent history, slot settings, etc. If you delete the configuration from the endpoints.yml, it still creates an in memory tracker store.

You stated it would be even nicer if the tracker got automatically updated on the fly without the user having to call anything on it. Do you know about the auto_fill?

You’ll find a working example of this in our financial bot here.

If you ask this bot, I want to make a payment, it will put you in this form. You can then say something like I want to pay the minimum balance on my emblem credit card which provides two entities in the utterance.

Greg

5

Hi Greg,

I think auto_fill is for a purpose different than my own (isn’t auto_fill set to True by default anyway?). I did give it a shot nevertheless - to no avail, the same issue persists.

The issue here is that the tracker may pass through one validation function, a certain value get updated in it and upon entering the next function, the tracker will still have the old value for that certain value. This is closely tested and verified and it is a strange issue.