Relaunch NLU pipeline from custom action (Rasa 1.X)

Hello all,

Is there any way to relaunch the rasa NLU pipeline from a custom action ?

What we want to do is :

When a user says something to our bot that is not clearly understood, we want to call a spellchecker from a custom action, replace the user input with the corrected input, and relaunch the NLU pipeline in order for Rasa to understand it better.

Many thanks,

1 Like

Hi @gdebaisieux, welcome to the forum!

Could you explain what you mean by “not clearly understood”? Is this more an issue of intent classification, or entity extraction?

If it’s the former, I’d advise against your suggested approach, and instead suggest you try to have your training data look more like the data you expect to see in the wild (i.e. including spelling mistakes). The best way to do this is to share your bot and collect real conversation data. This will make your model more robust. You can do something like this to confirm with the user what the intent should have been as a fallback.

If you’re concerned about entity extraction, I’d suggest using the spell checker in the custom action, and asking the user for confirmation about the corrected entities, similar to the action linked above which asks the user to confirm the intent. In general, I would be wary of relying on spellcheckers to correct entities without confirmation from your users :slight_smile:

How does that sound?

Hi, thank you for your reply,

Indeed, this is a classification issue, due to incorrect spelling by the user.

So if I understand it well, trying to relaunch the entire NLU pipeline in the case if the classification fails is a bad idea, and you suggest us to add a few mispelling examples so that our model could classify it.

Am I right ?

Another solution could be to use the spellchecker in a custom component in our NLU pipeline.

In any way, could you confirm us there is no way to change the user input and and “make a new turn” of the NLU pipeline, from a custom action ?

Many thanks for your replies,

Yes, you’re right, I would recommend you add these examples.

Another solution could be to use the spellchecker in a custom component in our NLU pipeline.

Exactly. This would require lots of testing, since spellcheckers can distort the user messages in an unintended way. For example, a user may be trying to talk about Google, and a spell-checker could turn this into goggle. The benefits of correctly classifying an intent that contained misspellings may not be worth introducing confusions like these.

In any way, could you confirm us there is no way to change the user input and and “make a new turn” of the NLU pipeline, from a custom action ?

This can’t be done out-of-the-box. The closest (and better) thing would be to revert the incorrect user utterance, and ask the user to confirm which intent they meant through the use of buttons, like here

@gdebaisieux, one more thing I didn’t mention is that you can use the add message and predict endpoints to create your own action to re-parse the corrected text. You can also parse a text to see what the nlu model would predict its intent to be. Using these endpoints you should be able to do :

change the user input and and “make a new turn” of the NLU pipeline, from a custom action

Hello @fkoerner, sorry for my very late answer, I was stuck on other business…

Many thanks for your answer ! Using these endpoints (parse the message, add it to the message queue and predict the next action), we were able to make this work for a dummy conversation.

However, when we try to add the corrected text to the message queue with the conversations/{id}/messages endpoint, we are facing another issue :

Failed to acquire lock for conversation ID '<sender_id>'. Retrying...

I believe we have this error because we are trying to add a message to the queue, while executing the default_fallback_action which has activated this lock.

Is there a way to bypass this conversation lock issue, so that we can add the message to the user current session ?

Many thanks,

Never mind, actually I was able to find a pretty simple solution for my problem.

To be able to relaunch the classification process for a new user sentence, from a custom action, I simply had to :

  • Retrieve the last user input with tracker.latest_message['text']

  • Correct it with our autocorrect system

  • Call the /model/parse API input to NLU process this new input and retrieve the intents ranking :

           response = requests.post("http://localhost:5005/model/parse", data= json.dumps({
              "text": corrected_text
          }))
          parsed_data = json.loads(response.text)
    
  • Return theses events at the end of the custom action run method :

    return [UserUtteranceReverted(), ActionExecuted("action_listen"), UserUttered(corrected_text, parsed_data)]
    

This way, the NLU process has been called manually, and the intent classification process is being called automatically at the end of the custom action !

Many thanks for your help @fkoerner :wink:

1 Like

Oh I’m glad to see you figured it out! Thank you for updating.

Happy bot-building :slight_smile: