Supervised Response Selector [Experimental]

I meant this Fallback Actions Also it is implemented here rasa-demo/actions.py at master · RasaHQ/rasa-demo · GitHub Will i have opportunity using response selector to choose answer if low nlu confidence?

@Nick_Kazakov Not yet. But that’s a good point. We’ll give this a thought. Thanks

Hello, and what if I need to run story depending on what specific intent was identified. It is possible to transform these two scenarios with retrieval actions?

* greet
  - utter_greet
  - utter_introduce_1
  - utter_possibilities
* goodbye
  - utter_goodbye
  - utter_always_here

It would be great to be able to use both aggregate intent and specific intent.

Classic stories:

* chitchat/greet
  - utter_greet
  - utter_introduce_1
  - utter_possibilities
* chitchat/goodbye
  - utter_goodbye
  - utter_always_here

Form stories:

* contact_sales
    - utter_moreinformation
    - sales_form
    - form{"name": "sales_form"}
* chitchat
    - respond_chitchat
* contact_sales
    - utter_moreinformation
    - sales_form
    - form{"name": "sales_form"}
* chitchat/goodbye
    - utter_goodbye
    - action_deactivate_form
    - form{"name": null}

Is there a way to execute “respond_chitchat” for example from a custom action? I’ve tried dispatcher.utter_template(“respond_chitchat”) which doesn’t work, any ideas?

@tocosastalo It isn’t possible as of now, but will give more thought to it how it should be incorporated. Thanks

This is a great step towards handling single-turn interactions separately. However, I have a few questions based on how I was using the traditional approach for single turn interactions along with rasa core and how using response retrieval models would change it.

Let me explain with an example, I have 4 intents and 1 entity of Car Model

intents:
  - cost
  - specification
  - downPayment
  - expectedDelivery

entity:
  - carModel

slots:
 carModel:
    type: categorical
    values:
        - modelX
        - cybertruck

actions:
  - cost
  - specification
  - downPayment
  - expectedDelivery
  - cost_modelX
  - specification_modelX
  - downPayment_modelX
  - expected_delivery_modelX
  - cost_cybertruck
  - specification_cybertruck
  - downPayment_cybertruck
  - expectedDelivery_cybertruck

obvious way of making this work is when both intent and entity is specified in the the utterance, for example, ‘what are the specs of cybertruck?’ or ‘when can i expect the delivery of Model X?’

But for a much better experience you cant force the user to mention a car model everytime such as this conversation

> What are the specs of Cybertruck?
> Whats the cost of Cybertruck?
> What about the down payment of Cybertruck?
> when can i expect delivery of Cybertruck?

instead you’d like the flow of conversation from user to be

> What are the specs of Cybertruck?
> How about the cost?
> What about the down payment?
> when can i expect its delivery?

So i want to use the slot set in the first utterance to influence the next predictions (to predict cost_cybertruck without user mentioning it ), and when that same slot is not set (implying that user never mentioned an entity in previous turns) you provide generic response (thus, only predicts cost action).

I was writing stories as following:

Generic cost response story

* cost
   - cost

Generic specification response story

* specification
   - specification

Generic downPayment response story

 * downPayment
    - downPayment

Generic expectedDelivery response story

 * expectedDelivery
    - expectedDelivery

Intent + Entity story (similar stories exist for all intent+entity combinations)

 * downPayment{"carModel": "modelX"}
    - slot{"carModel": "modelX"}
    - downPayment_modelX

Story to remember slot and influence next turn predictions

 * specification{"carModel": "cybertruck"}
    - slot{"carModel": "cybertruck"}
    - specification_cybertruck
 * cost
    - cost_cybertruck
 * downPayment
    - downPayment_cybertruck
 * expectedDelivery
    - expectedDelivery_cybertruck 

So if I were to use response retrieval models, how would that affect the stories? Since all of the responses would be mapped to something like faq. So how can I mimic the above behaviour with response retrieval models?

1 Like

I think a fallback policy for response selector would be a really nice feature.

1 Like

Hi @dakshvar22! Do you continue on Rasa X and response retrieval integration? Thanks!

Hi @madstuntman, it is definitely on the roadmap but no release date has been set yet.

3 Likes

Has anyone developed FAQ bot with this option that is actually being used on their website or somewhere else?

If someone did, can he say does it make sense to work with this if I’m planning to have 100 FAQ questtions about a restaurant implemented with it. :smiley:

@ kavan

Did you manage to do what you asked?

I’ve followed the article and implemented everything and get this error:

Any help?

Hey @rasafan, you have not written the responses properly in responses.md file.

It should be in this way:

## greet
* ask_faq/greet:
  -  Hey! Can I help you with something?

Reference:

https://rasa.com/docs/rasa/core/retrieval-actions/#training-data

https://github.com/JiteshGaikwad/Restaurant-Search-Bot/blob/master/data/responses.md

let me know if this helps you. :slight_smile:

1 Like

I definitely need the feature in Rasa X too. +1

Currently, if I upload the nlu.md file using Rasa X HTTP API, all my chitchat intents get merged and then respond_chitchat is unable to retrieve responses.

I patched rasa/data.py to load a static responses.md file, but it does not help.

# grep opt /usr/local/lib/python3.6/site-packages/rasa/data.py
    nlu_data_files = set(["/opt/responses.md"])
    nlu_data_files = set(["/opt/responses.md"])

Static /opt/responses.md content:

# head /opt/responses.md 
## ask_1
* chitchat/ask_1
  - La temporalité ne s'applique pas réellement à moi

## ask_2
* chitchat/ask_2
  - Je suis plus efficace dès lors qu'il s'agit de répondre à des questions

The supervised response selector and retrieval actions is an excellent feature in Rasa. At my place we prefer different response selector intents instead of just one and we definitely want to see this feature in Rasa X

Can response actions have custom information (besides “text”), just like normal messages?

@dakshvar22

I have the same request, see:

Hey! Can I use a custom action as a response selector response?

Best

What type of model does response selector use. Is it a simple feed forwared network with the sparse and dense features as input ?

@dingusagar I am not an expert, but from what I understand, the response selector is based on DIET, which uses a transformer. In rasa 1.x it goes directly from input text to response text, in rasa 2.x you can also use the response intents also (there is an option), so multiple responses are supported.

From the tests I have done, response selector seems to dislike dense representations. In all the tests I have done, the best features are the ones from CountVectorsFeaturizer (using words, not characters).

1 Like

Lets say I have a response selector model for chitchat intent with different variations of chitchat and their responses configured. I noticed that the response selector model in the pipeline gets executed even if the predicted intent from the DIET classifier is not chitchat.

Would like to know the design thought process behind this behaviour. Would’t it be better if the response model in the pipeline gets executed only if the predicted intent is the one to which response selector was configured for. @dakshvar22

In the above example, if predicted intent is not chitchat, what is the need for unnecessarily executing the response selector in the pipeline. Is it not an unnecessary overhead ?