We have been working on a new exciting feature which includes two new components - retrieval actions and supervised response selector. Checkout the introduction to the feature, give it a go and share your feedback with us on the overall feature idea, possible use cases you envision, ease of integration and any other suggestions you may have.
hi @dakshvar22, Thanks for the the feature, it was easy to handle the small talk and interruptions without writing stories for each small talk within the stories. Below are few things which I would like to share
I am using the latest Rasa(1.3.7) and Rasa - X(0.21.3) version
this is how my
nlu.md fine looks like:
and it responds properly for the small talk:
but the problem occurs when I test it in Rasa X, I dont see any response from the bot
I don’t know whether Rasa X supports but
Retrieval Actions as of now but when I close Rasa X and checked my
nlu.md file , this is what I found:
All the small talk intents were deleted and merged under single intent
chitchat probably by Rasa X and when I train the bot, this is what I got:
The problem is I have lost all the small talk Intents and again I had to rewrite those
@JiteshGaikwad Thanks for writing about the problem in detail. Yes, Rasa X / Interactive learning do not support
Retrieval Actions as of now, since the feature is experimental and we want to gather substantial feedback before fully integrating it with the complete stack.
@dakshvar22, it would have been better if it was mentioned in the Rasa docs that Rasa X doesn’t support the
Retrieval Actions as of now else I won’t have lost the data
@JiteshGaikwad Completely agree with your suggestion. Will be added in the docs in the next patch release. Thanks
@JiteshGaikwad Regarding the first part of the problem, where you mentioned that the reply from the bot is not displayed: I tested with same versions of Rasa and Rasa-X and it works on my side. Here is what I did:
- Created a rasa project using
- Added some retrieval actions and retrieval intents.
- Trained the bot using
- Started Rasa X with
- Tested an example conversation and retrieval action gets triggered with correct bot message getting displayed.
Did you do anything different?
Ya I had tried the same steps but I didn’t found the responses but one thing I noticed when I switch the tabs within the Rasa-X I get the response, pls see the image below:
but this wasn’t interactive learning mode, I was talking w.r.t
Interactive Learning mode
@dakshvar22, If you see the screenshot here, its in interactive learning
@JiteshGaikwad Not sure if I understand you correctly. The first screenshots that you uploaded were also from
Talk to your bot(Interactive Learning) screen only or was it that screen after you did some steps in between(like annotating new data, retraining model from Rasa X, etc.)
Attaching some ss for your reference:
- when I test it without interactive learning mode:
- after I click on some other tabs and again come back to the chat tab, I get the response:
If you can see for the first time I didn’t get the response unless an until I switchthe tabs
hey @dakshvar22, Sorry I might have confused you, In the first screen I had clicked on
switch to strict conversation mode
I found out that Only when I click on
switch to strict conversation mode my
Retrieval Actions intents are getting merged under one single intent as
Anyways thanks for reaching out, I am not using Rasa X for
Retrieval Actions as of now till the time this
Retrieval Actions feature is completed for the stack
@JiteshGaikwad Thanks for finding the exact step. Noted your feedback.
Hi @dakshvar22. Any updates about the longer term functionality of the feature? i.e. Will it be taken away or radically changed in the near future? I have bot I’d like to try this feature to see if there is a performance upgrade over the current but hesitant to do so given the experimental nature of the feature and the amount of code changes I have to do . Does it look like it will go away in the next version ;)?
Also, a couple quick question about the modeling part.
- Can the embedding layer of the ResponseSelector be shared with the EmbeddingIntentClassifier?
- what exactly is the target variable in the training process? i.e. I want to know if the response text has any relevance or the response text only serves as a target label ?
@thusithaC Thanks for your question. We are currently collecting feedback on the training data format for retrieval actions because we are not completely sure of it at this moment. So, once we have that, we’ll make a decision on the next steps for the feature. We are definitely very excited about the feature and have already collected some constructive feedback on it. It will be tough to comment on the extent of the change that it may undergo. Nevertheless I would recommend you to try it out on maybe a smaller dataset of yours and comment on what you like/dislike.
For your specific questions -
- No, not as of now. You can use a shared glove featurizer, through
SpacyFeaturizerbut the learnt embedding layer will be different.
- The target variable is the similarity between user utterance and a candidate bot utterance. The response text and the user utterance text are featurized by an embedding layer and the similarity between positive pairs is maximized and negative pairs is minimized. Feel free to take a look at the diagram in the related blogpost - Integrate response retrieval models in assistants built with Rasa
Thanks for the answer @dakshvar22. Understanding the logic was indeed helpful. I have one more question that you might be able to assist. There always data-cleaning we would like to perform before passing on text to a ML component. i.e. the response text we like the user to see might be have to be cleaned a bit to make it more suitable for training the ResponseSelector component. In the normal pipeline, we can do any cleaning we like by adding a custom component at the top of the pipeline. However is this a possibility for Supervised Response Selector response Text?
@thusithaC Yes, the response text is stored as
response attribute within the
Message object inside NLU pipeline. So,
process method of your custom component can basically act on them and perform any cleaning that’s needed.