Hi guys, I need to solve a problem for my thesis.
I built a bot about some architectural structures of my city, and I implemented this bot in my local webpage where I instantiated Mirador with direct opening on an image.
This is an example:
Now I’d like to create some interactions , like for example if I ask “what is this?” without adding the thing itself like “what is cryptoporticus?” , the bot answers in the right way according to the photo that the user clicked on.
Can you help me?
you can inject context to your bot using the Rasa HTTP API by appending SlotSet event to the tracker.
Assuming here you have some kind of web front end(react, vue etc), you can take the image context, and create a slot in Rasa domain, create stories with the context of this slot value and add value in real time to the slot using the SlotSet Event, this way, the bot already has the context of what the image is about, thus any question triggered on this image page, would be about the image already.
at last I created the file mir_action.php in this way just for testing:
[ {
"action" :
{
"type" : "CHANGE_CANVAS",
"parameters" :
{
"window_id": "main_window",
"argument" : "I put here the link of the first img that the user views in mirador viewer"
}
}
}
]
Now, if I click on a photo that isn’t the first one and if I ask to bot anything, like “hallo” or any quote of my intents, instantly mirador changes photo and goes on the first one. So something works. Now I’d like to understand how to do what I asked in the topic, for example, how can I put in a slot the value of entity linked to the photo that I clicked , like u suggested in the reply ? Can you give me an example? Thank you so much
Hi , I tried to create the interaction in an other way: every time I change page on website, I trigger a function “transmit” that sends to rasa bot an intent with the link of the photo. Every time this intent comes to the bot , the intent triggers a custom action that sets the slot I need with the link of the page.
But there is a problem: if I do "“rasa run actions” and “rasa run -m models --enable-api --cors “*” --debug -p 5005” the intent looks like it doesn’t come (or the custom action isn’t triggered, I don’t know). But if I close the cmd with “rasa run actions”, it works like in the photo:
nono when I turn off the server for the custom action it works.
This is the intent sent by the agent to the bot every time a user change photo in the webpage:
- intent: EXTERNAL_page_change
examples: |
- page change [https://parsec2.unicampania.it/cleopatra/cleodemo/summerschool/index.json/canvas/0](page_url)```
this is the action :
utter_EXTERNAL_page_change:
text: “You have changed page : {page_url}.”
this is the CUSTOM action:
from rasa_sdk import Action, Tracker
from rasa_sdk.executor import CollectingDispatcher
If I turn on the server for rasa, and the server for the custom actions, when I change photo the utter_EXTERNAL_page_change isn't triggered (it should appears in this way : "You have changed page : none." because the server for rasa action is off, but nothing, this msg doesn't appear).
If I turn on the server for rasa and turn off the server for rasa custom actions, the interaction works like in the previous photo, the problem is that the custom action is important for me to set an important slot.
Is it clear now? Thank you anyway for you reply :clap:
Now I have a new question :
I would like to create another interaction from BOT --TO–> AGENT , that is : if I am on the photo of X , but I ask the bot “What is Y?” , I’d like to change photo of Y automatically in the website. Could you help me for this interaction?
@souvikg10 Thinking about the problem, the solution I want to implement is to create a custom action in rasa that send to the webpage the link of the photo, and when the site receive the link he changes photo thanks to the link that I send with the custom action. Do you know how to do this?
@luisshinken - i suppose you answered your own question here. you can create a custom action to send your webpage the link of the photo to show.
Your manipulation of the chat is mostly related to the front end. Rasa here, plays little role, it feeds your front end widget with what to do next given an intent. so you can customize what should happen next and in this case forwarding an image url to the widget.