I currently have a web chatbot UI that is connected to my RASA bot. For now I am able to recieve and send text output. But now, if the intent requires I want to be able to send an image file (or image file location) along with text output so that the image is rendered alongside the chatbot in the UI. The rendering part I can do. But I need to understand how to modify my action or the agent.handle_message() part to send text+image.
For example I currently have an action like-
class ActionAchievementReport(Action):
def name(self):
# type: () -> Text
return "action_achievement_report"
def run(self, dispatcher, tracker, domain):
# type: (Dispatcher, DialogueStateTracker, Domain) -> List[Event]
my_period = tracker.get_slot('period')
my_period_compare = tracker.get_slot('period_compare')
my_bucket = tracker.get_slot('bucket')
result =random.uniform(-0.5, 0.5) # eventually get from db
###
# Some code to generate image.png file
###
if result > 0.0:
dispatcher.utter_template("utter_achievement_up", tracker, achieve_amt=result,period=my_period)
if result < 0.0:
dispatcher.utter_template("utter_achievement_down", tracker, achieve_amt=result, period=my_period)
return [SlotSet("achieve_amt", result if result is not None else []),SlotSet("report_list",report_list)]
so basically my utter_template takes care of response and when i run the model for this action my output is of the form-
rasa_output = [{"text": "You revenue is up by 0.4123."}]
Now i want this to have -
rasa_output = [{"text": "You revenue is up by 0.4123."},{"image":"image.png}"]
This “image” part can be empty or filled depending upon whether my action generated an image.
Where and what changes can I make to achieve this output?
Any thoughts/ideas are greatly appreciated
I tried to use the solution from domain format but what if I want to keep the image path variable based on input from my action for example I have-
utter_growth_up:
-text:"{period}revenue is up by {growth_amt}"
image:"{image_growth_up}"
In my utter_dispatch (called in the action), I fill the values for period and growth_amount. I tried doing the same for image_growth_up but it did not work.
Any suggestions about being able to change the {image_growth_up} variable. In my action I have-
@akelad Do we have any solution to this ? {image_growth_up} that gets passed as a parameter in the utter_template shows as {image_growth_up} in the attachment tag in the json object rather than the URL. Any documentation how can we send a value to the parameter ?
Actually,my problem was not really to send it to utter_template. I have an chatbot where I want the relevant image to be along the bot window and only the text part gets written in bot window.
so my utter_template looks like-
the temperature for today is {temp} || {img_temp_chart}
{imp_temp_chart} holds the path to the temperature chart file
So just before I sent the output to the chatbot window on the browser, I split the above output by ‘||’ , take first part and pass on to the bot window,where as the {img_temp_chart} part is used to grab the required file to be displayed along the bot window
Hope this helps. Let me know if you have any more questions.
If we need to send a image dynamically instead of static one, we should give image url or make local image to run on server and send this local file path as dispatcher message. as shown below:
We are running Django Sever, and in its images folder, we are dynamically saving an image.
This path is given as utterance of the dispatcher
I would suggest you to use image url as in some cases it is better to give url to UI. For me when i needed to send image to react UI so i had to give url for image preview as well as for storing it in mongoDB
