# Question about Rasa and Mirador

**URL:** <https://forum.rasa.com/t/question-about-rasa-and-mirador/53703>\
**Category:** Rasa Open Source\
**Created:** [June 10, 2022, 11:23am UTC](https://forum.rasa.com/t/question-about-rasa-and-mirador/53703 "2022-06-10T11:23:26Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![luisshinken](https://avatars.discourse-cdn.com/v4/letter/l/df788c/32.png) [@luisshinken](https://forum.rasa.com/u/luisshinken)\
**Post date:** [June 10, 2022, 11:23am UTC](https://forum.rasa.com/t/question-about-rasa-and-mirador/53703/1 "2022-06-10T11:23:27Z")

</div>

Hi guys, I need to solve a problem for my thesis. I built a bot about some architectural structures of my city, and I implemented this bot in my local webpage where I instantiated Mirador with direct opening on an image. This is an example:

 ![esempio](https://europe1.discourse-cdn.com/flex013/uploads/rasa/original/3X/9/c/9c7eae26da40cbd88bc1f7d8e8851d8879102eba.jpeg)

Now I’d like to create some interactions , like for example if I ask “what is this?” without adding the thing itself like “what is cryptoporticus?” , the bot answers in the right way according to the photo that the user clicked on. Can you help me?

---

<div class="post-metadata">

**Author:** ![souvikg10](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/souvikg10/32/93_2.png) [@souvikg10](https://forum.rasa.com/u/souvikg10)\
**Post date:** [June 10, 2022, 11:32am UTC](https://forum.rasa.com/t/question-about-rasa-and-mirador/53703/2 "2022-06-10T11:32:37Z")

</div>

you can inject context to your bot using the Rasa HTTP API by appending SlotSet event to the tracker.

Assuming here you have some kind of web front end(react, vue etc), you can take the image context, and create a slot in Rasa domain, create stories with the context of this slot value and add value in real time to the slot using the SlotSet Event, this way, the bot already has the context of what the image is about, thus any question triggered on this image page, would be about the image already.

> **[Rasa Open Source Documentation](https://rasa.com/docs/rasa/pages/http-api/)**

---

<div class="post-metadata">

**Author:** ![luisshinken](https://avatars.discourse-cdn.com/v4/letter/l/df788c/32.png) [@luisshinken](https://forum.rasa.com/u/luisshinken)\
**Post date:** [June 10, 2022, 3:58pm UTC](https://forum.rasa.com/t/question-about-rasa-and-mirador/53703/3 "2022-06-10T15:58:03Z")

</div>

Thank you so much for the reply! Unluckily, I still don’t understand well. I share some codes I did:

1. this is a code I inserted in my doctype html:

2. the function in the code is :

3. at last I created the file mir\_action.php in this way just for testing:

```auto

[ {
	"action" : 
	{
		"type" : "CHANGE_CANVAS", 
		"parameters" : 
		{
			"window_id": "main_window", 
			"argument" : "I put here the link of the first img that the user views in mirador viewer"
			}
		}
			}
		  ]
		  

```

Now, if I click on a photo that isn’t the first one and if I ask to bot anything, like “hallo” or any quote of my intents, instantly mirador changes photo and goes on the first one. So something works. Now I’d like to understand how to do what I asked in the topic, for example, how can I put in a slot the value of entity linked to the photo that I clicked , like u suggested in the reply ? Can you give me an example? Thank you so much

---

<div class="post-metadata">

**Author:** ![luisshinken](https://avatars.discourse-cdn.com/v4/letter/l/df788c/32.png) [@luisshinken](https://forum.rasa.com/u/luisshinken)\
**Post date:** [June 13, 2022, 11:43am UTC](https://forum.rasa.com/t/question-about-rasa-and-mirador/53703/4 "2022-06-13T11:43:11Z")

</div>

Hi , I tried to create the interaction in an other way: every time I change page on website, I trigger a function “transmit” that sends to rasa bot an intent with the link of the photo. Every time this intent comes to the bot , the intent triggers a custom action that sets the slot I need with the link of the page. But there is a problem: if I do "“rasa run actions” and “rasa run -m models --enable-api --cors “\*” --debug -p 5005” the intent looks like it doesn’t come (or the custom action isn’t triggered, I don’t know). But if I close the cmd with “rasa run actions”, it works like in the photo:

 ![cambio](https://europe1.discourse-cdn.com/flex013/uploads/rasa/original/3X/d/2/d2dc015e96fd380eabbeabe77664e3c1d47f3f3b.png)Can you help me?

---

<div class="post-metadata">

**Author:** ![souvikg10](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/souvikg10/32/93_2.png) [@souvikg10](https://forum.rasa.com/u/souvikg10)\
**Post date:** [June 13, 2022, 11:56am UTC](https://forum.rasa.com/t/question-about-rasa-and-mirador/53703/5 "2022-06-13T11:56:29Z")

</div>

@luisshinken - i don’t understand the issue you mentioned.

You have two separate terminals running with each of the command, and when you turn off rasa run models it is working?

---

<div class="post-metadata">

**Author:** ![luisshinken](https://avatars.discourse-cdn.com/v4/letter/l/df788c/32.png) [@luisshinken](https://forum.rasa.com/u/luisshinken)\
**Post date:** [June 13, 2022, 12:27pm UTC](https://forum.rasa.com/t/question-about-rasa-and-mirador/53703/6 "2022-06-13T12:27:01Z")

</div>

nono when I turn off the server for the custom action it works. This is the intent sent by the agent to the bot every time a user change photo in the webpage:

````auto
- intent: EXTERNAL_page_change
  examples: |
    - page change [https://parsec2.unicampania.it/cleopatra/cleodemo/summerschool/index.json/canvas/0](page_url)```

this is the action :

````

utter\_EXTERNAL\_page\_change:

- text: “You have changed page : {page\_url}.”

```auto

this is the CUSTOM action:

```

from rasa\_sdk import Action, Tracker

from rasa\_sdk.executor import CollectingDispatcher

from rasa\_sdk.events import SlotSet

class setTopic(Action):

```
def name(self):

    return "action_set_Topic"

def run(self, dispatcher, tracker, domain):

    link_page = tracker.get_slot("page_url")    

    return[SlotSet("topic", link_page)]

```

```auto

At last, this is the rule:

```

- rule: Say that page has changed

```auto

If I turn on the server for rasa, and the server for the custom actions, when I change photo the utter_EXTERNAL_page_change isn't triggered (it should appears in this way : "You have changed page : none." because the server for rasa action is off, but nothing, this msg doesn't appear).

If I turn on the server for rasa and turn off the server for rasa custom actions, the interaction works like in the previous photo, the problem is that the custom action is important for me to set an important slot.

Is it clear now? Thank you anyway for you reply :clap:
```

---

<div class="post-metadata">

**Author:** ![souvikg10](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/souvikg10/32/93_2.png) [@souvikg10](https://forum.rasa.com/u/souvikg10)\
**Post date:** [June 13, 2022, 1:01pm UTC](https://forum.rasa.com/t/question-about-rasa-and-mirador/53703/7 "2022-06-13T13:01:09Z")

</div>

probably something you have done but still asking, did you retrain your bot after adding the rule?

---

<div class="post-metadata">

**Author:** ![luisshinken](https://avatars.discourse-cdn.com/v4/letter/l/df788c/32.png) [@luisshinken](https://forum.rasa.com/u/luisshinken)\
**Post date:** [June 13, 2022, 1:23pm UTC](https://forum.rasa.com/t/question-about-rasa-and-mirador/53703/8 "2022-06-13T13:23:47Z")

</div>

Dont’t worry, every suggestion is welcome. Anyway yes, I did it everytime I change something in nlu , domain or any file linked to the bot

---

<div class="post-metadata">

**Author:** ![souvikg10](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/souvikg10/32/93_2.png) [@souvikg10](https://forum.rasa.com/u/souvikg10)\
**Post date:** [June 13, 2022, 2:01pm UTC](https://forum.rasa.com/t/question-about-rasa-and-mirador/53703/9 "2022-06-13T14:01:20Z")

</div>

does the action server works for other actions? I am going to go through a list of diagnostics

---

<div class="post-metadata">

**Author:** ![luisshinken](https://avatars.discourse-cdn.com/v4/letter/l/df788c/32.png) [@luisshinken](https://forum.rasa.com/u/luisshinken)\
**Post date:** [June 13, 2022, 2:12pm UTC](https://forum.rasa.com/t/question-about-rasa-and-mirador/53703/10 "2022-06-13T14:12:49Z")

</div>

UPDATE: I think the problem is solved:

I added the 3 lines of my rules:

```auto
- rule: Say that page has changed
  steps:
    - intent: EXTERNAL_page_change
    - action: utter_EXTERNAL_page_change
    - action: action_set_Topic

```

I added them to the first part of my stories:

```auto
# what is stories

- story: what_is_arch
  steps:
  - intent: EXTERNAL_page_change
  - action: utter_EXTERNAL_page_change
  - action: action_set_Topic
  - intent: what_is  
    entities:
    - topic: arch  
  - slot_was_set:
    - topic: arch
  - action: utter_what_is
  

- story: what_is_triumphal_arch
  steps:
  - intent: EXTERNAL_page_change
  - action: utter_EXTERNAL_page_change
  - action: action_set_Topic
  - intent: what_is    
    entities:
    - topic: triumphal_arch  
  - slot_was_set:
    - topic: triumphal_arch
  - action: utter_what_is

- story: what_is_conocchia
  steps:
  - intent: EXTERNAL_page_change
  - action: utter_EXTERNAL_page_change
  - action: action_set_Topic
  - intent: what_is    
    entities:
    - topic: conocchia  
  - slot_was_set:
    - topic: conocchia
  - action: utter_what_is

- story: what_is_cryptoporticus
  steps:
  - intent: EXTERNAL_page_change
  - action: utter_EXTERNAL_page_change
  - action: action_set_Topic
  - intent: what_is    
    entities:
    - topic: cryptoporticus  
  - slot_was_set:
    - topic: cryptoporticus
  - action: utter_what_is

- story: what_is_theater
  steps:
  - intent: EXTERNAL_page_change
  - action: utter_EXTERNAL_page_change
  - action: action_set_Topic
  - intent: what_is 
    entities:
    - topic: theater  
  - slot_was_set:
    - topic: theater
  - action: utter_what_is

- story: what_is_domus
  steps:
  - intent: EXTERNAL_page_change
  - action: utter_EXTERNAL_page_change
  - action: action_set_Topic
  - intent: what_is 
    entities:
    - topic: domus  
  - slot_was_set:
    - topic: domus
  - action: utter_what_is

- story: what_is_domus_confuleius
  steps:
  - intent: EXTERNAL_page_change
  - action: utter_EXTERNAL_page_change
  - action: action_set_Topic
  - intent: what_is 
    entities:
    - topic: domus_confuleius  
  - slot_was_set:
    - topic: domus_confuleius
  - action: utter_what_is

- story: what_is_domus_orti
  steps:
  - intent: EXTERNAL_page_change
  - action: utter_EXTERNAL_page_change
  - action: action_set_Topic
  - intent: what_is 
    entities:
    - topic: domus_orti  
  - slot_was_set:
    - topic: domus_orti
  - action: utter_what_is   

- story: what_is not set
  steps:
    - entities:
      - topic: other
      intent: what_is
    - slot_was_set:
        - topic: __other__
    - action: utter_ask_topic

```

Now everytime I change page , the bot notify me of that and if I ask “What is this?” he answers me in the right way:

 ![finalmente](https://europe1.discourse-cdn.com/flex013/uploads/rasa/original/3X/b/0/b03f1d2a16161158042f36e5c2d88d2acda9c394.png)

Thank you so much for your support @souvikg10

Now I have a new question : I would like to create another interaction from BOT --TO–\> AGENT , that is : if I am on the photo of X , but I ask the bot “What is Y?” , I’d like to change photo of Y automatically in the website. Could you help me for this interaction?

---

<div class="post-metadata">

**Author:** ![luisshinken](https://avatars.discourse-cdn.com/v4/letter/l/df788c/32.png) [@luisshinken](https://forum.rasa.com/u/luisshinken)\
**Post date:** [June 13, 2022, 4:54pm UTC](https://forum.rasa.com/t/question-about-rasa-and-mirador/53703/11 "2022-06-13T16:54:53Z")

</div>

@souvikg10 Thinking about the problem, the solution I want to implement is to create a custom action in rasa that send to the webpage the link of the photo, and when the site receive the link he changes photo thanks to the link that I send with the custom action. Do you know how to do this?

---

<div class="post-metadata">

**Author:** ![souvikg10](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/souvikg10/32/93_2.png) [@souvikg10](https://forum.rasa.com/u/souvikg10)\
**Post date:** [June 14, 2022, 7:44am UTC](https://forum.rasa.com/t/question-about-rasa-and-mirador/53703/12 "2022-06-14T07:44:26Z")

</div>

@luisshinken - i suppose you answered your own question here. you can create a custom action to send your webpage the link of the photo to show.

Your manipulation of the chat is mostly related to the front end. Rasa here, plays little role, it feeds your front end widget with what to do next given an intent. so you can customize what should happen next and in this case forwarding an image url to the widget.

---

<div class="post-metadata">

**Author:** ![luisshinken](https://avatars.discourse-cdn.com/v4/letter/l/df788c/32.png) [@luisshinken](https://forum.rasa.com/u/luisshinken)\
**Post date:** [June 14, 2022, 10:43am UTC](https://forum.rasa.com/t/question-about-rasa-and-mirador/53703/13 "2022-06-14T10:43:24Z")

</div>

I solved the problem. If it can help anyone:

I created a custom action that sends a JSON to my webpage in this way :

```auto
class get_and_send_Topic(Action):

    def name(self):
        return "send_Topic"

    def run(self, dispatcher, tracker, domain):
        get_topic = tracker.get_slot("topic")
   j = [{"action" : {"type" : "CHANGE_CANVAS", "parameters" : {"window_id": "main_window", "argument" : "URL OF THE PHOTO"}}}]

    dispatcher.utter_message(attachment = J)
return []

```

and in my page I acquire di url with JSON.parse

it works! Thanks anyway!