Hi guys,
I have implemented my rasa chatbot using botfront as front end. I have a custom action to perform some kind of computer vision tasks on webcam frames, now I’d like to show the webcam frames in the website. To do that i would like to use Flask framework but i don’t know how to implement flask in a custom action. Is it the correct way to do or should I use another python script?
@Giustino98 Hello, and welcome to the forum!
Can you confirm, you want to use webcam for website or webcam for chatbot the top layer on website? Thanks.
Hi Nik, thank you so much for the answer.
I want to use the webcam in rasa custom action to make some operations on frames acquired, but i want that the webcam is shown in a website (actually it is shown in an opencv box). I already see that it’s possible to do through Flask but i don’t know how to integrate it into the custom action.
To make it clear i show you my custom class code:
class ActionStartWebcam(Action)
frame = cv2.imread('placeholder-image.jpg')
# encode the frame in JPEG format
(flag, imagetoshow) = cv2.imencode(".jpg", frame)
encodedImage = imagetoshow.tobytes()
def name(self) -> Text:
return "action_start_webcam"
async def run(
self, dispatcher, tracker: Tracker, domain: Dict[Text, Any]
) -> List[Dict[Text, Any]]:
# custom behavior
..... some code....
while loop:
img = ....some code...
cv2.imshow('Image', img)
return [SlotSet("items", objects), SlotSet("object_detected", object_detected), SlotSet("room", room)]
As you can see i have “cv2.imshow(…)” that shows my webcam in opencv box, instead i want to integrate it into the website.
@Giustino98 Right, means you want that on your website. But, if you aware custom action is used for rasa specificly, and rasa not deal with any frontend, only to render the response such as image, button, text etc using third party and in your case you are using bot front as your frontend for chat widget. If you have developed your website using flask, then sky is the limit. But, its not related to rasa in this context. it much related to website and customisations of the code. I hope I am clear to you? Do you have any further doubt please share or discuss with me.
please share the link, if you have?
@nik202 You have been so much clear, thank you.
I have just one more doubt: in my custom class “ActionStartWebcam” I have an attribute “encodedImage” that is updated in the “run” method of the class in a while loop. Is it possible to access this attribute from another python script? I mean I have “main.py” in which I handle Flask connection, I’d like to access ActionStartWebcam attribute from main.py script, is it possible or the fact that custom action runs on another server makes classes unaccessable to other local script?
I want to apologize if I am asking silly questions but this is the first time i work with Rasa and this is for a university thesis project.
@Giustino98 yes, you can import the class and file in custom action, just you need to import, make sure the files are with custom action on a safer side.
No worries, as as much as questions and even ask some question with google 
All the best for your thesis project, and if you need further clarification please do come with new questions 
@Giustino98 If I answer all your questions, can I request to please close this topic for your reference and for others, thanks.
Of course, thank you again. 