Working with User Images Sent to the Bot

Rasa Open Source
1

Hey everyone!

We are currently working with something like this:

  1. user intent is to have information about their status
  2. bot utters message asking for their ID image
  3. they take a picture/send it
  4. image gets sent to an API - custom action
  5. bot utters an answer / returns user information

I was wondering if there is possible to recognize an user sent message like that (image)? (or in other words how to make it possible to use that message/image to make that next custom action?) I’m aware that rasa works with NLU and not image recognition, but I’m stuck in finding an solution to make that example above work while still working with rasa. Could anyone help with ideas?

Thanks a lot in advance! :relaxed:

3 Likes
2

Hey, right now I think we just ignore that input. You’d have to customise the relevant connector for the messaging platform to actually receive that image. Something like this in the telegram connector for example: rasa_core/telegram.py at master · RasaHQ/rasa_core · GitHub

1 Like
3

hii @akelad, rasa_core/telegram.py at master · RasaHQ/rasa_core · GitHub this shows 404 error not found while opening

4

Here’s the updated link: rasa/telegram.py at master · RasaHQ/rasa · GitHub

5

I am working on similar Rasa chat bot which has to take a picture of the document the user presents, and the bot has to click a picture via the webcam. I am new to Rasa and do not know how to work around with custom actions, if there are any implementations I can use for reference, it would be of great help.

Thanks in advance! :smiley:

6

@carla.lmeida , did you find any solution?i have similar requiremnt with Telegram. Thanks

7

@kabeer I made a custom action where the bot triggers the camera and takes a picture, when you hit a key on the keyboard.

photo.py (1.8 KB)

8

Hi , i have similar requirement. I have created a beauty bot. Now i want to add one feature where can upload his/her image. is there any solution where image can be captured irrespective of any device ?

9

Hi, @kiranmah92

Are you tried on upload option to get user uploaded image. If you got that please help me on that.

Thanks in advance.

1 Like
10

I’m interested in this, did anyone find a solution?

11

I want help regarding getting image from user by chatbot and applying algorithm on that image. After processing send characteristic of that image to user through chat or send image to user

12

hi Siddhanth, can this method work when you are using whatsapp or telegram as your front end