Working with User Images Sent to the Bot

#1

Hey everyone!

We are currently working with something like this:

  1. user intent is to have information about their status
  2. bot utters message asking for their ID image
  3. they take a picture/send it
  4. image gets sent to an API - custom action
  5. bot utters an answer / returns user information

I was wondering if there is possible to recognize an user sent message like that (image)? (or in other words how to make it possible to use that message/image to make that next custom action?) I’m aware that rasa works with NLU and not image recognition, but I’m stuck in finding an solution to make that example above work while still working with rasa. Could anyone help with ideas?

Thanks a lot in advance! :relaxed:

#2

Hey, right now I think we just ignore that input. You’d have to customise the relevant connector for the messaging platform to actually receive that image. Something like this in the telegram connector for example: rasa_core/telegram.py at master · RasaHQ/rasa_core · GitHub

#3

hii @akelad, rasa_core/telegram.py at master · RasaHQ/rasa_core · GitHub this shows 404 error not found while opening

#4

Here’s the updated link: rasa/telegram.py at master · RasaHQ/rasa · GitHub