How can I achieve Image Recognition in Rasa?

I want to be able to implement a situation whereby the user inputs an image of a meal and the chatbot captures and displays the nutritional aspects of the food

Hi @hellen17

Then you have to write a custom action that sends the image to a computer vison algorithm.

Nice use case. If you are going to work on this, maybe you can share your progress on this project. Sounds very interesting.

By default rasa has no solution for this. You would have to write it yourself.

Thanks @Johan1us do you mind sharing your email with me so that I can contact you


@hellen17 you can check Google Vision API for starter and integrate them in rasa custom action. How to take user input image is a different aspect of frontend. Also i m not sure Google Vision API will be able to give nutritional components or not. You can check it yourself here Vision AI | Derive Image Insights via ML  |  Cloud Vision API

1 Like