Rasa and Facebook Messenger integration: What if the user send an image?

I’m working on a school project to create a chatbot that helps non-English speaking student learn English and improve their score in the English test. I have already implemented a Grammarly-alike action where user can ask to find grammar and spelling mistakes and input a paragraph then the bot will respond the mistakes. My professor recommend I should take this to the new level by implementing an image scanning method so students don’t have to type their writing text but rather take a photo of the text they’ve written and the bot should be able to convert the image to text using OCR and then feed to the analysis module. I think of integrating my bot to Facebook Messenger because the platform supports image sending features and then store the image in a slot and the action should handle it using OCR. Can anyone give me some instructions on how to implement this?

1 Like

Does anyone have a take on this

Hi @thanhmannguyen3158,

Try implementing this via a custom action, which takes the user image as input then calls your OCR code to produce your desired bot response. Check out the custom action documentation and video tutorial for details.

I hope this helps.


1 Like