Hi, I am trying to build a clothing recommender system using CLIP transformer model(by Open AI) as CLIP works with multimodal data(text and image) and tells us how close the ‘embedding of a text’ is to the ‘embedding of an image’. Therefore I want to take the responses of my users from my rasa bot and get the closest image recommendation from CLIP model. My question: How I do integrate CLIP model with my rasa bot. Please help me, thank you!

Thank you so much Greg, I am looking into it, if I run into any issues ill let you know sir.

How can I use CLIP model(by Open AI) in my rasa project

stephens (Greg Stephens) January 12, 2023, 4:22pm 2

I’m not familiar with CLIP but you can create a custom component. I posted about it recently here.

Topic		Replies	Views
Integration of GPT model with Rasa 3.1 Rasa Open Source	0	386	July 24, 2023
Enhance rasa with a pre-trained classification model Rasa Open Source	4	2131	October 22, 2019
Rasa Hybrid Chatbot Rasa Open Source	1	503	April 20, 2022
How to add the Ollama (llama Model ) to Rasa Chatbot? Rasa Open Source	0	143	October 4, 2024
Upload an image through Rasa Rasa Open Source	4	733	May 17, 2023