Supervised Embeddings Pipeline initialized with pre-trained word embeddings

Rasa Open Source
1

Hi All,

I have been experimenting with the Spacy pretrained pipeline and the Supervised Embeddings pipeline, along with the ResponseSelector component.

While I like the fact that Spacy -pretrained pipeline allows your to utilize pre-trained word embeddings, i find that the Supervised-Embeddings pipeline is more versatile (ResponseSelector uses this, supports multi intents, uses a more sophisticated classifier etc).

However, the limitation of the SupervisedEmbeddings pipeline as I see is that it does not let you utilize all the great work done by others (eg, use pre-trained word embeddings etc). Is there a workflow which would allow me to get the benefits of both worlds? I.e. can we initialize supervised embeddings with fast-text or other word vectors? I’m sure this could be done if I write my own classifier, but I’m trying to understand if there has any work been done in this direction? Also, is the ConveRTFeaturizer a tool which tried to achieve something similar?

I believe @dakshvar22 might be able to give me some insights on this :slight_smile:

Thanks! Thusitha