How to architect and use HFTransformersNLP

Hi everyone,

I’m looking for some architectural/guidance. I’m trying to now, pay around with some of the text-generation NLP models, like GPT-2. What I’d like to do, is allow a user to say a phrase, and generate some short text based on the the phrase, basically, recreate gpt2 · Hugging Face for myself with Rasa.

I think I have to do this with custom actions, because my understanding is that the pipelines in Rasa, only help improve understanding of what the user says. But should I be creating a formAction to use/access a model? For example, instead of from transformers import pipeline in my action.py, is there a way to call the model in my config file and use it in my action.py?

- name: HFTransformersNLP
    model_name: "gpt2"
    model_weights: "gpt2"

I’m not sure if I’m thinking about this correctly, so asking here. Apologies if this is confusing.

The config.yml file is meant to configure the NLP pipeline. Since you’re not using gpt2 here as a featurizer for intents/entities I wouldn’t add it in the config file. Instead what you mention about calling it from a custom action sounds like a better idea to get all of this working.

That said … there is a fair warning to be told here. Generative algorithms have some serious ethical concerns attached to them. We’ve outlined a few here; GPT-3: Careful First Impressions

Thanks for confirming. I did the old, trial and error, and it does work in a custom action.

Thanks for the article. Yes, I did read the GPT-3 paper, and the authors mention some similiar concerns.

The broader impacts of language models like this are numerous. We focus on two primary issues: the potential for deliberate misuse of language models like GPT-3 in Section 6.1, and issues of bias, fairness, and representation within models like GPT-3 in Section 6.2

Since I don’t have access to GPT-3, I just wanted to see what GPT-2 could do, and I do see it’s failures (it goes into the repeat loop sometimes, it generates text that is grammatically correct, but doesn’t make “real” sense". We also locked in the concept of using it to seed our chatbot, since getting test users…is a bit challenging at times.

Thanks though