Actually, i was trying to use gpt2 and other bert models tuned on the Spanish language on my rasa 2.2.5 and I guess doing install rasa[full] it by default downloads transformer version 2.11, and on transformer version 2.11 the model i want to use is giving error that the model was not found on huggingface/models.
And when i manually upgrade the transformer to 4 then i had following outcomes:
- for bert model i was trying i.e. dccuchile/bert-base-spanish-wwm-cased which was giving model not found on huggingface/models was able to download weights and train nly on it.
- But for gpt2 models i.e. DeepESP/gpt2-spanish and datificate/gpt2-small-spanish was able to download weights but training nlu is not happening neither any error mesaage is showing.
- So, i tried training pretrained model listed on huggingface/pretrained_models.html i.e. gpt2 model id of gpt2 architeture to check if error is on those finetuned gpt2 models. And i found on transformer 2.11 it was able to download weights and training was done. But for transformer 4 it downloaded the weights but training didn’t start at all.
So, which is upgrading transformer only affecting gpt2 but not bert models. I haven’t cheked other models. Or is there any fixed requirements that transformers cannot be updated above 2.12? Please help