Hi guys,
I’m total newbie to Rasa dan TensorFlow. Can I check how do we set our Rasa X configuration page to use pre-trained BERT model as in my screenshot? I would like to set cache_dir to grab model from my local instead of downloading a few GB after I click “Train”
Any example what file is permitted to set at cache_dir? Example cache_dir : “C:\MyModel\Rasa\abc.xxx”
language: ms
pipeline:
name: LanguageModelFeaturizer
model_name: bert
model_weights: rasa/LaBSE
cache_dir: null
language: ms
pipeline:
- name: LanguageModelTokenizer
- name: LanguageModelFeaturizer
model_name: "bert"
model_weights: "setu4993/LaBSE"
cache_dir: "/rasa/model"
After I run “rasa train”, it says OSError: Model name “setu4993/LaBSE” was not found. Any idea how to integrate this huggingface model to Rasa? setu4993/LaBSE · Hugging Face
Hey @Hamiksu, welcome to the forum 
It seems to me that your issue is related to the model name, not to cache_dir. For the start, can try to load the rasa/LaBSE weights with cache_dir: null? I also recommend that you change your config to the recommended format, i.e. specify the model and weights under HFTransformersNLP like in the docs and later in the pipeline include LanguageModelTokenizer and HFTransformersNLP.
If the previously suggested thing work, then I’d recommend changing the model weights to setu4993/LaBSE (still without specifying cache_dir) to see what happens.
Let me know how it goes!
Hi @SamS ,
Thanks for the reply, I did:
pipeline:
- name: HFTransformersNLP
model_name: "bert"
model_weights: "bert-base-uncased"
cache_dir: null
and it can train and proceed, however when I changed to:
pipeline:
- name: LanguageModelTokenizer
- name: LanguageModelFeaturizer
model_name: "bert"
model_weights: "setu4993/LaBSE"
cache_dir: null
Rasa train nlu returns the following error:
OSError: Model name ‘setu4993/LaBSE’ was not found in tokenizers model name list (bert-base-uncased, bert-large-uncased, bert-base-cased, bert-large-cased, bert-base-multilingual-uncased, bert-base-multilingual-cased, bert-base-chinese, bert-base-german-cased, bert-large-uncased-whole-word-masking, bert-large-cased-whole-word-masking, bert-large-uncased-whole-word-masking-finetuned-squad, bert-large-cased-whole-word-masking-finetuned-squad, bert-base-cased-finetuned-mrpc, bert-base-german-dbmdz-cased, bert-base-german-dbmdz-uncased, TurkuNLP/bert-base-finnish-cased-v1, TurkuNLP/bert-base-finnish-uncased-v1, wietsedv/bert-base-dutch-cased). We assumed ‘setu4993/LaBSE’ was a path, a model identifier, or url to a directory containing vocabulary files named [‘vocab.txt’] but couldn’t find such vocabulary files at this path or url.
Ah, right. This is something on HuggingFace’s side though. The error comes from within their code. Not sure which version of transformers you’re using, but for the most up-to-date version the relevant code is here and it says that any models from their website can be used, not just ones from a short list like in the error you’re getting.
Can you try with rasa/LaBSE model weights? Supposedly, that won’t work either. In that case, I guess your best bet is updating transformers to a newer version 
I did:
pip install git+https://github.com/huggingface/transformers
and successfully installed tokenizers-0.10.2 transformers-4.6.0.dev0
after I run Rasa train nlu I still receive error:
OSError: Can't load weights for 'setu4993/LaBSE'. Make sure that:
- 'setu4993/LaBSE' is a correct model identifier listed on 'https://huggingface.co/models'
- or 'setu4993/LaBSE' is the correct path to a directory containing a file named one of tf_model.h5, pytorch_model.bin
I’ve check setu4993/LaBSE model at HuggingFace, the two files tf_model.h5 and pytorch_model.bin are there
I change model weights to rasa/LaBSE and got the following error:
.
Alright then. I think you should raise an issue on HuggingFace’s GitHub in this case – it looks like it’s something about the specific model weights you’re trying to use, and there’s nothing Rasa can do…
As for the error that you’re getting (now disregarding the other warnings above it because I don’t entirely understand them), most likely the training data doesn’t fit in the memory. How much NLU training data do you have? And what’s the memory size?
Hi @SamS
Hmm I tried other models instead of setu4993/LaBSE and it still return same error message.
As for the second error, yes its a memory issue and I’ve solved it. That also means that I can successfully train “rasa/LaBSE”
I would still hope to be able to use other models at HuggingFace because of the language I wanted integrate. Any syntax I can configure “LanguageModelFeaturizer” to grab from my local directory instead of downloading from HuggingFace?
“cache_dir” means to save model into that directory rather than to inform Rasa to grab pre-trained model from that directory right?
Thank you
@Hamiksu I get you, but there’s not much I can do. All that happens inside Rasa code is calling HuggingFace’s from_pretrained here with the provided name of model weights and cache_dir. The rest is left on the code in transformers. All I can recommend is to install transformers from source so that you can add some helpful logging in that code to see what exactly is going wrong…
“cache_dir” means to save model into that directory rather than to inform Rasa to grab pre-trained model from that directory right?
As far as my understanding of the code in transformers goes, it means both – if the model is not found in that directory, it gets downloaded and saved there.
