Hi I am trying to train my DIET based model using BIOBERT embeddings. I am getting the following error. File “/home/xxxxx/anaconda3/envs/rasa_spacy/lib/python3.7/site-packages/transformers/modeling_tf_utils.py”, line 349, in from_pretrained [WEIGHTS_NAME, TF2_WEIGHTS_NAME], pretrained_model_name_or_path OSError: Error no file named [‘pytorch_model.bin’, ‘tf_model.h5’] found in directory /home/xxxxx/model_biobert/out_base or from_pt set to False

I already have model in the folder “home/xxxxx/model_biobert/out_base” I am using the following as pipeline

language: en pipeline:

  • name: HFTransformersNLP

    model_name: “bert”

    model_weights: “/home/xxxxx/model_biobert/out_base”

  • name: LanguageModelTokenizer

  • name: LanguageModelFeaturizer

  • name: CountVectorsFeaturizer

    analyzer: char_wb

    min_ngram: 1

    max_ngram: 4

  • name: CountVectorsFeaturizer

  • name: DIETClassifier

    epochs: 100

    num_transformer_layers: 4

    transformer_size: 256

    use_masked_language_model: True

    drop_rate: 0.25

    weight_sparsity: 0.7

    batch_size: [64, 256]

    embedding_dimension: 30


    text: [512, 128]

Hi @rah_pandey! I’m excited that you are using BIOBERT!! :rocket:

Are you trying to use a custom trained model from pytorch to plug into rasa? Because currently we sadly don’t support pytorch as backend

Hi @kalkbrennerei , yes I am trying to use the pytorch…if you dont support pytorch then the error message is misleading? How should i go about it. I downloaded the model ckpt files. Should they be converted into tfmodel files. Can you point to some step by step guide?

Hi @rah_pandey! There is a script for converting your pytorch model! This discussion may also help with errors that may occur.

Hi @rah_pandey! Did this help you? If yes, you can mark the answer as solution to help other people to find their solutions more quickly! Thanks :smiley:

I dont think so. Not able to implement the solution. I am now using Bio Bert outside of RASA(so sad). Let me know when you have implementation. I can give it a try again

I’m sorry that the script couldn’t help! We’ll let you know, when we’re building backend support for pytorch.

hi can you please tell me how you implemented biobert with rasa?