RASA with BioBERT

Rasa Open Source
1

Hi I am trying to train my DIET based model using BIOBERT embeddings. I am getting the following error. File “/home/xxxxx/anaconda3/envs/rasa_spacy/lib/python3.7/site-packages/transformers/modeling_tf_utils.py”, line 349, in from_pretrained [WEIGHTS_NAME, TF2_WEIGHTS_NAME], pretrained_model_name_or_path OSError: Error no file named [‘pytorch_model.bin’, ‘tf_model.h5’] found in directory /home/xxxxx/model_biobert/out_base or from_pt set to False

I already have model in the folder “home/xxxxx/model_biobert/out_base” I am using the following as pipeline

language: en pipeline:

  • name: HFTransformersNLP

    model_name: “bert”

    model_weights: “/home/xxxxx/model_biobert/out_base”

  • name: LanguageModelTokenizer

  • name: LanguageModelFeaturizer

  • name: CountVectorsFeaturizer

    analyzer: char_wb

    min_ngram: 1

    max_ngram: 4

  • name: CountVectorsFeaturizer

  • name: DIETClassifier

    epochs: 100

    num_transformer_layers: 4

    transformer_size: 256

    use_masked_language_model: True

    drop_rate: 0.25

    weight_sparsity: 0.7

    batch_size: [64, 256]

    embedding_dimension: 30

    hidden_layer_sizes:

    text: [512, 128]

2

Hi @rah_pandey! I’m excited that you are using BIOBERT!! :rocket:

Are you trying to use a custom trained model from pytorch to plug into rasa? Because currently we sadly don’t support pytorch as backend

3

Hi @kalkbrennerei , yes I am trying to use the pytorch…if you dont support pytorch then the error message is misleading? How should i go about it. I downloaded the model ckpt files. Should they be converted into tfmodel files. Can you point to some step by step guide?

4

Hi @rah_pandey! There is a script for converting your pytorch model! This discussion may also help with errors that may occur.

5

Hi @rah_pandey! Did this help you? If yes, you can mark the answer as solution to help other people to find their solutions more quickly! Thanks :smiley:

6

I dont think so. Not able to implement the solution. I am now using Bio Bert outside of RASA(so sad). Let me know when you have implementation. I can give it a try again

7

I’m sorry that the script couldn’t help! We’ll let you know, when we’re building backend support for pytorch.

8

hi can you please tell me how you implemented biobert with rasa?