Learn how to make BERT smaller and faster

Getting Started with Rasa Tutorials, Resources & Videos
22

Hi @cyrilthank,

no - there won’t be any infringmenet since we could start from scratch with your problem - maybe that helps others aswell. Can you give us a concrete document based example on:

  1. The training data
  2. The entities that might occur
  3. The expected outcome of a classifier/extractor

Such that I can model your problem better?

Regards Julian

23

Thanks @JulianGerhard for your patient replies

I am wondering if this is best done this through our partnership relationship with rasa

Please advise how best we may leverage the current existing partnership relationship with rasa

Appreciate it if you could point to a contact in case that may help

Thanks

24

Hi @cyrilthank,

since I am not a Rasa employee I am afraid that I can’t answer this properly. However, writing @Juste could be a possible approach.

I think that linked entities / BERT pruning strategies are very interesting for the whole community so my preferred way would be that you give me an example of what you want to achieve exactly and then we try to figure it out!

Regards Julian

25

Thanks a lot @JulianGerhard Sorry based on your expertise I just assumed you were Rasa employee

@Juste can you please advise how our organization may leverage the existing rasa partnership we have to work on this

Thanks

Cyril

26

@JulianGerhard Hey Julian, I have a small question regarding the git you linked (which is very nice btw :smiley: ). I have finetuned the “en_trf_bertbaseuncased_lg” model with domain specific data and pip installed this new package. Tryind to load it with spacy fails and gives me this error:

[E149] Error deserializing model. Check that the config used to create the component matches the model being loaded.

I changed the meta.json, to register the new model name, but there seems to be a config file I need to change as well, but cant find it.

Cheers!

27

Hi @MaZe,

thanks a lot. There are several things that might cause this issue. Can you please post your spacy and spacy-transformer versions?

Did you change the meta.json before aor after packaging it with python -m spacy package ?

Regards Julian

28

Hey @JulianGerhard,

So here are my versions of spacy and spacy-transformers: spacy (2.2.3) spacy-transformers (0.5.1)

I changed the meta.json in before packaging. I change it in the FinetuningOutput folder.

EDIT: I was able to fix it in the end, and it was due to the spacy version. I am not sure why, but going back to spacy 2.2.1 solves this issue and I’m able to load the finetuned model.

Thanks.

29

Hi @MaZe,

cool - I already wondered what might have happened but I am going to investigate the thing to keep the repo updated. Feel free to come back to me if there are questions.

Regards Julian