This thread Training fails when using HFTransformers contains the solution for the BERT-transformer. I had to add cache_dir: /tmp
to the pipeline component.
However after training, I’ve got this error in my production container log :
**********************************************************************
Resource vader_lexicon not found.
Please use the NLTK Downloader to obtain the resource:
>>> import nltk
>>> nltk.download('vader_lexicon')
For more information see: https://www.nltk.org/data.html
Attempted to load sentiment/vader_lexicon.zip/vader_lexicon/vader_lexicon.txt
Searched in:
- '/nltk_data'
- '/opt/venv/nltk_data'
- '/opt/venv/share/nltk_data'
- '/opt/venv/lib/nltk_data'
- '/usr/share/nltk_data'
- '/usr/local/share/nltk_data'
- '/usr/lib/nltk_data'
- '/usr/local/lib/nltk_data'
- ''
**********************************************************************
Did I install the lexicon incorrectly or in the wrong location?