Hey everyone,
Recently I’ve migrated from 2.x to 3.5.0. I’ve included LanguageModelFeaturizer in my pipeline trained it on ec2 instance (Let’s say instance A). Training was successful and able to load the trained nlu model on same instance A(with rasa run) and get the predictions also. However when I downloaded the model to local and copied it to some other ec2 instance(say instance B), im unable to load the model because it’s getting killed. Attaching logs below.
2023-08-04 11:39:44 INFO root - Starting Rasa server on http://0.0.0.0:5005
2023-08-04 11:39:44 DEBUG rasa.core.utils - Using the default number of Sanic workers (1).
<frozen importlib._bootstrap>:283: DeprecationWarning: the load_module() method is deprecated and slated for removal in Python 3.12; use exec_module() instead
2023-08-04 11:39:45 DEBUG urllib3.connectionpool - Starting new HTTPS connection (1): api.segment.io:443
2023-08-04 11:39:46 DEBUG urllib3.connectionpool - https://api.segment.io:443 "POST /v1/track HTTP/1.1" 200 21
2023-08-04 11:39:46 DEBUG rasa.core.tracker_store - Connected to InMemoryTrackerStore.
2023-08-04 11:39:46 DEBUG rasa.core.lock_store - Connected to lock store 'InMemoryLockStore'.
2023-08-04 11:39:46 DEBUG rasa.core.nlg.generator - Instantiated NLG to 'TemplatedNaturalLanguageGenerator'.
2023-08-04 11:39:46 INFO rasa.core.processor - Loading model nlu-20230803-122108-natural-suitcase.tar.gz...
2023-08-04 11:39:47 DEBUG rasa.engine.storage.local_model_storage - Extracted model to '/tmp/tmpe52q2zsv'.
2023-08-04 11:39:47 DEBUG rasa.engine.graph - Node 'nlu_message_converter' loading 'NLUMessageConverter.load' and kwargs: '{}'.
2023-08-04 11:39:47 DEBUG rasa.engine.graph - Node 'run_WhitespaceTokenizer0' loading 'WhitespaceTokenizer.load' and kwargs: '{}'.
2023-08-04 11:39:47 DEBUG rasa.engine.graph - Node 'run_RegexFeaturizer1' loading 'RegexFeaturizer.load' and kwargs: '{}'.
2023-08-04 11:39:47 DEBUG rasa.engine.storage.local_model_storage - Resource 'train_RegexFeaturizer1' was requested for reading.
2023-08-04 11:39:47 DEBUG rasa.engine.graph - Node 'run_LexicalSyntacticFeaturizer2' loading 'LexicalSyntacticFeaturizer.load' and kwargs: '{}'.
2023-08-04 11:39:47 DEBUG rasa.engine.storage.local_model_storage - Resource 'train_LexicalSyntacticFeaturizer2' was requested for reading.
2023-08-04 11:39:47 DEBUG rasa.engine.graph - Node 'run_CountVectorsFeaturizer3' loading 'CountVectorsFeaturizer.load' and kwargs: '{}'.
2023-08-04 11:39:47 DEBUG rasa.engine.storage.local_model_storage - Resource 'train_CountVectorsFeaturizer3' was requested for reading.
2023-08-04 11:39:47 DEBUG rasa.engine.graph - Node 'run_LanguageModelFeaturizer4' loading 'LanguageModelFeaturizer.load' and kwargs: '{}'.
2023-08-04 11:39:48 DEBUG rasa.nlu.featurizers.dense_featurizer.lm_featurizer - Loading Tokenizer and Model for bert
2023-08-04 11:39:48 DEBUG urllib3.connectionpool - Starting new HTTPS connection (1): huggingface.co:443
2023-08-04 11:39:48 DEBUG urllib3.connectionpool - https://huggingface.co:443 "HEAD /rasa/LaBSE/resolve/main/vocab.txt HTTP/1.1" 200 0
2023-08-04 11:39:50 DEBUG urllib3.connectionpool - https://huggingface.co:443 "HEAD /rasa/LaBSE/resolve/main/config.json HTTP/1.1" 200 0
Killed
And below is how my nlu pipeline looks like
recipe: default.v1
language: en
pipeline:
- name: WhitespaceTokenizer
- name: RegexFeaturizer
use_word_boundaries: true
case_sensitive: false
- name: LexicalSyntacticFeaturizer
- name: CountVectorsFeaturizer
- name: LanguageModelFeaturizer
model_name: bert
model_weights: rasa/LaBSE
cache_dir: "./cache"
- name: DIETClassifier
epochs: 200
- name: EntitySynonymMapper
- name: ResponseSelector
epochs: 100
constrain_similarities: true
- name: FallbackClassifier
threshold: 0.95
ambiguity_threshold: 0.1
Please help if anyone knows how to proceed. May be I might be missing out on cache and /tmp directories, but not sure on how to proceed and make it work. Without LanguageModelFeaturizer, I was able to load the model from anywhere I wanted.
Thanks in advance