The code is always stuck here when training or runing

os: ubuntu 20.4 rasa v3.0.8 (just updated yesterday) when I trained a rasa bot or ran a trained bot recently, the code always stucked for some minutes here is a example in debug mode:

/home/xxx/rasa_env/lib/python3.8/site-packages/rasa/shared/core/slot_mappings.py:219: UserWarning: Slot auto-fill has been removed in 3.0 and replaced with a new explicit mechanism to set slots. Please refer to https://rasa.com/docs/rasa/domain#slots to learn more.
  rasa.shared.utils.io.raise_warning(
2022-05-08 06:26:55 DEBUG    h5py._conv  - Creating converter from 7 to 5
2022-05-08 06:26:55 DEBUG    h5py._conv  - Creating converter from 5 to 7
2022-05-08 06:26:55 DEBUG    h5py._conv  - Creating converter from 7 to 5
2022-05-08 06:26:55 DEBUG    h5py._conv  - Creating converter from 5 to 7
/home/xxx/rasa_env/lib/python3.8/site-packages/flatbuffers/compat.py:19: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
/home/xxx/rasa_env/lib/python3.8/site-packages/sklearn/utils/multiclass.py:14: DeprecationWarning: Please use `spmatrix` from the `scipy.sparse` namespace, the `scipy.sparse.base` namespace is deprecated.
  from scipy.sparse.base import spmatrix
2022-05-08 06:26:56 DEBUG    urllib3.connectionpool  - Starting new HTTPS connection (1): api.segment.io:443
2022-05-08 06:26:57 DEBUG    urllib3.connectionpool  - https://api.segment.io:443 "POST /v1/track HTTP/1.1" 200 21
2022-05-08 06:26:57 DEBUG    rasa.cli.utils  - Parameter 'credentials' not set. Using default location 'credentials.yml' instead.
2022-05-08 06:26:57 INFO     root  - Connecting to channel 'cmdline' which was specified by the '--connector' argument. Any other channels will be ignored. To connect to all given channels, omit the '--connector' argument.
2022-05-08 06:26:57 DEBUG    sanic.root  - Sanic-CORS: Configuring CORS with resources: {'/*': {'origins': [''], 'methods': 'DELETE, GET, HEAD, OPTIONS, PATCH, POST, PUT', 'allow_headers': ['.*'], 'expose_headers': 'filename', 'supports_credentials': True, 'max_age': None, 'send_wildcard': False, 'automatic_options': True, 'vary_header': True, 'resources': {'/*': {'origins': ''}}, 'intercept_exceptions': True, 'always_send': True}}
2022-05-08 06:26:57 DEBUG    rasa.core.utils  - Available web server routes: 
/webhooks/rest                                     GET                            rasa_core_no_api.custom_webhook_CmdlineInput.health
/webhooks/rest/webhook                             POST                           rasa_core_no_api.custom_webhook_CmdlineInput.receive
/                                                  GET                            rasa_core_no_api.hello
2022-05-08 06:26:57 INFO     root  - Starting Rasa server on http://0.0.0.0:5005
2022-05-08 06:26:57 DEBUG    rasa.core.utils  - Using the default number of Sanic workers (1).
/home/xxx/rasa_env/lib/python3.8/site-packages/rasa/shared/core/slot_mappings.py:219: UserWarning: Slot auto-fill has been removed in 3.0 and replaced with a new explicit mechanism to set slots. Please refer to https://rasa.com/docs/rasa/domain#slots to learn more.
  rasa.shared.utils.io.raise_warning(
2022-05-08 06:26:58 DEBUG    urllib3.connectionpool  - Starting new HTTPS connection (1): api.segment.io:443
2022-05-08 06:26:58 DEBUG    urllib3.connectionpool  - https://api.segment.io:443 "POST /v1/track HTTP/1.1" 200 21
2022-05-08 06:26:58 DEBUG    rasa.core.tracker_store  - Connected to InMemoryTrackerStore.
2022-05-08 06:26:58 DEBUG    rasa.core.lock_store  - Connected to lock store 'InMemoryLockStore'.
2022-05-08 06:26:58 DEBUG    rasa.core.nlg.generator  - Instantiated NLG to 'TemplatedNaturalLanguageGenerator'.
2022-05-08 06:26:58 INFO     rasa.core.processor  - Loading model models/20220508-062240-many-sortie.tar.gz...
2022-05-08 06:26:59 DEBUG    rasa.engine.storage.local_model_storage  - Extracted model to '/tmp/tmprfk08i7_'.
2022-05-08 06:26:59 DEBUG    rasa.engine.graph  - Node 'nlu_message_converter' loading 'NLUMessageConverter.load' and kwargs: '{}'.
2022-05-08 06:26:59 DEBUG    rasa.engine.graph  - Node 'run_JiebaTokenizer0' loading 'JiebaTokenizer.load' and kwargs: '{}'.
2022-05-08 06:26:59 DEBUG    rasa.engine.graph  - Node 'run_LanguageModelFeaturizer1' loading 'LanguageModelFeaturizer.load' and kwargs: '{}'.
2022-05-08 06:26:59 DEBUG    rasa.nlu.featurizers.dense_featurizer.lm_featurizer  - Loading Tokenizer and Model for bert
2022-05-08 06:26:59 DEBUG    urllib3.connectionpool  - Starting new HTTPS connection (1): huggingface.co:443
2022-05-08 06:27:40 DEBUG    urllib3.connectionpool  - https://huggingface.co:443 "HEAD /bert-base-chinese/resolve/main/vocab.txt HTTP/1.1" 200 0
2022-05-08 06:27:40 DEBUG    urllib3.connectionpool  - Starting new HTTPS connection (1): huggingface.co:443
2022-05-08 06:28:21 DEBUG    urllib3.connectionpool  - https://huggingface.co:443 "HEAD /bert-base-chinese/resolve/main/added_tokens.json HTTP/1.1" 404 0
2022-05-08 06:28:21 DEBUG    urllib3.connectionpool  - Starting new HTTPS connection (1): huggingface.co:443
2022-05-08 06:29:02 DEBUG    urllib3.connectionpool  - https://huggingface.co:443 "HEAD /bert-base-chinese/resolve/main/special_tokens_map.json HTTP/1.1" 404 0
2022-05-08 06:29:02 DEBUG    urllib3.connectionpool  - Starting new HTTPS connection (1): huggingface.co:443
2022-05-08 06:29:43 DEBUG    urllib3.connectionpool  - https://huggingface.co:443 "HEAD /bert-base-chinese/resolve/main/tokenizer_config.json HTTP/1.1" 200 0
2022-05-08 06:29:43 DEBUG    urllib3.connectionpool  - Starting new HTTPS connection (1): huggingface.co:443
2022-05-08 06:30:24 DEBUG    urllib3.connectionpool  - https://huggingface.co:443 "HEAD /bert-base-chinese/resolve/main/config.json HTTP/1.1" 200 0
2022-05-08 06:30:24 DEBUG    urllib3.connectionpool  - Starting new HTTPS connection (1): huggingface.co:443
2022-05-08 06:31:05 DEBUG    urllib3.connectionpool  - https://huggingface.co:443 "HEAD /bert-base-chinese/resolve/main/config.json HTTP/1.1" 200 0
2022-05-08 06:31:05 DEBUG    urllib3.connectionpool  - Starting new HTTPS connection (1): huggingface.co:443
2022-05-08 06:31:46 DEBUG    urllib3.connectionpool  - https://huggingface.co:443 "HEAD /bert-base-chinese/resolve/main/tf_model.h5 HTTP/1.1" 302 0

Last week I never came across such a situation, and the code loading in some second and then run smoothly until complete training or servering.

In terms of the debug info, it seemed that rasa queried some info from huggingface.co, before training or serving( loading a model).

Anyone can help me to speed up to start model training or start serving, as it is really boring waiting for some minutes to run a rasa bot.

thanks in advance!!!

Looks like something related to your pipeline. Can you share the config.yml and confirm that this is for Chinese?

@stephens I hava same problem. here is my config.yml:

rasa 3.0.8

@rctatman Do you know what could be going on here?

Hmm… did y’all specify that slots with the same name as an entity are explicitly filled using that entity? It’s new in 3.X. Domain

Yeah, but I don’t think that’s the problem here. Also, I tried adding an underscore to the entity name to differentiate it. Like the code snippet below, but the problem is not solved.


slots:
  intent:
    type: text
    mappings:
      - type: custom

  area:
    type: text
    mappings:
      - type: from_entity
        entity: area_
      - type: custom

  branch:
    type: text
    mappings:
      - type: from_entity
        entity: branch_
      - type: custom

key of the log:

2022-05-18 17:05:48 DEBUG    urllib3.connectionpool  - Starting new HTTPS connection (1): huggingface.co:443
2022-05-18 17:05:49 DEBUG    urllib3.connectionpool  - https://huggingface.co:443 "HEAD /hfl/chinese-roberta-wwm-ext/resolve/main/vocab.txt HTTP/1.1" 200 0
2022-05-18 17:05:49 DEBUG    urllib3.connectionpool  - Starting new HTTPS connection (1): huggingface.co:443
2022-05-18 17:05:50 DEBUG    urllib3.connectionpool  - https://huggingface.co:443 "HEAD /hfl/chinese-roberta-wwm-ext/resolve/main/added_tokens.json HTTP/1.1" 200 0
2022-05-18 17:05:50 DEBUG    urllib3.connectionpool  - Starting new HTTPS connection (1): huggingface.co:443
2022-05-18 17:05:51 DEBUG    urllib3.connectionpool  - https://huggingface.co:443 "HEAD /hfl/chinese-roberta-wwm-ext/resolve/main/special_tokens_map.json HTTP/1.1" 200 0
2022-05-18 17:05:51 DEBUG    urllib3.connectionpool  - Starting new HTTPS connection (1): huggingface.co:443
2022-05-18 17:05:52 DEBUG    urllib3.connectionpool  - https://huggingface.co:443 "HEAD /hfl/chinese-roberta-wwm-ext/resolve/main/tokenizer_config.json HTTP/1.1" 200 0
2022-05-18 17:05:52 DEBUG    urllib3.connectionpool  - Starting new HTTPS connection (1): huggingface.co:443
2022-05-18 17:05:53 DEBUG    urllib3.connectionpool  - https://huggingface.co:443 "HEAD /hfl/chinese-roberta-wwm-ext/resolve/main/config.json HTTP/1.1" 200 0
2022-05-18 17:05:53 DEBUG    urllib3.connectionpool  - Starting new HTTPS connection (1): huggingface.co:443
2022-05-18 17:05:54 DEBUG    urllib3.connectionpool  - https://huggingface.co:443 "HEAD /hfl/chinese-roberta-wwm-ext/resolve/main/config.json HTTP/1.1" 200 0
2022-05-18 17:05:54 DEBUG    urllib3.connectionpool  - Starting new HTTPS connection (1): huggingface.co:443
2022-05-18 17:05:55 DEBUG    urllib3.connectionpool  - https://huggingface.co:443 "HEAD /hfl/chinese-roberta-wwm-ext/resolve/main/tf_model.h5 HTTP/1.1" 302 0

This looks like getting model information from huggingface website.