New trained model cannot be loaded: No module named 'scipy.sparse._coo'

[Deprecated] Rasa X Community Edition
1

Whenever a new model is trained, I get the following error, Any ideas?

Traceback (most recent call last):
  File "/opt/venv/lib/python3.8/site-packages/rasa/core/agent.py", line 160, in _update_model_from_server
    _load_and_set_updated_model(agent, model_directory, new_fingerprint)
  File "/opt/venv/lib/python3.8/site-packages/rasa/core/agent.py", line 133, in _load_and_set_updated_model
    interpreter = _load_interpreter(agent, nlu_path)
  File "/opt/venv/lib/python3.8/site-packages/rasa/core/agent.py", line 92, in _load_interpreter
    return rasa.core.interpreter.create_interpreter(nlu_path)
  File "/opt/venv/lib/python3.8/site-packages/rasa/core/interpreter.py", line 33, in create_interpreter
    return RasaNLUInterpreter(model_directory=obj)
  File "/opt/venv/lib/python3.8/site-packages/rasa/core/interpreter.py", line 127, in __init__
    self._load_interpreter()
  File "/opt/venv/lib/python3.8/site-packages/rasa/core/interpreter.py", line 165, in _load_interpreter
    self.interpreter = Interpreter.load(self.model_directory)
  File "/opt/venv/lib/python3.8/site-packages/rasa/nlu/model.py", line 341, in load
    return Interpreter.create(
  File "/opt/venv/lib/python3.8/site-packages/rasa/nlu/model.py", line 416, in create
    component = component_builder.load_component(
  File "/opt/venv/lib/python3.8/site-packages/rasa/nlu/components.py", line 902, in load_component
    component = registry.load_component_by_meta(
  File "/opt/venv/lib/python3.8/site-packages/rasa/nlu/registry.py", line 177, in load_component_by_meta
    return component_class.load(
  File "/opt/venv/lib/python3.8/site-packages/rasa/nlu/classifiers/diet_classifier.py", line 1075, in load
    ) = cls._load_from_files(meta, model_dir)
  File "/opt/venv/lib/python3.8/site-packages/rasa/nlu/classifiers/diet_classifier.py", line 1115, in _load_from_files
    data_example = io_utils.pickle_load(model_dir / f"{file_name}.data_example.pkl")
  File "/opt/venv/lib/python3.8/site-packages/rasa/utils/io.py", line 81, in pickle_load
    return pickle.load(f)
ModuleNotFoundError: No module named 'scipy.sparse._coo'
2022-02-24 19:45:25 DEBUG rasa.core.agent - Unzipped model to '/tmp/tmph0624cs9' 2022-02-24 19:45:25 DEBUG rasa.core.agent - Found new model with fingerprint 5a3f95e601eca715afdf9e0c8739dbfd. Loading... 2022-02-24 19:45:25 ERROR rasa.core.agent - Failed to update model. The previous model will stay loaded instead. Traceback (most recent call last): File "/opt/venv/lib/python3.8/site-packages/rasa/core/agent.py", line 160, in _update_model_from_server _load_and_set_updated_model(agent, model_directory, new_fingerprint) File "/opt/venv/lib/python3.8/site-packages/rasa/core/agent.py", line 133, in _load_and_set_updated_model interpreter = _load_interpreter(agent, nlu_path) File "/opt/venv/lib/python3.8/site-packages/rasa/core/agent.py", line 92, in _load_interpreter return rasa.core.interpreter.create_interpreter(nlu_path) File "/opt/venv/lib/python3.8/site-packages/rasa/core/interpreter.py", line 33, in create_interpreter return RasaNLUInterpreter(model_directory=obj) File "/opt/venv/lib/python3.8/site-packages/rasa/core/interpreter.py", line 127, in __init__ self._load_interpreter() File "/opt/venv/lib/python3.8/site-packages/rasa/core/interpreter.py", line 165, in _load_interpreter self.interpreter = Interpreter.load(self.model_directory) File "/opt/venv/lib/python3.8/site-packages/rasa/nlu/model.py", line 341, in load return Interpreter.create( File "/opt/venv/lib/python3.8/site-packages/rasa/nlu/model.py", line 416, in create component = component_builder.load_component( File "/opt/venv/lib/python3.8/site-packages/rasa/nlu/components.py", line 902, in load_component component = registry.load_component_by_meta( File "/opt/venv/lib/python3.8/site-packages/rasa/nlu/registry.py", line 177, in load_component_by_meta return component_class.load( File "/opt/venv/lib/python3.8/site-packages/rasa/nlu/classifiers/diet_classifier.py", line 1075, in load ) = cls._load_from_files(meta, model_dir) File "/opt/venv/lib/python3.8/site-packages/rasa/nlu/classifiers/diet_classifier.py", line 1115, in _load_from_files data_example = io_utils.pickle_load(model_dir / f"{file_name}.data_example.pkl") 
File "/opt/venv/lib/python3.8/site-packages/rasa/utils/io.py", line 81, in pickle_load return pickle.load(f) ModuleNotFoundError: No module named 'scipy.sparse._coo'
2

I’m running Rasa versions 2.8.14 and Rasa X 0.42.6.

The errors appear in the rasa-production logs.

The logs on the initialization of the server shows the following:

I added pip list to the logs, and the Scipy that is installed is 1.5.X

3

Maybe try to use conda install Scipy instead of pip ?

4

I got same problem. I follow this. Downgrade scipy at local to 1.7.3 and retrain model. Then upload the model to my Rasa X server and it work, no error from rasa-production logs.

5

I had to downgrade Scipy as indicated above.

6

I downgraded to Scipy 1.7.3 in my local machine (where the model is trained) and then re-uploaded to Rasa X, but still getting the same error. Any ideas?

image
image1724×426 93.8 KB

7

I didn’t solve my problem by training locally and uploading. I actually had to downgrade Scipy on the rasa-production docker container.

I already had a Dockerfile for it (usually you only have a Dockerfile for the custom action server) and then updated the docker-compose file to use this new Dockerfile. In that Dockerfile, I have the following lines:

RUN pip3 install --force-reinstall scipy==1.5.4
RUN pip3 install --force-reinstall numpy==1.18.5

Hope that helps!

8

Hi @arneverster, how do I find such Dockerfile? I’ve installed Rasa X 1.1.0 with the Ephemeral installer.

9

Well, apparently what was missing from my side after re-training locally with scipy 1.7.3 and uploading back to Rasa X was to refresh the browser’s cache. Now it’s working :slight_smile:

10

HI, i’m having this issue right now. I already reverted to scipy version 1.7.3 but its not working. I still get the same error. I am not using rasa X right now im using rasa open source command line inside a docker container.