Rasa X Docker ModuleNotFoundError with a Custom Component

[Deprecated] Rasa X Community Edition
1

Hello,

I’m trying to deploy an instance of Rasa X 0.24.6 with Rasa 1.6.1 using Docker. The model has been trained on a different machine using local Rasa 1.6.1 installation without Docker and is loaded into Rasa X. docker-compose.yml is the default version, the contents of docker-compose.override.yml

version: '3.4'
services:
  app:
    image: myimage:latest
    environment:
      - PYTHONPATH=/app/
    volumes:
      - ./actions:/app/actions
    expose:
      - "5055"

The app uses a custom component customNLU.py and is packed with the following Dockerfile:

FROM rasa/rasa-sdk:1.6.1
COPY ./* /app/
ENV PYTHONPATH=$PYTHONPATH:/app/

That is, the structure of the app is pretty flat:

- app
   - __init__.py
   - actions
       - __init__.py
       - actions.py
   - customNLU.py
   - config.yml
   - [everything else]

The content of config.yml:

language: de_core_news_sm
pipeline:
- name: "SpacyNLP"
- name: "SpacyTokenizer"
- name: "SpacyFeaturizer"
- name: "RegexFeaturizer"
- name: "CRFEntityExtractor"
- name: "EntitySynonymMapper"
- name: "SklearnIntentClassifier"
- name: "customNLU.MinimalKeywordSearch"

policies:
- name: MemoizationPolicy
- name: MappingPolicy
- name: KerasPolicy

Every docker-compose up call throws a sequence of errors:

rasa-production_1  | 2020-01-29 14:58:02 DEBUG    rasa.core.agent  - Found new model with fingerprint f31b006c6e2504bcdc051c4b3f532bb6. Loading...
rasa-x_1           | INFO:rasax.community.services.event_consumers.pika_consumer:Start consuming queue 'rasa_production_events' on pika host 'rabbit'.
rabbit_1           |  14:58:11.73 INFO  ==> Stopping RabbitMQ...
rasa-production_1  | 2020-01-29 14:58:12 ERROR    rasa.core.agent  - Could not load model due to Failed to find module 'customNLU'. 
rasa-production_1  | No module named 'customNLU'.
rasa-production_1  | [2020-01-29 14:58:12 +0000] [1] [ERROR] Experienced exception while trying to serve
rasa-production_1  | Traceback (most recent call last):
rasa-production_1  |   File "/build/lib/python3.6/site-packages/rasa/nlu/registry.py", line 154, in get_component_class
rasa-production_1  |     return class_from_module_path(component_name)
rasa-production_1  |   File "/build/lib/python3.6/site-packages/rasa/utils/common.py", line 195, in class_from_module_path
rasa-production_1  |     m = importlib.import_module(module_name)
rasa-production_1  |   File "/usr/local/lib/python3.6/importlib/__init__.py", line 126, in import_module
rasa-production_1  |     return _bootstrap._gcd_import(name[level:], package, level)
rasa-production_1  |   File "<frozen importlib._bootstrap>", line 994, in _gcd_import
rasa-production_1  |   File "<frozen importlib._bootstrap>", line 971, in _find_and_load
rasa-production_1  |   File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
rasa-production_1  | ModuleNotFoundError: No module named 'customNLU'
rasa-production_1  | 
rasa-production_1  | During handling of the above exception, another exception occurred:
rasa-production_1  | 
rasa-production_1  | Traceback (most recent call last):
rasa-production_1  |   File "/build/lib/python3.6/site-packages/sanic/app.py", line 1133, in run
rasa-production_1  |     serve(**server_settings)
rasa-production_1  |   File "/build/lib/python3.6/site-packages/sanic/server.py", line 857, in serve
rasa-production_1  |     trigger_events(before_start, loop)
rasa-production_1  |   File "/build/lib/python3.6/site-packages/sanic/server.py", line 634, in trigger_events
rasa-production_1  |     loop.run_until_complete(result)
rasa-production_1  |   File "uvloop/loop.pyx", line 1456, in uvloop.loop.Loop.run_until_complete
rasa-production_1  |   File "/build/lib/python3.6/site-packages/rasa/core/run.py", line 247, in load_agent_on_start
rasa-production_1  |     action_endpoint=endpoints.action,
rasa-production_1  |   File "/build/lib/python3.6/site-packages/rasa/core/agent.py", line 248, in load_agent
rasa-production_1  |     model_server,
rasa-production_1  |   File "/build/lib/python3.6/site-packages/rasa/core/agent.py", line 64, in load_from_server
rasa-production_1  |     await _update_model_from_server(model_server, agent)
rasa-production_1  |   File "/build/lib/python3.6/site-packages/rasa/core/agent.py", line 126, in _update_model_from_server
rasa-production_1  |     _load_and_set_updated_model(agent, model_directory, new_model_fingerprint)
rasa-production_1  |   File "/build/lib/python3.6/site-packages/rasa/core/agent.py", line 87, in _load_and_set_updated_model
rasa-production_1  |     interpreter = RasaNLUInterpreter(model_directory=nlu_path)
rasa-production_1  |   File "/build/lib/python3.6/site-packages/rasa/core/interpreter.py", line 272, in __init__
rasa-production_1  |     self._load_interpreter()
rasa-production_1  |   File "/build/lib/python3.6/site-packages/rasa/core/interpreter.py", line 295, in _load_interpreter
rasa-production_1  |     self.interpreter = Interpreter.load(self.model_directory)
rasa-production_1  |   File "/build/lib/python3.6/site-packages/rasa/nlu/model.py", line 301, in load
rasa-production_1  |     return Interpreter.create(model_metadata, component_builder, skip_validation)
rasa-production_1  |   File "/build/lib/python3.6/site-packages/rasa/nlu/model.py", line 323, in create
rasa-production_1  |     components.validate_requirements(model_metadata.component_classes)
rasa-production_1  |   File "/build/lib/python3.6/site-packages/rasa/nlu/components.py", line 38, in validate_requirements
rasa-production_1  |     component_class = registry.get_component_class(component_name)
rasa-production_1  |   File "/build/lib/python3.6/site-packages/rasa/nlu/registry.py", line 180, in get_component_class
rasa-production_1  |     raise ModuleNotFoundError(exception_message)
rasa-production_1  | ModuleNotFoundError: Failed to find module 'customNLU'. 
rasa-production_1  | No module named 'customNLU'
rasa-production_1  | Starting Rasa X in production mode... 🚀
rasa-production_1  | Traceback (most recent call last):
rasa-production_1  |   File "/build/lib/python3.6/site-packages/rasa/nlu/registry.py", line 154, in get_component_class
rasa-production_1  |     return class_from_module_path(component_name)
rasa-production_1  |   File "/build/lib/python3.6/site-packages/rasa/utils/common.py", line 195, in class_from_module_path
rasa-production_1  |     m = importlib.import_module(module_name)
rasa-production_1  |   File "/usr/local/lib/python3.6/importlib/__init__.py", line 126, in import_module
rasa-production_1  |     return _bootstrap._gcd_import(name[level:], package, level)
rasa-production_1  |   File "<frozen importlib._bootstrap>", line 994, in _gcd_import
rasa-production_1  |   File "<frozen importlib._bootstrap>", line 971, in _find_and_load
rasa-production_1  |   File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
rasa-production_1  | ModuleNotFoundError: No module named 'customNLU'

and the same error within rasa-worker. I checked out similar issues here on the forum and github and tried putting PYTHONPATH in various containers, I can see /app/actions when executing echo $PYTHONPATH within the running app container. However, the whole service never starts completely and shuts down eventually.

Is there any clue what could go wrong here? Thanks!

2

I’m still looking into this, but since the custom component is used during the training/execution of the model and not just the action server, it somehow needs to be added to the rasa services, not the app service. Will update if I find something out.

3

+1 for this.

4

@sureshvarman @mgalkin as @mloubser said, the file needs to be mounted to the rasa containers. You can achieve this in the docker-compose.override.yml:

version: '3.4'
services:
  rasa-production:
    volumes:
      - ./customNLU.py:/app/customNLU.py
  rasa-worker:
    volumes:
      - ./customNLU.py:/app/customNLU.py
4 Likes
5

@sureshvarman @mgalkin did my comment help you resolve your issue?

6

@erohmensing I followed your advise after adding a pre-trained sentiment analyzer (nltk) as a custom component. It is mounted to the containers and trained, but now the interactive learning does not react, predicts no intent and loads forever. When using the share feature to test the model the user’s messages are not even shown. Locally everything works fine.

EDIT: It seems like it has to do with the requirements of the component. I want to use the vader lexicon, which requires me to run a command nltk.download('vader_lexicon)'. How can I add the requirements to the rasa containers? I can pip install by accessing the containers, but having to do this every time I recompose is not fun. Also there is still no way to download the lexicon into the containers.

7

Just came across this again - You could also load in the lexicon as a volume, you’d just need to put it in a directory NLTK is aware of. Then you wouldn’t have to download it inside the container.

8

Thanks, I actually already solved this problem. It indeed saved the lexicon in an inaccessible location, so I just moved it to the usr folder.

9

Hi @erohmensing and @mloubser how can the same thing be achieved when deployment is done by use of Helm chart?

I am assuming I’ll have to add a line in Dockerfile to copy the required files to rasa-production and rasa-worker pods. Please correct me if I am wrong.

Thanks in advance!

10

Yes, you could extend the rasa image to include copying the lexicon into the image.

11

Hey @mloubser, I am not having much background about the working of helm charts. I saw a lot of discussions in the forum for rasa x deployed with docker. So I am not very clear about where exactly will the files copying should be done and how… :pensive:

All I could make out from the discussions is - the copying should be specified here or here. I am NOT sure where it will sit also, I am not sure how to specify it.

It will be really very helpful if you could help!! :pray:

12

Helm charts are used to create a kubernetes deployment. They still use docker images, so passing the same custom image in the helm chart as you were passing in docker-compose should work fine. In that case, you would update your values.yml like this:

rasa:
    name: "<your-custom-image-name e.g. mydockerhubaccount/rasa"
    # tag of the Rasa Open Source image to use
    tag: "<your-custom-image-tag e.g. lexicon"

Alternatively, this PR on the helm charts: Add support for extra volumes by tczekajlo · Pull Request #74 · RasaHQ/rasa-x-helm · GitHub made it possible to mount extra volumes on the containers in helm. Then you’d put something like this in values.yml:

rasa:
...
  extraVolumeMounts: 
    - name: lexicon
      mountPath: <path>

I believe you’d need to use a configMap to create the volume to mount.

1 Like
13

@mloubser you are a saviour! :partying_face: I followed the first way of creating custom image and passing its name and its working!

Thanks a ton!

1 Like
14

Do we not need to mount these to the worker and production pods as well? After mounting to the rasa pod, my worker pod is complaining that the component does not exist in the path when I try to train a model.

rasa:
# token Rasa accepts as authentication token from other Rasa services
token: xxxxxxxxx
tag: "2.0.2-full"
extraVolumeMounts:
    - name: MyComponent
      mountPath: /app/my_component.py
extraVolume:
    - name: MyComponent
      hostPath:
          path: ./my_package/components/my_component.py
          type: DirectoryOrCreate

language: en
pipeline:
  - name: WhitespaceTokenizer
  - name: RegexFeaturizer
  - name: LexicalSyntacticFeaturizer
  - name: CountVectorsFeaturizer
  - name: CountVectorsFeaturizer
    analyzer: char_wb
    min_ngram: 1
    max_ngram: 4
  - name: DIETClassifier
    epochs: 100
  - name: EntitySynonymMapper
  - name: my_component.MyComponent
  - name: FallbackClassifier
    threshold: 0.3
    ambiguity_threshold: 0.1
policies: null

        2021-01-25 00:26:36 ERROR    rasa.server  - Traceback (most recent call last):
      File "/opt/venv/lib/python3.7/site-packages/rasa/nlu/registry.py", line 121, in get_component_class
        return rasa.shared.utils.common.class_from_module_path(component_name)
      File "/opt/venv/lib/python3.7/site-packages/rasa/shared/utils/common.py", line 19, in class_from_module_path
        m = importlib.import_module(module_name)
      File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
        return _bootstrap._gcd_import(name[level:], package, level)
      File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
      File "<frozen importlib._bootstrap>", line 983, in _find_and_load
      File "<frozen importlib._bootstrap>", line 965, in _find_and_load_unlocked
    ModuleNotFoundError: No module named 'my_component'
15

Yes, you need to mount it to both worker and prod pods. No need to mount it to Rasa X though. It might just be a format issue for you based on the code snippet - here’s an example of what it would look like:

rasa:  
  versions: 
    rasaProduction: 
      extraVolumeMounts:
        - name: MyComponent
          mountPath: /app/my_component.py
     extraVolume:
        - name: MyComponent
          hostPath:
              path: ./my_package/components/my_component.py
              type: DirectoryOrCreate
        
    rasaWorker: 
      extraVolumeMounts:
        - name: MyComponent
          mountPath: /app/my_component.py
     extraVolume:
        - name: MyComponent
          hostPath:
              path: ./my_package/components/my_component.py
              type: DirectoryOrCreate
1 Like
16

Thanks, @mloubser!

Two modifications are required for anyone that comes across this thread.

  1. change extraVolume to extraVolumes
  2. relative paths are not supported, so you’ll need to use the absolute path (which is why I believe Melinda recommended using a configMap)

This results in the following:

rasa:  
  versions: 
    rasaProduction: 
      extraVolumeMounts:
        - name: MyComponent
          mountPath: /app/components
      extraVolumes:
        - name: MyComponent
          hostPath:
              path: /path/to/your/project/my_package/components
              type: DirectoryOrCreate
        
    rasaWorker: 
      extraVolumeMounts:
        - name: MyComponent
          mountPath: /app/components
      extraVolumes:
        - name: MyComponent
          hostPath:
              path: /path/to/your/project/my_package/components
              type: DirectoryOrCreate

Edit:This code and the snippet provided in the other posts above do not result in the file being copied into the pod. I’ve solved this by using a custom Rasa image.