Can't sync git repo with rasa X 0.35.0

[Deprecated] Rasa X Community Edition
1

I’m trying to synchronize my git repo to my rasa x server but an error appear in rasa_rasa-x_1 container :

WARNING:rasax.community.telemetry:An error occured when trying to send the telemetry event: HTTPSConnectionPool(host='api.segment.io', port=443): Max retries exceeded with url: /v1/track (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7fc1ca91fcd0>: Failed to establish a new connection: [Errno -2] Name or service not known'))
[2021-01-22 13:46:05 +0000] - (sanic.access)[INFO][172.20.0.11:38410]: POST http://my.domain/api/projects/default/git_repositories?api_token=f53a989dec987af5cef7348b3cffb752158dccf3  201 207
INFO:rasax.community.services.integrated_version_control.git_service:Cloning git repository from URL 'git@git.server.git/my_project.git'.
Job "run_background_synchronization (trigger: cron[minute='*'], next run at: 2021-01-22 13:47:00 UTC)" raised an exception
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/apscheduler/executors/base.py", line 125, in run_job
    retval = job.func(*job.args, **job.kwargs)
  File "/usr/local/lib/python3.8/dist-packages/rasax/community/services/integrated_version_control/git_service.py", line 131, in run_background_synchronization
    loop.run_until_complete(git_service.synchronize_project(force_data_injection))
  File "uvloop/loop.pyx", line 1456, in uvloop.loop.Loop.run_until_complete
  File "/usr/local/lib/python3.8/dist-packages/rasax/community/services/integrated_version_control/git_service.py", line 1102, in synchronize_project
    await self._inject_data()
  File "/usr/local/lib/python3.8/dist-packages/rasax/community/services/integrated_version_control/git_service.py", line 1146, in _inject_data
    await rasax.community.initialise.inject_files_from_disk(
  File "/usr/local/lib/python3.8/dist-packages/rasax/community/initialise.py", line 350, in inject_files_from_disk
    inject_domain(
  File "/usr/local/lib/python3.8/dist-packages/rasax/community/initialise.py", line 185, in inject_domain
    if not rasa.shared.core.domain.Domain.is_domain_file(
  File "/usr/local/lib/python3.8/dist-packages/rasa/shared/core/domain.py", line 1582, in is_domain_file
    content = rasa.shared.utils.io.read_yaml_file(filename)
  File "/usr/local/lib/python3.8/dist-packages/rasa/shared/utils/io.py", line 368, in read_yaml_file
    return read_yaml(read_file(filename, DEFAULT_ENCODING))
  File "/usr/local/lib/python3.8/dist-packages/rasa/shared/utils/io.py", line 349, in read_yaml
    return yaml_parser.load(content) or {}
  File "/usr/local/lib/python3.8/dist-packages/ruamel/yaml/main.py", line 343, in load
    return constructor.get_single_data()
  File "/usr/local/lib/python3.8/dist-packages/ruamel/yaml/constructor.py", line 113, in get_single_data
    return self.construct_document(node)
  File "/usr/local/lib/python3.8/dist-packages/ruamel/yaml/constructor.py", line 123, in construct_document
    for _dummy in generator:
  File "/usr/local/lib/python3.8/dist-packages/ruamel/yaml/constructor.py", line 723, in construct_yaml_map
    value = self.construct_mapping(node)
  File "/usr/local/lib/python3.8/dist-packages/ruamel/yaml/constructor.py", line 440, in construct_mapping
    return BaseConstructor.construct_mapping(self, node, deep=deep)
  File "/usr/local/lib/python3.8/dist-packages/ruamel/yaml/constructor.py", line 255, in construct_mapping
    value = self.construct_object(value_node, deep=deep)
  File "/usr/local/lib/python3.8/dist-packages/ruamel/yaml/constructor.py", line 146, in construct_object
    data = self.construct_non_recursive_object(node)
  File "/usr/local/lib/python3.8/dist-packages/ruamel/yaml/constructor.py", line 181, in construct_non_recursive_object
    data = constructor(self, node)
  File "/usr/local/lib/python3.8/dist-packages/rasa/shared/utils/io.py", line 311, in env_var_constructor
    raise ValueError(
ValueError: Error when trying to expand the environment variables in 'echo "${USER} $(date)" > {{ RASA_HOME }}/terms/agree.txt'. Please make sure to also set these environment variables: '['"${USER}', '$(date)"']'.

And there is no problem during training model with docker :

docker run -u 0 -it -v /my_project:/app rasa/rasa:2.2.5 train

The generated model can be used with rasa shell :

docker run -u 0 -it -p 5005:5005 -v /my_project:/app rasa/rasa:2.2.5 shell

I tried rasa data validate but no error or any conflict stories :

docker run -u 0 -it -p 5005:5005 -v /my_project:/app rasa/rasa:2.2.5 data validate

And no problem is find. This is the result :

2021-02-02 11:01:12.571577: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcudart.so.10.1'; dlerror: libcudart.so.10.1: cannot open shared object file: No such file or directory
2021-02-02 11:01:12.571620: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk
x Rasa Open Source reports anonymous usage telemetry to help improve the product x
x for all its users.                                                             x
x                                                                                x
x If you'd like to opt-out, you can use `rasa telemetry disable`.                x
x To learn more, check out https://rasa.com/docs/rasa/telemetry/telemetry.       x
mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj
2021-02-02 11:01:13.999572: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory
2021-02-02 11:01:13.999614: W tensorflow/stream_executor/cuda/cuda_driver.cc:312] failed call to cuInit: UNKNOWN ERROR (303)
2021-02-02 11:01:13.999646: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (0248a65be535): /proc/driver/nvidia/version does not exist
The configuration for pipeline was chosen automatically. It was written into the config file at 'config.yml'.
2021-02-02 11:01:14 INFO     rasa.shared.utils.validation  - The 'version' key is missing in the training data file /app/domain.yml. Rasa Open Source will read the file as a version '2.0' file. See https://rasa.com/docs/rasa/training-data-format.
2021-02-02 11:01:14 INFO     rasa.shared.utils.validation  - The 'version' key is missing in the training data file /app/data/rules.yml. Rasa Open Source will read the file as a version '2.0' file. See https://rasa.com/docs/rasa/training-data-format.
2021-02-02 11:01:15 INFO     rasa.shared.utils.validation  - The 'version' key is missing in the training data file /app/data/nlu.yml. Rasa Open Source will read the file as a version '2.0' file. See https://rasa.com/docs/rasa/training-data-format.
2021-02-02 11:01:15 INFO     rasa.validator  - Validating intents...
2021-02-02 11:01:15 INFO     rasa.validator  - Validating uniqueness of intents and stories...
2021-02-02 11:01:15 INFO     rasa.validator  - Validating utterances...
2021-02-02 11:01:15 INFO     rasa.validator  - Story structure validation...
Processed story blocks: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 154/154 [00:00<00:00, 1983.18it/s, # trackers=1]
2021-02-02 11:01:15 INFO     rasa.core.training.story_conflict  - Considering all preceding turns for conflict analysis.
2021-02-02 11:01:15 INFO     rasa.validator  - No story structure conflicts found.

So I really don’t understand why I can’t synchronise my rasa X server to my git repo because everything is ok except the ValueError: Error when trying to expand the environment variables in 'echo "${USER} $(date)" > {{ RASA_HOME }}/terms/agree.txt'

If someone have an idea !

Rasa Open source version : 2.2.5 Rasa X version : 0.35.0 Rasa SDK : 2.2.0

2

This is a known issue. Rasa X 0.35.1 will be released later today or tomorrow along with Rasa OSS 2.2.10 to address it.

3

I ran into a similar issue that the github repository would not sync. Not sure if it is the same though.

This is how i troubleshoot it:

  1. I first created a new project using rasa init
  2. I put all these files into the repository
  3. I noticed that Rasa X now synced the repository

Just like you, I had on my local machine all stories etc running. Everything valided perfectly.

Then I wen’t into debug mode and copied stuff line by line. I still don’t fully understand where I made the mistake, but somewhere there must have been an error. The strange thing is that locally everything is working perfectly.

I noticed in the log that rasa is first doing some parsing before it copies over the files. I saw in the log that sometimes errors were produced that did not occur on the local machine.

Again, maybe this is not your issue. But for sure try out with a clean rasa project.

4

Thks for answer,

I will try again with Rasa X 0.35.1 and update this topic.

5

The git sync with rasa X 0.35.1 is now ok. But I have a new problem with Stories sync. Only 20 first stories of my stories.yml avec loaded in rasa X 0.35.1.
Howewer, no errors noted in containers. Maybe I will open new ticket for this problem.

6

Oh I didn’t see logs in rasa_rasa-x_1 container, there is userWarning that said :

/usr/local/lib/python3.8/dist-packages/rasa/shared/utils/io.py:93: UserWarning: Issue found in '/tmp/tmpwbxzk24b':
Found intent 'affirm' in stories which is not part of the domain.
  More info at https://rasa.com/docs/rasa/stories

This userWarning appear for all intents. I don’t understand this userWarning because the intents are declared in domain.yml and used in stories without problem.

7

Hi Clem,

You should run a rasa data validate on your project from the command line before syncing with Rasa X. This will show you issues like the affirm issue you are seeing.

Greg

8

@stephens,

This is the return of the command docker run -it -u 0 -p 5005:5005 -v /root/chatbot:/app rasa/rasa:latest-full data validate

2021-02-05 08:10:24.699400: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcudart.so.10.1'; dlerror: libcudart.so.10.1: cannot open shared object file: No such file or directory
2021-02-05 08:10:24.699450: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk
x Rasa Open Source reports anonymous usage telemetry to help improve the product x
x for all its users.                                                             x
x                                                                                x
x If you'd like to opt-out, you can use `rasa telemetry disable`.                x
x To learn more, check out https://rasa.com/docs/rasa/telemetry/telemetry.       x
mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj
The configuration for pipeline was chosen automatically. It was written into the config file at 'config.yml'.
2021-02-05 08:10:26 INFO     rasa.shared.utils.validation  - The 'version' key is missing in the training data file /app/domain.yml. Rasa Open Source will read the file as a version '2.0' file. See https://rasa.com/docs/rasa/training-data-format.
2021-02-05 08:10:26 INFO     rasa.shared.utils.validation  - The 'version' key is missing in the training data file /app/data/rules.yml. Rasa Open Source will read the file as a version '2.0' file. See https://rasa.com/docs/rasa/training-data-format.
2021-02-05 08:10:28 INFO     rasa.shared.utils.validation  - The 'version' key is missing in the training data file /app/data/nlu.yml. Rasa Open Source will read the file as a version '2.0' file. See https://rasa.com/docs/rasa/training-data-format.
2021-02-05 08:10:28 INFO     rasa.validator  - Story structure validation...
Processed story blocks: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 154/154 [00:00<00:00, 1495.39it/s, # trackers=1]
2021-02-05 08:10:28 INFO     rasa.core.training.story_conflict  - Considering all preceding turns for conflict analysis.
Traceback (most recent call last):
  File "/opt/venv/bin/rasa", line 8, in <module>
    sys.exit(main())
  File "/opt/venv/lib/python3.7/site-packages/rasa/__main__.py", line 116, in main
    cmdline_arguments.func(cmdline_arguments)
  File "/opt/venv/lib/python3.7/site-packages/rasa/cli/data.py", line 278, in validate_stories
    validate_files(args, stories_only=True)
  File "/opt/venv/lib/python3.7/site-packages/rasa/cli/data.py", line 257, in validate_files
    all_good = _validate_story_structure(validator, args)
  File "/opt/venv/lib/python3.7/site-packages/rasa/cli/data.py", line 297, in _validate_story_structure
    not args.fail_on_warnings, max_history=args.max_history
  File "/opt/venv/lib/python3.7/site-packages/rasa/validator.py", line 249, in verify_story_structure
    trackers, self.domain, max_history, self.nlu_config
  File "/opt/venv/lib/python3.7/site-packages/rasa/core/training/story_conflict.py", line 185, in find_story_conflicts
    trackers, domain, max_history, tokenizer
  File "/opt/venv/lib/python3.7/site-packages/rasa/core/training/story_conflict.py", line 245, in _find_conflicting_states
    for element in _sliced_states_iterator(trackers, domain, max_history, tokenizer):
  File "/opt/venv/lib/python3.7/site-packages/rasa/core/training/story_conflict.py", line 333, in _sliced_states_iterator
    _apply_tokenizer_to_states(tokenizer, sliced_states)
  File "/opt/venv/lib/python3.7/site-packages/rasa/core/training/story_conflict.py", line 351, in _apply_tokenizer_to_states
    Message({TEXT: state[USER][TEXT]}), TEXT
KeyError: 'text'

I tried to comment all stories and let this one :

version: "2.0"
stories:
- story: out_of_scope
  steps:
  - intent: out_of_scope
  - action: utter_cannot_help

And error is always here with command rasa data validate.

This is my config.yml file :

language: en
pipeline:
policies:
  - name: MemoizationPolicy
  - name: TEDPolicy
    max_history: 5
    epochs: 100
  - name: RulePolicy

I really don’t understand where is the story conflict…

9

I think this is a YAML type error (not a story conflict).

Are you using the response selector? I think you may be running into the issue discussed here.

10

I just ran that and the validation found no conflicts, yet I keep getting it with Rasa-X.

2021-02-19 12:20:58 INFO     rasa.validator  - Validating intents...
2021-02-19 12:20:58 INFO     rasa.validator  - Validating uniqueness of intents and stories...
2021-02-19 12:20:58 INFO     rasa.validator  - Validating utterances...
2021-02-19 12:20:58 INFO     rasa.validator  - Story structure validation...
Processed story blocks: 100%|█████████████████| 14/14 [00:00<00:00, 495.36it/s, # trackers=1]
2021-02-19 12:20:58 INFO     rasa.core.training.story_conflict  - Considering all preceding turns for conflict analysis.
2021-02-19 12:20:58 INFO     rasa.validator  - No story structure conflicts found.

I have my domain.yml in a different folder but I specify this using the --domain flag whenever loading rasa, rasa-x, or the data validation command.

11

Have you tried 0.36.0? Are you running Rasa X locally or via docker container? Can you provide the full command line used to start Rasa X?

12

For this particular error, I am using rasa-x 0.36.0 and `rasa 2.2.10. I am running it locally without any container or venv (I’ve got it installed globally). The command I use is this:

rasa x --endpoint config/endpoints.yml --domain data/domain.yml --data data/training -m models --config config/config.yml --enable-api --cors "*" -p 2004

As you can notice, I have a slightly different directory structure which is why I point out exactly where my data, config, domain, and endpoints are.

13

@snek It sounds like you’re dealing with an issue un-related to this topic (git repo sync issue). I don’t see any error in your original post.

14

You’re right, I was referring to this error here since I was getting the exact same one even though all intents are declared in the domain as well. I probably picked a bad place to point it out. :smiley:

1 Like