Rasa data validate throws an error while using Response Selector in version 2.2.1

Rasa Open Source
1

Hi all. I have been experimenting with retrieval intents and response selectors and I am experiencing an issue with my own project as well as with the responseselectorbot example from rasaHQ\rasa\examples\responseselectorbot.

Error below occurs when using rasa data validate with rasa version 2.2.1. This problem is not occurring in version 2.1.3.

Despite this error rasa train seems to run but throws a warning:

UserWarning: Action 'utter_chitchat' is listed as a response action in the domain file, but there is no matching response defined. Please check your domain.

Thanks for your help, Gino

(rasa) C:\GitHub\rasaHQ\rasa\examples\responseselectorbot>rasa data validate
2020-12-17 22:32:15.010480: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cudart64_
101.dll
2020-12-17 22:32:18.508830: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library nvcuda.dl
l
2020-12-17 22:32:18.769856: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties:
pciBusID: 0000:02:00.0 name: GeForce GTX 1080 computeCapability: 6.1
coreClock: 1.7335GHz coreCount: 20 deviceMemorySize: 8.00GiB deviceMemoryBandwidth: 298.32GiB/s
2020-12-17 22:32:18.783858: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 1 with properties:
pciBusID: 0000:01:00.0 name: GeForce GTX 980 computeCapability: 5.2
coreClock: 1.2155GHz coreCount: 16 deviceMemorySize: 4.00GiB deviceMemoryBandwidth: 208.91GiB/s
2020-12-17 22:32:18.789858: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cudart64_
101.dll
2020-12-17 22:32:18.801859: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cublas64_
10.dll
2020-12-17 22:32:18.815861: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cufft64_1
0.dll
2020-12-17 22:32:18.823862: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library curand64_
10.dll
2020-12-17 22:32:18.838863: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cusolver6
4_10.dll
2020-12-17 22:32:18.845864: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cusparse6
4_10.dll
2020-12-17 22:32:18.855865: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cudnn64_7
.dll
2020-12-17 22:32:18.884868: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1858] Adding visible gpu devices: 0, 1
2020-12-17 22:32:19 INFO     rasa.validator  - Validating intents...
2020-12-17 22:32:19 INFO     rasa.validator  - Validating uniqueness of intents and stories...
2020-12-17 22:32:19 INFO     rasa.validator  - Validating utterances...
2020-12-17 22:32:19 INFO     rasa.validator  - Story structure validation...
Processed story blocks: 100%|██████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 749.88it/s, # trackers=1]
2020-12-17 22:32:19 INFO     rasa.core.training.story_conflict  - Considering all preceding turns for conflict analysis.
Traceback (most recent call last):
  File "c:\programdata\anaconda3\envs\rasa\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\programdata\anaconda3\envs\rasa\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\ProgramData\Anaconda3\envs\rasa\Scripts\rasa.exe\__main__.py", line 7, in <module>
  File "c:\programdata\anaconda3\envs\rasa\lib\site-packages\rasa\__main__.py", line 116, in main
    cmdline_arguments.func(cmdline_arguments)
  File "c:\programdata\anaconda3\envs\rasa\lib\site-packages\rasa\cli\data.py", line 262, in validate_files
    and _validate_story_structure(validator, args)
  File "c:\programdata\anaconda3\envs\rasa\lib\site-packages\rasa\cli\data.py", line 297, in _validate_story_structure
    not args.fail_on_warnings, max_history=args.max_history
  File "c:\programdata\anaconda3\envs\rasa\lib\site-packages\rasa\validator.py", line 249, in verify_story_structure
    trackers, self.domain, max_history, self.nlu_config
  File "c:\programdata\anaconda3\envs\rasa\lib\site-packages\rasa\core\training\story_conflict.py", line 185, in find_story_conflicts
    trackers, domain, max_history, tokenizer
  File "c:\programdata\anaconda3\envs\rasa\lib\site-packages\rasa\core\training\story_conflict.py", line 245, in _find_conflicting_states
    for element in _sliced_states_iterator(trackers, domain, max_history, tokenizer):
  File "c:\programdata\anaconda3\envs\rasa\lib\site-packages\rasa\core\training\story_conflict.py", line 333, in _sliced_states_iterator
    _apply_tokenizer_to_states(tokenizer, sliced_states)
  File "c:\programdata\anaconda3\envs\rasa\lib\site-packages\rasa\core\training\story_conflict.py", line 351, in _apply_tokenizer_to_stat
es
    Message({TEXT: state[USER][TEXT]}), TEXT
KeyError: 'text'

(rasa) D:\GitHub\rasaHQ\rasa\examples\responseselectorbot>conda list rasa*
# packages in environment at C:\ProgramData\Anaconda3\envs\rasa:
#
# Name                    Version                   Build  Channel
keras-preprocessing       1.1.2                    pypi_0    pypi
rasa                      2.2.1                    pypi_0    pypi
rasa-sdk                  2.2.0                    pypi_0    pypi
rasa-x                    0.34.0                   pypi_0    pypi
1 Like
2

May be this blog will help you.

3

I have followed the mentioned blog and Still facing a similar issue. Attaching the complete log for the same

Matplotlib created a temporary config/cache directory at /tmp/matplotlib-ddibi__k because the default path (/.config/matplotlib) is not a writable directory; it is highly recommended to set the MPLCONFIGDIR environment variable to a writable directory, in particular to speed up the import of Matplotlib and to better support multiprocessing.
2020-12-18 08:46:15.673330: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcudart.so.10.1'; dlerror: libcudart.so.10.1: cannot open shared object file: No such file or directory
2020-12-18 08:46:15.673360: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2020-12-18 08:46:17 DEBUG    rasa.telemetry  - Could not read telemetry settings from configuration file: Configuration 'metrics' key not found.
2020-12-18 08:46:17 WARNING  rasa.utils.common  - Failed to write global config. Error: [Errno 13] Permission denied: '/.config'. Skipping.
2020-12-18 08:46:17 DEBUG    rasa.shared.nlu.training_data.loading  - Training data format of 'data/nlu.yml' is 'rasa_yml'.
2020-12-18 08:46:17 DEBUG    rasa.shared.nlu.training_data.loading  - Training data format of 'data/responses.yml' is 'rasa_yml'.
2020-12-18 08:46:17 DEBUG    rasa.shared.nlu.training_data.loading  - Training data format of 'data/rules.yml' is 'unk'.
2020-12-18 08:46:17 DEBUG    rasa.shared.nlu.training_data.loading  - Training data format of 'data/stories.yml' is 'unk'.
2020-12-18 08:46:20 DEBUG    rasa.shared.nlu.training_data.loading  - Training data format of 'data/nlu.yml' is 'rasa_yml'.
2020-12-18 08:46:20 DEBUG    rasa.shared.nlu.training_data.loading  - Training data format of 'data/responses.yml' is 'rasa_yml'.
2020-12-18 08:46:21 INFO     rasa.validator  - Validating intents...
2020-12-18 08:46:21 DEBUG    rasa.validator  - The intent 'back' is listed in the domain file, but is not found in the NLU training data.
2020-12-18 08:46:21 DEBUG    rasa.validator  - The intent 'bot_challenge' is listed in the domain file, but is not found in the NLU training data.
2020-12-18 08:46:21 DEBUG    rasa.validator  - The intent 'nlu_fallback' is listed in the domain file, but is not found in the NLU training data.
2020-12-18 08:46:21 DEBUG    rasa.validator  - The intent 'out_of_scope' is listed in the domain file, but is not found in the NLU training data.
2020-12-18 08:46:21 DEBUG    rasa.validator  - The intent 'restart' is listed in the domain file, but is not found in the NLU training data.
2020-12-18 08:46:21 DEBUG    rasa.validator  - The intent 'session_start' is listed in the domain file, but is not found in the NLU training data.
2020-12-18 08:46:21 DEBUG    rasa.validator  - The intent 'back' is not used in any story.
2020-12-18 08:46:21 DEBUG    rasa.validator  - The intent 'out_of_scope' is not used in any story.
2020-12-18 08:46:21 DEBUG    rasa.validator  - The intent 'restart' is not used in any story.
2020-12-18 08:46:21 DEBUG    rasa.validator  - The intent 'session_start' is not used in any story.
2020-12-18 08:46:21 INFO     rasa.validator  - Validating uniqueness of intents and stories...
2020-12-18 08:46:21 INFO     rasa.validator  - Validating utterances...
2020-12-18 08:46:21 DEBUG    rasa.validator  - The utterance 'utter_default' is not used in any story.
2020-12-18 08:46:21 INFO     rasa.validator  - Story structure validation...
2020-12-18 08:46:21 DEBUG    rasa.shared.core.generator  - Number of augmentation rounds is 0
2020-12-18 08:46:21 DEBUG    rasa.shared.core.generator  - Starting data generation round 0 ... (with 1 trackers)
Processed story blocks: 100%|██████████| 168/168 [00:00<00:00, 900.54it/s, # trackers=1] 
2020-12-18 08:46:21 DEBUG    rasa.shared.core.generator  - Finished phase (159 training samples found).
2020-12-18 08:46:21 DEBUG    rasa.shared.core.generator  - Data generation rounds finished.
2020-12-18 08:46:21 DEBUG    rasa.shared.core.generator  - Found 0 unused checkpoints
2020-12-18 08:46:21 DEBUG    rasa.shared.core.generator  - Found 159 training trackers.
2020-12-18 08:46:21 INFO     rasa.core.training.story_conflict  - Considering all preceding turns for conflict analysis.
Traceback (most recent call last):
  File "/opt/venv/bin/rasa", line 8, in <module>
    sys.exit(main())
  File "/opt/venv/lib/python3.7/site-packages/rasa/__main__.py", line 116, in main
    cmdline_arguments.func(cmdline_arguments)
  File "/opt/venv/lib/python3.7/site-packages/rasa/cli/data.py", line 262, in validate_files
    and _validate_story_structure(validator, args)
  File "/opt/venv/lib/python3.7/site-packages/rasa/cli/data.py", line 297, in _validate_story_structure
    not args.fail_on_warnings, max_history=args.max_history
  File "/opt/venv/lib/python3.7/site-packages/rasa/validator.py", line 249, in verify_story_structure
    trackers, self.domain, max_history, self.nlu_config
  File "/opt/venv/lib/python3.7/site-packages/rasa/core/training/story_conflict.py", line 185, in find_story_conflicts
    trackers, domain, max_history, tokenizer
  File "/opt/venv/lib/python3.7/site-packages/rasa/core/training/story_conflict.py", line 245, in _find_conflicting_states
    for element in _sliced_states_iterator(trackers, domain, max_history, tokenizer):
  File "/opt/venv/lib/python3.7/site-packages/rasa/core/training/story_conflict.py", line 333, in _sliced_states_iterator
    _apply_tokenizer_to_states(tokenizer, sliced_states)
  File "/opt/venv/lib/python3.7/site-packages/rasa/core/training/story_conflict.py", line 351, in _apply_tokenizer_to_states
    Message({TEXT: state[USER][TEXT]}), TEXT
KeyError: 'text'
1 Like
4

Hello Uday. Thanks for you suggestions. I had previously read the blog you are suggesting; I believe however that there is an issue within the ‘retrieval intent’ / ResponseSelector / ‘rasa data validate’ since rasa 2.x.

There are have a number of post overs the last two months regarding the issues I am reporting but no real solution as far I could find. See for example:

Thank you for interest. Gino

3 Likes
5

The warning can be ignored and the issue is being tracked here.

If you move the responses out of the domain.yml to an nlu.yml, the warning should go away in rasa data validate