Hi all. there seems to be some problems with retrieval intents and the ResponseSelector responses in rasa open source 2.x
the problem can be reproduced using the responseselectorbot example provide in RasaHQ/rasa.
issue 1 - rasa 2.1.3 - Inconsistency between rasa documentation and rasa example in RasaHQ
Step-by-step Guide on Using Response Selector for FAQs and Chitchat indicates that ResponcesSelector responses should be added in domain.yml.
However, the responseselectorbot example rather place them outside of the domain file and more specifically in the nlu.yml.
Using the available responseselectorbot example (which defines responses in the nlu.yml), the command rasa data validate succeed but rasa train throws:
c:\programdata\anaconda3\envs\ihi\lib\site-packages\rasa\shared\utils\io.py:93: UserWarning: Action 'utter_chitchat' is listed as a response action in the domain
file, but there is no matching response defined. Please check your domain.
More info at https://rasa.com/docs/rasa/responses
However, if one put the responses in the domain.yml (as suggested by the documentation) the command rasa data validate fails with:
2020-12-18 11:17:37 INFO rasa.validator - Validating intents...
2020-12-18 11:17:37 INFO rasa.validator - Validating uniqueness of intents and stories...
2020-12-18 11:17:37 INFO rasa.validator - Validating utterances...
c:\programdata\anaconda3\envs\ihi\lib\site-packages\rasa\shared\utils\io.py:93: UserWarning: The action 'utter_chitchat' is used in the stories, but is not a vali
d utterance action. Please make sure the action is listed in your domain and there is a template defined with its name.
More info at https://rasa.com/docs/rasa/actions#utterance-actions
Project validation completed with errors.
Moreover, rasa train throws:
c:\programdata\anaconda3\envs\ihi\lib\site-packages\rasa\shared\utils\io.py:93: UserWarning: Action 'utter_chitchat' is listed as a response action in the domain
file, but there is no matching response defined. Please check your domain.
More info at https://rasa.com/docs/rasa/responses
This is obtained with:
* platform: win7-64
* rasa: 2.1.3
* rasa-sdk: 2.2.0
* rasa-x: 0.34.0
* tensorflow: 2.3.1
* cudatoolkit: 10.1.243
* python: 3.7.9
Several posts have reported related issues lately:
- Dec 3, 2020 Using an action instead of a normal utterance in ResponseSelector responses
- Dec 1, 2020 FAQ/Conversational hybrid bot
- Nov 23, 2020 Retrieval Intent responses not recognized when training a model through RasaX UI
- Nov 20, 2020 Retrieval Intents throws errors in rasa data validate
- Nov 4, 2020 Rasa 2 for FAQ - Button payload and intents problem
- Oct 30, 2020 Using Response Selector for FAQs don't return any Utterance
- Oct 24, 2020 Request for retrieval-based configuration(or github demo) for FAQ and chitchat in Rasa 2.0
issue 2 - rasa 2.2.1 - rasa data validate crashes with responses defined in nlu.yml
This obtained with:
* platform: win7-64
* rasa: 2.2.1
* rasa-sdk: 2.2.0
* rasa-x: 0.34.0
* tensorflow: 2.3.1
* cudatoolkit: 10.1.243
* python: 3.7.9
(ihi2) D:rasaHQ\rasa\examples\responseselectorbot>rasa data validate
2020-12-18 11:55:40.110817: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cudart64_101.dll
2020-12-18 11:55:44.278234: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library nvcuda.dll
2020-12-18 11:55:44.522258: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties:
pciBusID: 0000:02:00.0 name: GeForce GTX 1080 computeCapability: 6.1
coreClock: 1.7335GHz coreCount: 20 deviceMemorySize: 8.00GiB deviceMemoryBandwidth: 298.32GiB/s
2020-12-18 11:55:44.549261: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 1 with properties:
pciBusID: 0000:01:00.0 name: GeForce GTX 980 computeCapability: 5.2
coreClock: 1.2155GHz coreCount: 16 deviceMemorySize: 4.00GiB deviceMemoryBandwidth: 208.91GiB/s
2020-12-18 11:55:44.556262: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cudart64_101.dll
2020-12-18 11:55:44.567263: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cublas64_10.dll
2020-12-18 11:55:44.577264: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cufft64_10.dll
2020-12-18 11:55:44.583264: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library curand64_10.dll
2020-12-18 11:55:44.595266: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cusolver64_10.dll
2020-12-18 11:55:44.603266: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cusparse64_10.dll
2020-12-18 11:55:44.613267: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cudnn64_7.dll
2020-12-18 11:55:44.637270: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1858] Adding visible gpu devices: 0, 1
2020-12-18 11:55:45 INFO rasa.validator - Validating intents...
2020-12-18 11:55:45 INFO rasa.validator - Validating uniqueness of intents and stories...
2020-12-18 11:55:45 INFO rasa.validator - Validating utterances...
2020-12-18 11:55:45 INFO rasa.validator - Story structure validation...
Processed story blocks: 100%|āāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāā| 3/3 [00:00<00:00, 599.96it/s, # trackers=1]
2020-12-18 11:55:45 INFO rasa.core.training.story_conflict - Considering all preceding turns for conflict analysis.
Traceback (most recent call last):
File "c:\programdata\anaconda3\envs\ihi2\lib\runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "c:\programdata\anaconda3\envs\ihi2\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "C:\ProgramData\Anaconda3\envs\ihi2\Scripts\rasa.exe\__main__.py", line 7, in <module>
File "c:\programdata\anaconda3\envs\ihi2\lib\site-packages\rasa\__main__.py", line 116, in main
cmdline_arguments.func(cmdline_arguments)
File "c:\programdata\anaconda3\envs\ihi2\lib\site-packages\rasa\cli\data.py", line 262, in validate_files
and _validate_story_structure(validator, args)
File "c:\programdata\anaconda3\envs\ihi2\lib\site-packages\rasa\cli\data.py", line 297, in _validate_story_structure
not args.fail_on_warnings, max_history=args.max_history
File "c:\programdata\anaconda3\envs\ihi2\lib\site-packages\rasa\validator.py", line 249, in verify_story_structure
trackers, self.domain, max_history, self.nlu_config
File "c:\programdata\anaconda3\envs\ihi2\lib\site-packages\rasa\core\training\story_conflict.py", line 185, in find_story_conflicts
trackers, domain, max_history, tokenizer
File "c:\programdata\anaconda3\envs\ihi2\lib\site-packages\rasa\core\training\story_conflict.py", line 245, in _find_conflicting_states
for element in _sliced_states_iterator(trackers, domain, max_history, tokenizer):
File "c:\programdata\anaconda3\envs\ihi2\lib\site-packages\rasa\core\training\story_conflict.py", line 333, in _sliced_states_iterator
_apply_tokenizer_to_states(tokenizer, sliced_states)
File "c:\programdata\anaconda3\envs\ihi2\lib\site-packages\rasa\core\training\story_conflict.py", line 351, in _apply_tokenizer_to_states
Message({TEXT: state[USER][TEXT]}), TEXT
KeyError: 'text'
rasa train produces:
c:\programdata\anaconda3\envs\ihi2\lib\site-packages\rasa\shared\utils\io.py:93: UserWarning: Action 'utter_chitchat' is listed as a response action in th
is no matching response defined. Please check your domain.
More info at https://rasa.com/docs/rasa/responses
Thanks for considering