Started getting this issue recently now that my project’s gotten pretty complex, though I doubt that’s the issue.
I have a rule for saying goodbye when the user says goodbye. Pretty standard stuff.
version: "2.0"
rules:
- rule: say goodbye
steps:
- intent: goodbye
- action: utter_goodbye
And in the domain:
responses:
utter_goodbye:
- text: Have a nice day! We are looking forward to seeing you soon.
Now in most cases, whenever the user says goodbye, this rule is found and applied. But sometimes, after a very long conversation, I get a “There is no applicable rule” when running in debug mode, and TED’s prediction is applied, which as you can imagine, is pretty low confidence.
Debug logs:
2021-11-02 12:19:53 DEBUG rasa.core.lock_store - Issuing ticket for conversation 'dc5606eea45c482891fb996a3b885853'.
2021-11-02 12:19:53 DEBUG rasa.core.lock_store - Acquiring lock for conversation 'dc5606eea45c482891fb996a3b885853'.
2021-11-02 12:19:53 DEBUG rasa.core.lock_store - Acquired lock for conversation 'dc5606eea45c482891fb996a3b885853'.
2021-11-02 12:19:53 DEBUG rasa.core.tracker_store - Recreating tracker for id 'dc5606eea45c482891fb996a3b885853'
...
...
2021-11-02 12:19:53 DEBUG rasa.nlu.selectors.response_selector - Adding following selector key to message property: default
2021-11-02 12:19:53 DEBUG rasa.core.processor - Received user message 'bye' with intent '{'id': -699193297544572637, 'name': 'goodbye', 'confidence': 0.9923637509346008}' and entities '[]'
2021-11-02 12:19:53 DEBUG rasa.core.processor - Logged UserUtterance - tracker now has 157 events.
2021-11-02 12:19:53 DEBUG rasa.core.policies.memoization - Current tracker state:
...
... (27 events in the conversation)
2021-11-02 12:19:53 DEBUG rasa.core.policies.rule_policy - There is no applicable rule.
2021-11-02 12:19:53 DEBUG rasa.core.policies.rule_policy - Current tracker state:
...
...
2021-11-02 12:19:53 DEBUG rasa.core.policies.rule_policy - There is no applicable rule.
2021-11-02 12:19:53 DEBUG rasa.core.policies.ensemble - Made prediction using user intent.
2021-11-02 12:19:53 DEBUG rasa.core.policies.ensemble - Added `DefinePrevUserUtteredFeaturization(False)` event.
2021-11-02 12:19:53 DEBUG rasa.core.policies.ensemble - Predicted next action using policy_1_TEDPolicy.
2021-11-02 12:19:53 DEBUG rasa.core.processor - Predicted next action 'some_action' with confidence 0.56.
2021-11-02 12:19:53 DEBUG rasa.core.actions.action - Calling action endpoint to run action 'some_action'.
2021-11-02 12:19:53 DEBUG rasa.core.processor - Policy prediction ended with events '[<rasa.shared.core.events.DefinePrevUserUtteredFeaturization object at 0x7f49324e3990>]'.
2021-11-02 12:19:53 DEBUG rasa.core.processor - Action 'some_action' ended with events
...
And my config is:
language: en
pipeline:
- name: SpacyNLP
model: "en_core_web_md"
case_sensitive: False
- name: SpacyTokenizer
- name: RegexFeaturizer
case_sensitive: False
- name: SpacyFeaturizer
"pooling": "mean"
- name: CRFEntityExtractor
# BILOU_flag determines whether to use BILOU tagging or not.
"BILOU_flag": True
# features to extract in the sliding window
"features": [
["low", "title", "upper"],
[
"bias",
"low",
"prefix5",
"prefix2",
"suffix5",
"suffix3",
"suffix2",
"upper",
"title",
"digit",
"pattern",
],
["low", "title", "upper"],
]
# The maximum number of iterations for optimization algorithms.
"max_iterations": 100
# weight of the L1 regularization
"L1_c": 0.1
# weight of the L2 regularization
"L2_c": 0.1
# Name of dense featurizers to use.
# If list is empty all available dense features are used.
"featurizers": []
- name: DucklingEntityExtractor
# url of the running duckling server
url: "some_url"
# dimensions to extract
locale: "en_US"
dimensions: [some dimensions]
- name: EntitySynonymMapper
- name: DIETClassifier
entity_recognition: False
epochs: 700
tensorboard_log_directory: ""
- name: ResponseSelector
epochs: 700
- name: FallbackClassifier
threshold: 0.5
policies:
- name: MemoizationPolicy
epochs: 800
- name: TEDPolicy
max_history: 8
epochs: 800
tensorboard_log_directory: ""
- name: RulePolicy
core_fallback_threshold: 0.5
core_fallback_action_name: "action_default_fallback"
enable_fallback_prediction: True
As I mentioned, this rule works perfectly 90% of the time. Also, there are no rule story conflicts and the part of the conversation this rule represents is not mentioned in any stories. If I create a small experiment with this part of the conversation, the rule works properly. Only on including the entire project does it break.
There is one other rule for which this is happening, but a solution for this should fix that as well.
I can’t share the project but if you need some other information do let me know.
