I have been running dozens of tests using rasa test but the results files are showing no outputs from the TED Policy. Other models’ results are fine. I don’t remember it working, even at the very beginning, using defaults for all initial rasa projects; just changing my domain and nlu and creating test conversations.
Why is this?
The confusion matrix shows that all entities are predicted as no_entity, TEDPolicy_confusion_matrix.png:
The TEDPolicy_errors.json file shows no predicted entities. The TEDPolicy_report.json file shows all scores are zero. I’m not even sure why TED is predicting entities, even when I set entity_recognition: false in the config.yml file. Why are there no actions being predicted? I thought that was it’s main purpose, not NER.
Hello all! I have the exact same issue with Rasa 3.0. AugmentedMemoizationPolicy works fine, but the TedPolicy never extract any entities. All the scores reported in TedPolicy_report.json are zero (no matter what changes I make). Any resolution on this? (Please see my config.yml below):
- name: WhitespaceTokenizer
- name: RegexEntityExtractor
- name: CountVectorsFeaturizer
- name: LanguageModelFeaturizer
- name: DIETClassifier
epochs: 200 #260 #450
#number_of_transformer_layers: 2 #2
#text: [256, 128]
batch_size: [64, 128]
learning_rate: 4e-4 #0.001 #8e-4 #4e-4
- name: EntitySynonymMapper
- name: ResponseSelector
# # No configuration for policies was provided. The following default policies were used to train your model.
# # If you'd like to customize them, uncomment and adjust the policies.
# # See https://rasa.com/docs/rasa/policies for more information.
- name: AugmentedMemoizationPolicy
- name: TEDPolicy
batch_size: [32, 64]
- name: UnexpecTEDIntentPolicy