Intent MissClassification on Rasa Upgrade

Hi Team,

We have currently migrated to Rasa 1.3.9 from Rasa 0.13.8. It has a total of 136 intents, nearly 90 small talk intents. It seems to work fine before migration, and the confidence of prediction was always above 75 percentage.

But after rasa upgrade the confidence of prediction of intents went low, mainly for the small talk intents and for inform. We tried possible hyper parameter optimization. But was not able to replicate the previous results. Can you please help.

Please find the config used.

language: "en"

pipeline:

- name: "WhitespaceTokenizer"
- name: "RegexFeaturizer"
- name: "CRFEntityExtractor"
- name: "EntitySynonymMapper"
- name: "CountVectorsFeaturizer"
- name: "EmbeddingIntentClassifier"


# Configuration for Rasa Core.
# https://rasa.com/docs/rasa/core/policies/
policies:
  - name: KerasPolicy
    nlu_threshold: 0.6
    core_threshold: 0.6
    epochs: 300
    max_history: 3
  - name: MemoizationPolicy
    max_history: 3
  - name: FormPolicy
  - name: FallbackPolicy
    nlu_threshold: 0.70
    core_threshold: 0.75
    fallback_action_name: 'action_fallback'

Regards, Surya