Core Pipeline Results Variation from RASA 1.* to RASA 2.*

Rasa Open Source
1

Hi

I am trying to migrate from RASA 1.* to RASA 2.8.8. I had the following Core Policies in place which were working just fine:

policies:
  - name: FormPolicy
  - name: FallbackPolicy
    nlu_threshold: 0.3
    ambiguity_threshold: 0.1
    core_threshold: 0.05
    fallback_action_name: ## fallback action name
  - name: MappingPolicy
  - name: MemoizationPolicy
    featurizer:
      - name: MaxHistoryTrackerFeaturizer
        max_history: 11
        state_featurizer:
          - name: BinarySingleStateFeaturizer
  - name: TEDPolicy
    batch_strategy: 'sequence'
    epochs: 250
    featurizer:
      - name: MaxHistoryTrackerFeaturizer
        max_history: 5
        state_featurizer:
          - name: BinarySingleStateFeaturizer

After migration, this is how the policies look like:

policies:
  - name: RulePolicy
    core_fallback_threshold: 0.3
    core_fallback_action_name: ## fallback action name
  - name: MemoizationPolicy
    max_history: 11
  - name: TEDPolicy
    split_entities_by_comma: False
    constrain_similarities: True
    batch_strategy: 'balanced'
    epochs: 250
    max_history: 11

My stories are no longer being evaluated correctly. Am I doing something wrong here? Also, I see a large increase in time of evaluation of the core pipeline. What could be the cause for that? Is this a known behavior and is something being done for that?

2

Could you expand on what you mean with “stories are no longer being evaluated correctly”. Do you have any statistics you can share?