NLU test for the two same config files give me different results

Hi There,

I have the exact same config files. config2.yml and config3.yml. And I run the following command: rasa test nlu --nlu data/nlu.yml --config config2.yml config3.yml

How come I get different results for the two different config files?

My config files: language: en


  • name: WhitespaceTokenizer
  • name: LexicalSyntacticFeaturizer
  • name: CountVectorsFeaturizer
  • name: CountVectorsFeaturizer analyzer: char_wb min_ngram: 1 max_ngram: 4
  • name: DIETClassifier epochs: 100 constrain_similarities: True model_confidence: linear_norm
  • name: EntitySynonymMapper


  • name: MemoizationPolicy max_history: 15
  • name: RulePolicy
  • name: TEDPolicy max_history: 15 epochs: 100 constrain_similarities: True model_confidence: linear_norm


It’s because randomness is used in machine learning.

If you really want to compare, you should set random_seed: 1 (or any number) under DIET, ResponseSelector, and TED. Example:

- name: DIETClassifier
  epochs: 100
  model_confidence: linear_norm
  constrain_similarities: true
  random_seed: 1
1 Like

Thank you for the info!

Happy to help!

Please test again and tell me if it works or not :slight_smile:

The two tests are now exactly the same! Thanks for the help once again!

1 Like

Glad to know! :slight_smile: Please mark my answer as solution to mark the thread as closed