endreb
(endre)
November 3, 2021, 2:28pm
1
Hi There,
I have the exact same config files. config2.yml and config3.yml. And I run the following command:
rasa test nlu --nlu data/nlu.yml --config config2.yml config3.yml
How come I get different results for the two different config files?
My config files:
language: en
pipeline:
name: WhitespaceTokenizer
name: LexicalSyntacticFeaturizer
name: CountVectorsFeaturizer
name: CountVectorsFeaturizer
analyzer: char_wb
min_ngram: 1
max_ngram: 4
name: DIETClassifier
epochs: 100
constrain_similarities: True
model_confidence: linear_norm
name: EntitySynonymMapper
policies:
name: MemoizationPolicy
max_history: 15
name: RulePolicy
name: TEDPolicy
max_history: 15
epochs: 100
constrain_similarities: True
model_confidence: linear_norm
Endre
ChrisRahme
(Chris Rahmé)
November 4, 2021, 10:25am
2
It’s because randomness is used in machine learning.
If you really want to compare, you should set random_seed: 1
(or any number) under DIET, ResponseSelector, and TED. Example:
- name: DIETClassifier
epochs: 100
model_confidence: linear_norm
constrain_similarities: true
random_seed: 1
1 Like
ChrisRahme
(Chris Rahmé)
November 4, 2021, 11:13am
4
Happy to help!
Please test again and tell me if it works or not
endreb
(endre)
November 4, 2021, 12:12pm
5
ChrisRahme:
random_seed: 1
The two tests are now exactly the same! Thanks for the help once again!
1 Like
ChrisRahme
(Chris Rahmé)
November 4, 2021, 12:56pm
6
Glad to know! Please mark my answer as solution to mark the thread as closed