Hi All,
I have a rasa model trained on Linux Ubuntu OS. The model prediction for entity differs in Ubuntu & Windows OS.
Example : My utterance is “entity_1 entity_2 entity_3”. In Linux OS : all 3 entities are predicted as 1 entity. In Windows OS : all 3 entities are predicted as 3 entities ( expected response )
The model is trained on linux os. the same is used on Windows OS. The configuration, rasa version all remains same.
Any pointers on how should I troubleshoot this issue ?
Hi @siriusraja that’s really an unique use-case, ok can you share your rasa --version?
My rasa version is 1.10.10
Raja, it’s really an old version, so I am afraid I have no idea for this version but If you can migrate to 2.x (in 2.x I never got such an issue), this should also be the same in 1.x.
Please show me your config file
Don’t forget that Machine Learning uses randomness
Hi @ChrisRahme
Here is my config
# Configuration for Rasa NLU.
language: en
pipeline:
- name: "WhitespaceTokenizer"
case_sensitive: false
- name: "RegexFeaturizer"
- name: "LexicalSyntacticFeaturizer"
- name: CountVectorsFeaturizer
- name: "CountVectorsFeaturizer"
analyzer: "char_wb"
min_ngram: 1
max_ngram: 4
- name: "DIETClassifier"
batch_strategy: balanced
embedding_dimension: 50
random_seed: 42
epochs: 300
hidden_layers_sizes:
text: [256, 128]
number_of_transformer_layers: 4
weight_sparsity: 0
intent_classification: True
entity_recognition: True
use_masked_language_model: True
BILOU_flag: True
# evaluate_on_number_of_examples: 0
# evaluate_every_number_of_epochs: 10
# tensorboard_log_directory: "tensorboard"
# tensorboard_log_level: "epoch"
- name: "DucklingHTTPExtractor"
# url of the running duckling server
url: "<duckling url>"
# dimensions to extract
dimensions: ["time","email"]
# allows you to configure the locale, by default the language is
locale: "en_US"
# if not set the default timezone of Duckling is going to be used
# needed to calculate dates from relative expressions like "tomorrow"
# timezone: "Europe/Berlin"
# Timeout for receiving response from http url of the running duckling server
# if not set the default timeout of duckling http url is set to 3 seconds.
timeout : 3
- name: "EntitySynonymMapper"
- name: "ResponseSelector"
epochs: 100
# Configuration for Rasa Core.
policies:
- name: MemoizationPolicy
- name: TEDPolicy
max_history: 50
random_seed: 42
epochs: 300
- name: MappingPolicy
- name: FormPolicy
- name: FallbackPolicy
nlu_threshold: 0.3
core_threshold: 0.2
fallback_action_name: "action_custom_fallback"
ambiguity_threshold: 0.0
As you added a random_seed for DIET and TED, you can also add one for ResponseSelector.
Setting a random seed does not mean learning is not random anymore, but that the calculations that yield the random results are the same on each run (true randomness does not exist in computers).
But, even with a set random seed, Windows and Linux could use different “calculations”, since the core architecture is different. (No idea if that’s the case, but it could be an explanation if you get different results again 
)
Thank you @ChrisRahme .
@siriusraja Your EC2 is available to the public BTW. Visiting [REDACTED] yields “quack”… Possible security issue. Just a heads up
@kwehmeyer Thank you, I have removed the url from my post. Can you please remove the reference url and screenshot from your post.
I’ve had a similar experience where models behave differently on different OS setups. It can be super frustrating, right? I’ve found that sometimes it’s down to how paths are handled or even minor version differences in libraries. I wonder if there’s a resource like https://operatingsystems.info/ that lists known compatibility issues between OS and ML libraries. It could save a lot of headaches! What do you all think? Have you tried comparing results across different setups?