Different intent prediction for the same model on different PCs

Rasa Open Source
1

Hi,

I trained a model on two different machines, but I am getting different predictions for the same input sentences. For example, in our case, “Literatur” would get predicted as “literaturinfo” in one PC but “reacttoinsult” in the other one. What can be wrong here?

Thank you.

2

I am not sure if it can give you an accurate result each time.

it is out of over an amount of tests, how many times did you get it right?

Performance is a percentage and if 8/10 times you have correct predictions then your model is performing well.

3

It’s actually not about the number of times of trials. I am getting the same intent prediction and confidence level on my PC each time I test the same input sentence.

The problem here is that the same model gives us two different behavior on two different PCs. In one PC it predicts the wrong intent but in the other machine intent prediction is correct.

4

Which pipeline did you use?

Maybe it is the kernel that can make such an issue or the version of tensorflow.

5

if you continue to have portability issues, i would suggest using docker since you can port your virtual OS with the models itself

6

On both computers I used spacy_sklearn as pipeline. Different versions of Rasa are installed on the two PC’s, Maybe this is causing the problem…?

7

It could be. can you sync to the same version

8

We are facing the same problem.

Everything is working on one machine, but then the same code fails in predicting some intents or actions on another machine?

Any update on this topic?

9

There was clearly a version mismatch. Same data can generate different models depending on kernel, Tensorflow, etc.