Hi team, I am facing issues with confidence score with RASA NLU 15.1 version. In my local I am having Python 3.7 and i have trained the model using Tensorflow embedding pipeline. after training I was getting good accuracy. With same training data I have trained the model in AWS instance with python version 3.6. the confidence scores are different with same utterance one is 0.82 and in Aws it is 0.62. the training accuracy is 0.99 with loss 0.21. please help me
these are statistical models, the numbers differ from training to training
if you want to get reproducible training, you need to fix all random seeds
I have made random_seed : 1. I mean constant. still i am facing the issue.
but setting random_seed doesn’t guarantee the same behavior on different machines, since effectively you use different random number generators
Thanks for the response. we are actually moving our model to Production environment. One more question. Can we expect 100% accuracy from Training data means. all the utterances in Training data should give expected response with mapped intent. what is optimal number for the accuracy to go for Production deployment. I am training with 5000 utterances with 300+ intents. Shall i expect 100% from the model prediction accuracy.
what accuracy you want to achieve is defined by you. It depends on your systems and the way you handle possible mistakes