This depends on your configuration! If you are using components like:
DIETClassifier, ResponseSelector, … and policies like TEDPolicy you will get reproducible results only if you set random_seed to an int. Then models that were trained on the same data, configuration, rule, story, and have the same value for random_seed for a particular component or policy will behave identically.
Note that the random seed only needs to have the same value across a components or across a policy meaning, for example, the random seed for DIETClassifier could be 134 and the random seed for TEDPolicy could be 125.
You can check the possible parameters for a component or policy to see whether random seed can be set or not.