I’m working on intent classification with rasa 1.8 and want to perform model selection. So basically, I need access to the training metrics emerging each epoch when training the NLU.
I know that rasa version 1.9 provides some option to get access to the training metrics via TensorBoard. But migration is not possible in my case.
As rasa 1.8 supports Tensorflow 2.0 there should still be a way to use tf.keras.callbacks.TensorBoard to trace loss or accuracy.
I have also seen these questions:
Training metrics access and Get training metrics which both seem similar but don’t really help me out. I’m kind of lost at how to incorporate TensorBoard when training the NLU.
Hi @sokofski, you cannot simply add a callback to TensorBoard as we have a custom fit method. If you want to log the training metrics, you need to modify rasa.utils.tensorflow.models. The training metrics are stored in a variable in this line: rasa/models.py at 012e6d04b2e8d9103da01186a5760fb99acdd5e8 · RasaHQ/rasa · GitHub (Rasa version 1.8). Feel free to fork Rasa and update the code here to store the metrics in a file. If you want to log the metrics so that they can be used by TensorBoard, you can take a look at the current version (rasa/models.py at master · RasaHQ/rasa · GitHub). Does that help?
