How to print NLU model summary

Rasa Open Source
1

Hello,

not exactly a RASA specific question but maybe someone can help me with this. I just upgraded to RASA 1.8 and try to print model summaries for core and nlu model.

First of all, my bot is working fine. I can ask questions and he respond correctly. I use the RASA CLI rasa train to train my core and nlu model.

I have no problems printing the core summary with.

from tensorflow import keras
model = keras.models.load_model('mypath/keras_model.h5')
model.summary()



Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
masking (Masking)            (None, 5, 110)            0         
_________________________________________________________________
lstm (LSTM)                  (None, 32)                18304     
_________________________________________________________________
dense (Dense)                (None, 59)                1947      
_________________________________________________________________
activation (Activation)      (None, 59)                0         
=================================================================
Total params: 20,251
Trainable params: 20,251
Non-trainable params: 0
_________________________________________________________________

I also want to print a model summary for the nlu model. Here is the content from the nlu model folder. image

I never had a deeper look into the models because everything was working fine, but now I want to go deeper into RASA and the models and try to understand the model architecture. I guess the component_4_DIETClassifier.tf_model.data-00000-of-00001 is my tensorflow model.

I tried several things but nothing worked, for example:

nlu_path='mypath/component_4_DIETClassifier.tf_model.data-00000-of-00001'
new_model = tf.keras.models.load_model(nlu_path)
new_model.summary()

I get the message: OSError: SavedModel file does not exist at: mypath/component_4_DIETClassifier.tf_model/{saved_model.pbtxt|saved_model.pb}

I also tried the path mypath/component_4_DIETClassifier.tf_model but this also fails.

Here is my config.yml

language: de_core_news_md 
pipeline:
  - name: SpacyNLP
  - name: SpacyTokenizer
  - name: SpacyFeaturizer
  - name: EntitySynonymMapper
  - name: LexicalSyntacticFeaturizer
    features: [
    ["low", "title", "upper"],
    [
      "BOS",
      "EOS",
      "low",
      "prefix5",
      "prefix2",
      "suffix5",
      "suffix3",
      "suffix2",
      "upper",
      "title",
      "digit",
    ],
    ["low", "title", "upper"],
    ]
  - name: DIETClassifier
    intent_classification: True
    entity_recognition: True
    use_masked_language_model: False
    number_of_transformer_layers: 0

policies:
  - name: KerasPolicy
     epochs: 50
    batch_size: 50
    max_training_samples: 300
  - name: FallbackPolicy 
    nlu_threshold: 0.4 
    core_threshold: 0.3 
  - name: MemoizationPolicy
  - name: MappingPolicy

Tried someone to have a deeper look into the NLU-model? Rasa tools like nlu evaluation is pretty cool but as I said I want to see more from the actual nlu model. Also I have no clue how the new DIETClassifier is working :frowning:

Am I on the right track at all?

Thank´s in advance.

1 Like
2

@lindig DIETClassifier uses the tensorflow saving format to store its model and hence it cannot be loaded with tf.keras.models.load_model. As of now, i would suggest going through this video - YouTube . There will be more videos in the series going forward to explain it in even more depth.

1 Like
3

@dakshvar22 thank you very much!