Rasa train tensorboard error

File “c:\programdata\miniconda3\envs\nestle-chatbot\lib\site-packages\tensorflow\python\keras\engine\training.py”, line 1374, in evaluate tmp_logs = test_function(iterator) text_transformed, text_in, text_seq_ids, mlm_mask_boolean_text, TEXT c:\programdata\miniconda3\envs\nestle-chatbot\lib\site-packages\rasa\nlu\classifiers\diet_classifier.py:1477 _mask_loss tf.reduce_any(mlm_mask_boolean), c:\programdata\miniconda3\envs\nestle-chatbot\lib\site-packages\tensorflow\python\util\dispatch.py:201 wrapper return target(*args, **kwargs) c:\programdata\miniconda3\envs\nestle-chatbot\lib\site-packages\tensorflow\python\ops\math_ops.py:2896 reduce_any name=name)) c:\programdata\miniconda3\envs\nestle-chatbot\lib\site-packages\tensorflow\python\ops\gen_math_ops.py:721 _any name=name) c:\programdata\miniconda3\envs\nestle-chatbot\lib\site-packages\tensorflow\python\framework\op_def_library.py:493 _apply_op_helper (prefix, dtypes.as_dtype(input_arg.type).name))

TypeError: Input 'input' of 'Any' Op has type float32 that does not match expected type of bool.
  • name: DIETClassifier entity_recognition: true batch_size: [64, 256] epochs: 100 drop_rate: 0.2 random_seed: 42 number_of_transformer_layers: 4 use_masked_language_model: True constrain_similarities: True tensorboard_log_directory: train_logs

checkpoint_model: True

evaluate_every_number_of_epochs: 2 evaluate_on_number_of_examples: 300

@wireless911 Hi :wave: and welcome to our community!

Can you please share the rasa --version and are you using Miniconda3?

I installed rasa using pip even though I used conda

@wireless911 please share the screenshot of error? @wireless911 please even share config.yml file

I modified my batchsize, now it’s okay, sometimes it will appear

@wireless911 ok great! please close this thread.

@nik202 I also have this error. It appears that it happens where use_masked_language_model=True. I get the error at the first evaluation before anything is written to tensorboard. If I remove the masked_language_model then itworks.

Rasa Version : 2.8.11 Minimum Compatible Version: 2.8.9 Rasa SDK Version : 2.8.3 Rasa X Version : 0.42.5 Python Version : 3.8.8 Operating System : Linux-5.10.60.1-microsoft-standard-WSL2-x86_64-with-glibc2.10 Python Path : /home/simon/anaconda3/bin/python

language: en
pipeline:
- name: enzopreprocessor.EnzoPreprocessor

# language model
- name: WhitespaceTokenizer
- name: LanguageModelFeaturizer
  model_weights: distilbert-base-uncased
  model_name: distilbert

# Regex for phone numbers
- name: RegexFeaturizer

# dual intentity and entity
- name: DIETClassifier
  random_seed: 42
  batch_size: 16
  intent_classification: True
  entity_recognition: True
  use_masked_language_model: True
  constrain_similarities: True
  epochs: 200
  evaluate_on_number_of_examples: 200
  evaluate_every_number_of_epochs: 1
  tensorboard_log_directory: "./tbdiet"
  tensorboard_log_level: "batch"

############ ERROR MESSAGE Traceback (most recent call last): File “/home/simon/anaconda3/bin/rasa”, line 8, in sys.exit(main()) File “/home/simon/anaconda3/lib/python3.8/site-packages/rasa/main.py”, line 118, in main cmdline_arguments.func(cmdline_arguments) File “/home/simon/anaconda3/lib/python3.8/site-packages/rasa/cli/train.py”, line 59, in train_parser.set_defaults(func=lambda args: run_training(args, can_exit=True)) File “/home/simon/anaconda3/lib/python3.8/site-packages/rasa/cli/train.py”, line 91, in run_training training_result = train_all( File “/home/simon/anaconda3/lib/python3.8/site-packages/rasa/api.py”, line 109, in train return rasa.utils.common.run_in_loop( File “/home/simon/anaconda3/lib/python3.8/site-packages/rasa/utils/common.py”, line 296, in run_in_loop result = loop.run_until_complete(f) File “uvloop/loop.pyx”, line 1456, in uvloop.loop.Loop.run_until_complete File “/home/simon/anaconda3/lib/python3.8/site-packages/rasa/model_training.py”, line 108, in train_async return await _train_async_internal( File “/home/simon/anaconda3/lib/python3.8/site-packages/rasa/model_training.py”, line 288, in _train_async_internal await _do_training( File “/home/simon/anaconda3/lib/python3.8/site-packages/rasa/model_training.py”, line 334, in _do_training model_path = await _train_nlu_with_validated_data( File “/home/simon/anaconda3/lib/python3.8/site-packages/rasa/model_training.py”, line 758, in _train_nlu_with_validated_data await rasa.nlu.train.train( File “/home/simon/anaconda3/lib/python3.8/site-packages/rasa/nlu/train.py”, line 111, in train interpreter = trainer.train(training_data, **kwargs) File “/home/simon/anaconda3/lib/python3.8/site-packages/rasa/nlu/model.py”, line 221, in train component.train(working_data, self.config, **context) File “/home/simon/anaconda3/lib/python3.8/site-packages/rasa/nlu/classifiers/diet_classifier.py”, line 880, in train self.model.fit( File “/home/simon/anaconda3/lib/python3.8/site-packages/rasa/utils/tensorflow/temp_keras_modules.py”, line 217, in fit val_logs = self.evaluate( File “/home/simon/.local/lib/python3.8/site-packages/keras/engine/training.py”, line 1501, in evaluate tmp_logs = self.test_function(iterator) File “/home/simon/anaconda3/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py”, line 885, in call result = self._call(*args, **kwds) File “/home/simon/anaconda3/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py”, line 933, in _call self._initialize(args, kwds, add_initializers_to=initializers) File “/home/simon/anaconda3/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py”, line 759, in _initialize self._stateful_fn._get_concrete_function_internal_garbage_collected( # pylint: disable=protected-access File “/home/simon/anaconda3/lib/python3.8/site-packages/tensorflow/python/eager/function.py”, line 3066, in _get_concrete_function_internal_garbage_collected graph_function, _ = self._maybe_define_function(args, kwargs) File “/home/simon/anaconda3/lib/python3.8/site-packages/tensorflow/python/eager/function.py”, line 3463, in _maybe_define_function graph_function = self._create_graph_function(args, kwargs) File “/home/simon/anaconda3/lib/python3.8/site-packages/tensorflow/python/eager/function.py”, line 3298, in _create_graph_function func_graph_module.func_graph_from_py_func( File “/home/simon/anaconda3/lib/python3.8/site-packages/tensorflow/python/framework/func_graph.py”, line 1007, in func_graph_from_py_func func_outputs = python_func(*func_args, **func_kwargs) File “/home/simon/anaconda3/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py”, line 668, in wrapped_fn out = weak_wrapped_fn().wrapped(*args, **kwds) File “/home/simon/anaconda3/lib/python3.8/site-packages/tensorflow/python/framework/func_graph.py”, line 994, in wrapper raise e.ag_error_metadata.to_exception(e) TypeError: in user code:

/home/simon/.local/lib/python3.8/site-packages/keras/engine/training.py:1330 test_function  *
    return step_function(self, iterator)
/home/simon/.local/lib/python3.8/site-packages/keras/engine/training.py:1320 step_function  **
    outputs = model.distribute_strategy.run(run_step, args=(data,))
/home/simon/anaconda3/lib/python3.8/site-packages/tensorflow/python/distribute/distribute_lib.py:1286 run
    return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
/home/simon/anaconda3/lib/python3.8/site-packages/tensorflow/python/distribute/distribute_lib.py:2849 call_for_each_replica
    return self._call_for_each_replica(fn, args, kwargs)
/home/simon/anaconda3/lib/python3.8/site-packages/tensorflow/python/distribute/distribute_lib.py:3632 _call_for_each_replica
    return fn(*args, **kwargs)
/home/simon/.local/lib/python3.8/site-packages/keras/engine/training.py:1313 run_step  **
    outputs = model.test_step(data)
/home/simon/anaconda3/lib/python3.8/site-packages/rasa/utils/tensorflow/models.py:200 test_step        prediction_loss = self.batch_loss(batch_in)
/home/simon/anaconda3/lib/python3.8/site-packages/rasa/nlu/classifiers/diet_classifier.py:1567 batch_loss
    loss, acc = self._mask_loss(
/home/simon/anaconda3/lib/python3.8/site-packages/rasa/nlu/classifiers/diet_classifier.py:1477 _mask_loss
    tf.reduce_any(mlm_mask_boolean),
/home/simon/anaconda3/lib/python3.8/site-packages/tensorflow/python/util/dispatch.py:206 wrapper
    return target(*args, **kwargs)
/home/simon/anaconda3/lib/python3.8/site-packages/tensorflow/python/ops/math_ops.py:3267 reduce_any
    gen_math_ops._any(
/home/simon/anaconda3/lib/python3.8/site-packages/tensorflow/python/ops/gen_math_ops.py:685 _any
    _, _, _op, _outputs = _op_def_library._apply_op_helper(
/home/simon/anaconda3/lib/python3.8/site-packages/tensorflow/python/framework/op_def_library.py:544 _apply_op_helper
    raise TypeError("%s expected type of %s." %

TypeError: Input 'input' of 'Any' Op has type float32 that does not match expected type of bool.
1 Like

@simonm3 Thanks Simon for the feedback. What is your current TensorFlow version for the above rasa configuration?

2.6.2

@nik202 I have a language model featurizer earlier in the pipeline. Does setting use_masked_language_model=False make sense because it prevents an additional DIET masked model? Or does it disable the earlier masked language model featurizer?

Thank you for sharing this. I tried training my model with checkpoints and the use_masked_language_model flag set to true. This resulted in the same error as above. Fixed by setting to False