When I am trying to run this:
python -m rasa_nlu.evaluate --config conf/experiments/nlu4.yml --model /tmp/bert_exp/order_tracking/nlu/ --data data/nlu_test/
with the innatis bert classifier configured in nlu4.yml, I get the following:
/Users/yan/environments/rasa/lib/python3.6/site-packages/rasa_nlu/training_data/training_data.py:191: UserWarning: Intent ‘asking_refund+damaged_order’ has only 1 training examples! Minimum is 2, training may fail.
self.MIN_EXAMPLES_PER_INTENT))
/Users/yan/environments/rasa/lib/python3.6/site-packages/rasa_nlu/training_data/training_data.py:191: UserWarning: Intent ‘cancel_order+update_order’ has only 1 training examples! Minimum is 2, training may fail.
self.MIN_EXAMPLES_PER_INTENT))
/Users/yan/environments/rasa/lib/python3.6/site-packages/rasa_nlu/training_data/training_data.py:191: UserWarning: Intent ‘delayed_order+cancel_order’ has only 1 training examples! Minimum is 2, training may fail.
self.MIN_EXAMPLES_PER_INTENT))
/Users/yan/environments/rasa/lib/python3.6/site-packages/rasa_nlu/training_data/training_data.py:191: UserWarning: Intent ‘general_qa+get_order_info’ has only 1 training examples! Minimum is 2, training may fail.
self.MIN_EXAMPLES_PER_INTENT))
/Users/yan/environments/rasa/lib/python3.6/site-packages/rasa_nlu/training_data/training_data.py:191: UserWarning: Intent ‘get_order_info+cancel_order’ has only 1 training examples! Minimum is 2, training may fail.
self.MIN_EXAMPLES_PER_INTENT))
Traceback (most recent call last):
File “/Users/yan/.pyenv/versions/3.6.6/lib/python3.6/runpy.py”, line 193, in _run_module_as_main
“main”, mod_spec)
File “/Users/yan/.pyenv/versions/3.6.6/lib/python3.6/runpy.py”, line 85, in _run_code
exec(code, run_globals)
File “/Users/yan/environments/rasa/lib/python3.6/site-packages/rasa_nlu/evaluate.py”, line 978, in
main()
File “/Users/yan/environments/rasa/lib/python3.6/site-packages/rasa_nlu/evaluate.py”, line 972, in main
cmdline_args.histogram)
File “/Users/yan/environments/rasa/lib/python3.6/site-packages/rasa_nlu/evaluate.py”, line 732, in run_evaluation
test_data)
File “/Users/yan/environments/rasa/lib/python3.6/site-packages/rasa_nlu/evaluate.py”, line 636, in get_entity_predictions
res = interpreter.parse(e.text, only_output_properties=False)
File “/Users/yan/environments/rasa/lib/python3.6/site-packages/rasa_nlu/model.py”, line 357, in parse
component.process(message, **self.context)
File “/Users/yan/environments/rasa/lib/python3.6/site-packages/innatis/classifiers/bert_intent_classifier.py”, line 170, in process
index = tf.argmax(probabilities, axis=0).eval()
File “/Users/yan/environments/rasa/lib/python3.6/site-packages/tensorflow/python/framework/ops.py”, line 713, in eval
return _eval_using_default_session(self, feed_dict, self.graph, session)
File “/Users/yan/environments/rasa/lib/python3.6/site-packages/tensorflow/python/framework/ops.py”, line 5148, in _eval_using_default_session
raise ValueError("Cannot use the default session to evaluate tensor: "
ValueError: Cannot use the default session to evaluate tensor: the tensor’s graph is different from the session’s graph. Pass an explicit session to eval(session=sess).
Seems like some incorrect way of using the TF session, default vs. non default, but I am not really sure what’s going on. Have you seen something like that during testing?