BERT encoding

Hey guys!

Did anyone already implement BERT in Rasa?

Thanks, L-P

Hi, We’re working on finishing one up for our rasa-component suite innatis. It should be fully functional, we are just reviewing before pushing to pypi.

https://github.com/Revmaker/innatis/tree/bert

innatis/classifiers/bert_intent_classifier.py

4 Likes

Hey,

May be this might help you.

Hi @blakeandrewwood!

Thanks for that, I tested it and it works fine on my end :slight_smile:

Thanks as well @Varsha, will have a look at this.

1 Like

@blakeandrewwood

What base image are you using for Rasa NLU? I tried 0.14.14-tensorflow and spacy and I get killed processes.

Thanks, L-P

When I am trying to run this:

python -m rasa_nlu.evaluate --config conf/experiments/nlu4.yml --model /tmp/bert_exp/order_tracking/nlu/ --data data/nlu_test/

with the innatis bert classifier configured in nlu4.yml, I get the following:

/Users/yan/environments/rasa/lib/python3.6/site-packages/rasa_nlu/training_data/training_data.py:191: UserWarning: Intent ‘asking_refund+damaged_order’ has only 1 training examples! Minimum is 2, training may fail. self.MIN_EXAMPLES_PER_INTENT)) /Users/yan/environments/rasa/lib/python3.6/site-packages/rasa_nlu/training_data/training_data.py:191: UserWarning: Intent ‘cancel_order+update_order’ has only 1 training examples! Minimum is 2, training may fail. self.MIN_EXAMPLES_PER_INTENT)) /Users/yan/environments/rasa/lib/python3.6/site-packages/rasa_nlu/training_data/training_data.py:191: UserWarning: Intent ‘delayed_order+cancel_order’ has only 1 training examples! Minimum is 2, training may fail. self.MIN_EXAMPLES_PER_INTENT)) /Users/yan/environments/rasa/lib/python3.6/site-packages/rasa_nlu/training_data/training_data.py:191: UserWarning: Intent ‘general_qa+get_order_info’ has only 1 training examples! Minimum is 2, training may fail. self.MIN_EXAMPLES_PER_INTENT)) /Users/yan/environments/rasa/lib/python3.6/site-packages/rasa_nlu/training_data/training_data.py:191: UserWarning: Intent ‘get_order_info+cancel_order’ has only 1 training examples! Minimum is 2, training may fail. self.MIN_EXAMPLES_PER_INTENT)) Traceback (most recent call last): File “/Users/yan/.pyenv/versions/3.6.6/lib/python3.6/runpy.py”, line 193, in _run_module_as_main “main”, mod_spec) File “/Users/yan/.pyenv/versions/3.6.6/lib/python3.6/runpy.py”, line 85, in _run_code exec(code, run_globals) File “/Users/yan/environments/rasa/lib/python3.6/site-packages/rasa_nlu/evaluate.py”, line 978, in main() File “/Users/yan/environments/rasa/lib/python3.6/site-packages/rasa_nlu/evaluate.py”, line 972, in main cmdline_args.histogram) File “/Users/yan/environments/rasa/lib/python3.6/site-packages/rasa_nlu/evaluate.py”, line 732, in run_evaluation test_data) File “/Users/yan/environments/rasa/lib/python3.6/site-packages/rasa_nlu/evaluate.py”, line 636, in get_entity_predictions res = interpreter.parse(e.text, only_output_properties=False) File “/Users/yan/environments/rasa/lib/python3.6/site-packages/rasa_nlu/model.py”, line 357, in parse component.process(message, **self.context) File “/Users/yan/environments/rasa/lib/python3.6/site-packages/innatis/classifiers/bert_intent_classifier.py”, line 170, in process index = tf.argmax(probabilities, axis=0).eval() File “/Users/yan/environments/rasa/lib/python3.6/site-packages/tensorflow/python/framework/ops.py”, line 713, in eval return _eval_using_default_session(self, feed_dict, self.graph, session) File “/Users/yan/environments/rasa/lib/python3.6/site-packages/tensorflow/python/framework/ops.py”, line 5148, in _eval_using_default_session raise ValueError("Cannot use the default session to evaluate tensor: " ValueError: Cannot use the default session to evaluate tensor: the tensor’s graph is different from the session’s graph. Pass an explicit session to eval(session=sess).

Seems like some incorrect way of using the TF session, default vs. non default, but I am not really sure what’s going on. Have you seen something like that during testing?