# How to integrate BERT to the nlu model?

**URL:** <https://forum.rasa.com/t/how-to-integrate-bert-to-the-nlu-model/15047>\
**Category:** Rasa Open Source\
**Created:** [August 30, 2019, 1:15pm UTC](https://forum.rasa.com/t/how-to-integrate-bert-to-the-nlu-model/15047 "2019-08-30T13:15:45Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![ychen](https://avatars.discourse-cdn.com/v4/letter/y/7ea924/32.png) [@ychen](https://forum.rasa.com/u/ychen)\
**Post date:** [August 30, 2019, 1:15pm UTC](https://forum.rasa.com/t/how-to-integrate-bert-to-the-nlu-model/15047/1 "2019-08-30T13:15:45Z")

</div>

Hi, I was wondering if anyone has any advice on how to integrate bert embeddings, intent classifier, etc to the nlu model?

---

<div class="post-metadata">

**Author:** ![IgNoRaNt23](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/ignorant23/32/8029_2.png) [@IgNoRaNt23](https://forum.rasa.com/u/IgNoRaNt23)\
**Post date:** [September 2, 2019, 4:15am UTC](https://forum.rasa.com/t/how-to-integrate-bert-to-the-nlu-model/15047/2 "2019-09-02T04:15:59Z")

</div>

You’ll have to write custom components for that.

---

<div class="post-metadata">

**Author:** ![JulianGerhard](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/juliangerhard/32/3781_2.png) [@JulianGerhard](https://forum.rasa.com/u/JulianGerhard)\
**Post date:** [September 3, 2019, 5:22pm UTC](https://forum.rasa.com/t/how-to-integrate-bert-to-the-nlu-model/15047/3 "2019-09-03T17:22:42Z")

</div>

Hi @ychen , hi @IgNoRaNt23,

I have created a [repo](https://github.com/JulianGerhard21/bert_spacy_rasa) for exactly that - without the necessity to create custom components.

If you need help, feel free to ask!

Regards Julian

---

<div class="post-metadata">

**Author:** ![JulianGerhard](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/juliangerhard/32/3781_2.png) [@JulianGerhard](https://forum.rasa.com/u/JulianGerhard)\
**Post date:** [December 28, 2019, 6:00pm UTC](https://forum.rasa.com/t/how-to-integrate-bert-to-the-nlu-model/15047/4 "2019-12-28T18:00:03Z")

</div>

Hi everyone,

I updated the [repo](https://github.com/JulianGerhard21/bert_spacy_rasa) today. Two new features were added:

- Support for DistilBERT and other `transformer`-based architectures
- Support for `NER` on a `transformer`-based model

Using e.g. DistilBERT resulted in very good results in terms of intent-detection with an absolute reasonable amount of training time. Now you would even be able to add your own custom entities and simply using the hybrid of spacy and rasa to auto-extract those entities for e.g. the usage in slots.

Regards Julian