This is an interesting idea. You have to create the BERT model (or use anything existing) and make it part of the pipeline.
Does this link help? https://rasa.com/docs/nlu/0.13.8/customcomponents/
An interesting blog of someone adding a new model with Rasa is here https://scalableminds.com/blog/MachineLearning/2018/08/rasa-universal-sentence-encoder/
Hope this is helpful.