I wanted to ask if it is possible to use a custom model for Named Entity Recognition? I wanted to use a pretrained Bi-LSTM CRF for NER since ner_crf doent give great accuracy and Mitie takes about the heat death of the universe to train. Please help guys
You can create a custom model, yes! You can take the ner_crf extractor as an example of what things an entity extractor should return etc https://github.com/RasaHQ/rasa_nlu/blob/master/rasa_nlu/extractors/crf_entity_extractor.py
There’s some info here about how to add it to your pipeline as well: http://rasa.com/docs/nlu/master/customcomponents/
Can I ask what kind of entities it’s not giving good accuracy for?
Thanks for replying. It usually it messes up the location based entities, part of the problem was the presence of stop words, but despite removing them the accuracy doesnt budge from 95%.
Actually we’re currently working on location entity recognition (cc @tyler). 95% is a pretty high accuracy though 
locations are quite a hard entity as there’s no real pattern in them sadly
Hi Jahangeer, After playing with RNN/CNN models for this problem myself and doing a lot of testing, they only do marginally better than ner_crf in rasa_nlu and take much longer to train and much more data. My suggestion would be to play with the features in the ner_crf, especially prefix and suffix features and add as much data to the training set as you can of examples with the location entities you care about. That should help a good amount. We may have some new features in later releases of rasa_nlu that will make this simpler but could be a few months before they come out.
Tyler
Hi Tyler thanks for the insightful advice. Optimizing feature does sound better than using heavier models.