Is there a good approach to leveraging both the pre-trained power of spaCy and the flexibility/trainability of a CRF model?
Elaboration: Say you use ‘ner_spacy’ for entity extraction to reasonably good effect in most cases. But then you have case-specific phrasing that is handled poorly by spaCy, and the only way to get good performance is to train a custom entity extractor, e.g. ‘ner_crf’. Ideally, I would like to just generate relevant training data (using Chatito) that covers the problematic phrases/entities, and stick the ‘ner_crf’ component in the pipeline to cover those. What would be the best way to do this though?
I’m mainly worrying about whether/how to disambiguate when spaCy classifies as one entity and CRF as another. Or should the full information be left to Core to decide what to do? I’m currently not using Core yet (but I plan to), and for my dialogue handler, it’s certainly much better if there aren’t conflicting entities in the NLU output.
I’ve thought about creating a MetaEntityExtractor trainable component that picks an entity from the extracted ones, using as features the entity type, entity extractor, extraction confidence, etc… A simpler version of this would be to just set a hard confidence threshold on the CRF output (e.g. 70%) and if the confidence is lower, go with the spaCy prediction. Any thoughts on this?