Hi all, I’ve been playing around with Rasa (NLU-only) and am wondering how to achieve this –
I’m working on a slot-filling task, in a ‘low-data’ (i.e. 20 samples per label) setting. I want to train a custom entity extractor, and have decided to use CRFEntityExtractor (because I don’t have so much data for DIET).
Think of an utterance such as
"I want to depart from New York" with
NY = departure as slot label. My idea is to use a pre-trained NER extractor, e.g. from SpaCy to first extract New York as a city. Then, combine it with token embeddings, e.g. LanguageFeaturizer component to use a Transformer model to create contextual embeddings, and use both the entity label and token embeddings as features to train the CRF tagger.
- How could I combine an entity prediction from the pretrained NER and use it as a feature for the CRF?
- Would this be the correct config:
pipeline: - name: SpacyNLP model: en_core_web_trf case_sensitive: False - name: SpacyTokenizer - name: LanguageModelFeaturizer # Name of the language model to use # choose from ['bert', 'gpt', 'gpt2', 'xlnet', 'distilbert', 'roberta'] # or create a new class inheriting from this class to support your model. model_name: "bert" # Pre-Trained weights to be loaded model_weights: "bert-base-uncased" - name: LexicalSyntacticFeaturizer "features": [ # features for the word preceding the word being evaluated [ "suffix2", "prefix2", "pos2" ], # features for the word being evaluated [ "BOS", "EOS", "pos2" ], # features for the word following the word being evaluated [ "suffix2", "prefix2", "pos2" ]] - name: CRFEntityExtractor
Thanks for any help