Hello everyone, I actually have a bot that use lookup table. That’s why I need to use the CRFEntityExtractor to extract my entities. My intents are quite unbalanced, that’s why I would like to use the batch_strategy: sequence of DIETClassifier to help with this. Here is my Pipeline: language: "fr" pipeline: - name: WhitespaceTokenizer - name: RegexFeaturizer - name: LexicalSyntacticFeaturizer - name: CountVectorsFeaturizer - name: CountVectorsFeaturizer analyzer: "char_wb" min_ngram: 1 max_ngram: 4 - name: "CRFEntityExtractor" - name: DIETClassifier batch_strategy: sequence epochs: 50 - name: EntitySynonymMapper The CRFEntityExtractor detect correctly the entities, however the DIETClassifier will not detect the correct intent (because it doesn’t know the entities from the lookup table) Is there a way to specify which entity extractor we want to use to the DIETClassifier ? If not, is there anyone that have an idea how to handle unbalanced Intents and lookup tables ?

The CRFEntityExtractor detect correctly the entities, however the DIETClassifier will not detect the correct intent (because it doesn’t know the entities from the lookup table) Just to confirm, you’re worried that DIET doesn’t do the intents correctly or the entities? The way DIET handles intents is independant of the entities that are extracted from the CRFEntityExtractor.

DIET will not extract the entities that are in my lookuptable because only CRF does it. The DIETclassifier fail to recognize the correct intent if it cannot extract the entities associated with the Intent, even if CRF found it. It is sad that the DIETclassifier can not use the information of other entity extractor for his own classification. How could I use lookuptable and DIET classifier together?

@koaning was my reformulation clear ?

Hi @PaulB , i’m also curious about your request ,so here is what i think (please correct me if i’m wrong) from my understanding each component in the nlu pipeline will hand it’s result to the next one , so if CRFEntityExtractor will give it’s results to DIETClassifier , shouldn’t you consider setting entity_recognition to false if you don’t want the DIETClassifier influencing the entities extracted ?

Hello @pandaxar , Thank you for your suggestion. I already tried with the pipeline you suggest, but it’s not working. DIET seems to ignore previously extracted entities of CRF.

Perhaps you could disable the entity recognition part of the DIETClassifier? - name: DIETClassifier batch_strategy: sequence entity_recognition: False epochs: 50

Here is the exact pipeline I tried for your solution @n2718281 and @pandaxar Using a diet classifier with no entity extraction and a crf before in the pipeline language: "fr" pipeline: - name: WhitespaceTokenizer - name: RegexFeaturizer - name: LexicalSyntacticFeaturizer - name: CountVectorsFeaturizer - name: CountVectorsFeaturizer analyzer: "char_wb" min_ngram: 1 max_ngram: 4 - name: "CRFEntityExtractor" - name: DIETClassifier entity_recognition: False batch_strategy: sequence epochs: 50 - name: EntitySynonymMapper policies: - name: AugmentedMemoizationPolicy - name: TEDPolicy max_history: 5 epochs: 100 - name: FormPolicy - name: MappingPolicy - name: FallbackPolicy nlu_threshold: 0.1 ambiguity_threshold: 0.04 core_threshold: 0.3 Anyway, DIET is not using the entities extracted by CRF, so all the information of the lookup table is not used to help DIET with intent extraction.

Bonjour PaulB, I’m keen to help you as i’ll understand how things work too . Could you tell us what’s like the f1 score for entity extraction of the NLU pipeline used ?(remove the core policies for now)

Hello @pandaxar , Here are my results for entity extraction with CRF "micro avg": { "precision": 0.9542097488921714, "recall": 0.8765264586160109, "f1-score": 0.9137199434229137, "support": 737 }, "macro avg": { "precision": 0.938600900853585, "recall": 0.8516666782981712, "f1-score": 0.8925078225991568, "support": 737 }, "weighted avg": { "precision": 0.9543690335363008, "recall": 0.8765264586160109, "f1-score": 0.9131447598564554, "support": 737 } The report for DIET entity extractor is all on 0 because it’s disabled.

hm , the scores are pretty high , so how come the entities don’t get extracted ? can you enable diet for entity extraction and share with the us its scores again ? (does the pipeline without CRFEntityExtractor fails to extract entities in lookup tables ?) (if so will switching the position between the two components solves this issue ?) (edit1 :also ,did you include the lookup tables in the learning data ? xd it would be hilarious if it wasn’t) (edit2: hm , it seems CRFEntityExtractor and DIETClassifier aren’t used both in any rasa repositories)

Using the CRFEntityExtractor with the DIETClassifier

Rasa Open Source

sutgeorge (George Șut) July 22, 2024, 7:20am 18

Please don’t pollute threads with ChatGPT-generated responses…

Topic		Replies	Views
Is it possible to use CRFEntityExtractor after DIET? Rasa Open Source	1	496	September 28, 2020
Use DIETClassifier with custom rule-based entity extractor Rasa Open Source	5	1292	August 21, 2020
Is it possible to use DIETClassifier with lookup table to extract entities? Rasa Open Source	9	1973	February 2, 2023
Entity lookup in DIET Classifier Rasa Open Source	8	2312	May 2, 2022
Lookup table didn’t work for RegexFeaturizer + DIETClassifier Rasa Open Source	20	2151	February 4, 2022

Using the CRFEntityExtractor with the DIETClassifier

Related topics