# CrfExtractor Pipeline

**URL:** <https://forum.rasa.com/t/crfextractor-pipeline/41472>\
**Category:** Rasa Open Source\
**Created:** [March 19, 2021, 10:49am UTC](https://forum.rasa.com/t/crfextractor-pipeline/41472 "2021-03-19T10:49:32Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![boutaina](https://avatars.discourse-cdn.com/v4/letter/b/d07c76/32.png) [@boutaina](https://forum.rasa.com/u/boutaina)\
**Post date:** [March 19, 2021, 10:49am UTC](https://forum.rasa.com/t/crfextractor-pipeline/41472/1 "2021-03-19T10:49:32Z")

</div>

Hello everyone , I tried to train my model with ner crf Extractor to extract some custom entities using a custom whitespace tokenizer but it gives the bellow error :  
‘CRFEntityExtractor’ requires [‘Tokenizer’]

And also when I try to use the normal whitespacetokenizer it still throws the same error , should I use a specific tokenizer or is the pipeline wrong , can you please help : Here is my pipeline

language: “en”

pipeline:

- name: “WhitespaceTokenizer”

- name: “CRFEntityExtractor”

- name: “ner\_synonyms”

- name: “CountVectorsFeaturizer”

- name: “intent\_featurizer\_count\_vectors”

- name: “intent\_classifier\_tensorflow\_embedding”

---

<div class="post-metadata">

**Author:** ![souvikg10](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/souvikg10/32/93_2.png) [@souvikg10](https://forum.rasa.com/u/souvikg10)\
**Post date:** [March 19, 2021, 12:43pm UTC](https://forum.rasa.com/t/crfextractor-pipeline/41472/2 "2021-03-19T12:43:58Z")

</div>

what is in your custom whitespace tokenizer??

> [@boutaina](#):
>
> I tried to train my model with ner crf Extractor to extract some custom entities using a custom whitespace tokenizer but it gives the bellow error : ‘CRFEntityExtractor’ requires [‘Tokenizer’]

---

<div class="post-metadata">

**Author:** ![boutaina](https://avatars.discourse-cdn.com/v4/letter/b/d07c76/32.png) [@boutaina](https://forum.rasa.com/u/boutaina)\
**Post date:** [March 19, 2021, 1:57pm UTC](https://forum.rasa.com/t/crfextractor-pipeline/41472/3 "2021-03-19T13:57:16Z")

</div>

Hello , the custom whitespace tokenizer is working great it looks like this :

class WhitespaceTokenizer\_ar\_cdg(Tokenizer, Component):

```
provides = [TOKENS_NAMES[attribute] for attribute in MESSAGE_ATTRIBUTES]

def unique_words(lines):

    return set(chain(*(line.split() for line in lines if line)))

dict= {}

defaults = {

    # text will be tokenized with case sensitive as default

    "case_sensitive": True

}

def __init__ (self, component_config: Dict[Text, Any] = None) -> None:

    """Construct a new tokenizer using the WhitespaceTokenizer framework."""

    super(WhitespaceTokenizer_ar_cdg, self). __init__ (component_config)

    self.case_sensitive = self.component_config["case_sensitive"]

```

The same as the original whitespacetokenizer I just added some code , that’s not the issue cause it’s working fine if I remove the CRFextractor . Once I add this specific extractor , it throws the error and even when I use just the normal whitespacetokenizer it doesnt work , still gives me this error ‘CRFEntityExtractor’ requires [‘Tokenizer’] as I showed below .

---

<div class="post-metadata">

**Author:** ![souvikg10](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/souvikg10/32/93_2.png) [@souvikg10](https://forum.rasa.com/u/souvikg10)\
**Post date:** [March 19, 2021, 2:59pm UTC](https://forum.rasa.com/t/crfextractor-pipeline/41472/4 "2021-03-19T14:59:43Z")

</div>

i am not really sure, but you are inherting the Component class in your Tokenizer?

> [@boutaina](#):
>
> class WhitespaceTokenizer\_ar\_cdg(Tokenizer, Component):

could you simply inherit -

```auto
class WhitespaceTokenizer_ar_cdg(Tokenizer):

```

because the Tokenizer class already inherits the components

> <https://github.com/RasaHQ/rasa/blob/8f8da24b486e8d9fc66d5b13ab176ad1f40b6a03/rasa/nlu/tokenizers/tokenizer.py#L64>

it is possible that CRF extractor is not receiving the correct class type

```auto
 def required_components(cls) -> List[Type[Component]]:
        return [Tokenizer]

```