# Case insensitive whitespace tokenizer

**URL:** <https://forum.rasa.com/t/case-insensitive-whitespace-tokenizer/45714>\
**Category:** Rasa Open Source\
**Created:** [July 13, 2021, 9:06pm UTC](https://forum.rasa.com/t/case-insensitive-whitespace-tokenizer/45714 "2021-07-13T21:06:52Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Bitzmann](https://avatars.discourse-cdn.com/v4/letter/b/e8c25b/32.png) [@Bitzmann](https://forum.rasa.com/u/Bitzmann)\
**Post date:** [July 13, 2021, 9:06pm UTC](https://forum.rasa.com/t/case-insensitive-whitespace-tokenizer/45714/1 "2021-07-13T21:06:52Z")

</div>

Hi, I found several forum posts advising people to use the following for implementing a case insensitive pipeline.

```auto
  - name: WhitespaceTokenizer 
    case_sentitive: False

```

but when I tried to use this, I get the following warning when training my model:

```auto
UserWarning: You have provided an invalid key `case_sentitive` for component `WhitespaceTokenizer` in your pipeline. Valid options for `WhitespaceTokenizer` are:
- intent_tokenization_flag
- token_pattern
- intent_split_symbol

```

Python version: 3.8.0

Rasa version: 2.2.0

my pipeline:

```auto
  - name: WhitespaceTokenizer 
    case_sentitive: False        
  - name: RegexFeaturizer      
    case_sensitive: False
  - name: LexicalSyntacticFeaturizer
  - name: CountVectorsFeaturizer
  - name: CountVectorsFeaturizer
    analyzer: char_wb
    min_ngram: 1
    max_ngram: 4    
  - name: DIETClassifier
  # entity_recognition: False
    epochs: 150 
  - name: CRFEntityExtractor
  - name: EntitySynonymMapper        
  # - name: ResponseSelector
  # epochs: 100
  - name: FallbackClassifier
    threshold: 0.5

```

---

<div class="post-metadata">

**Author:** ![nik202](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/nik202/32/16098_2.png) [@nik202](https://forum.rasa.com/u/nik202)\
**Post date:** [July 13, 2021, 9:38pm UTC](https://forum.rasa.com/t/case-insensitive-whitespace-tokenizer/45714/2 "2021-07-13T21:38:13Z")

</div>

@Bitzmann Have you check in the documentation of Rasa 2.0, [Components](https://rasa.com/docs/rasa/components/#whitespacetokenizer) may be its depreciated? As I can’t see or can you share the forum post for my ref?. Still I will check the error,.

@Bitzmann It is used for Regex for sure:

```
pipeline:
- name: "RegexFeaturizer"
  # Text will be processed with case sensitive as default
  "case_sensitive": True
  # use match word boundaries for lookup table
  "use_word_boundaries": True

```

Yes, me sure now it’s only used for RegexFeaturizer 🙂 you scared me 😛 that’s why you seeing the error as he is asking for:

```
pipeline:
- name: "WhitespaceTokenizer"
  # Flag to check whether to split intents
  "intent_tokenization_flag": False
  # Symbol on which intent should be split
  "intent_split_symbol": "_"
  # Regular expression to detect tokens
  "token_pattern": None

```

I hope its a solution for you 🙂

@Bitzmann If this suggestion and solution solved your error, can I request please close this topic as a solution for others.