# Custom setting in CountVectorsFeaturizer pipeline?

**URL:** <https://forum.rasa.com/t/custom-setting-in-countvectorsfeaturizer-pipeline/19624>\
**Category:** Rasa Open Source\
**Created:** [October 11, 2019, 7:57am UTC](https://forum.rasa.com/t/custom-setting-in-countvectorsfeaturizer-pipeline/19624 "2019-10-11T07:57:20Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![piyush29programmer](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/piyush29programmer/32/4050_2.png) [@piyush29programmer](https://forum.rasa.com/u/piyush29programmer)\
**Post date:** [October 11, 2019, 7:57am UTC](https://forum.rasa.com/t/custom-setting-in-countvectorsfeaturizer-pipeline/19624/1 "2019-10-11T07:57:20Z")

</div>

Hi,

I was going through the article of CountVectorsFeaturizer. [Components](https://rasa.com/docs/rasa/nlu/components/#countvectorsfeaturizer).

Could you please explain what is the use of setting

1. **use\_shared\_vocab**

2. **lowercase: true**

3. **token\_pattern: r’(?u)\b\w\w+\b’**

4. **max\_features: None # int or None**

Please provide an example if possible.

```
pipeline:
- name: "CountVectorsFeaturizer"
  # whether to use a shared vocab
  "use_shared_vocab": False,
  # whether to use word or character n-grams
  # 'char_wb' creates character n-grams only inside word boundaries
  # n-grams at the edges of words are padded with space.
  analyzer: 'word' # use 'char' or 'char_wb' for character
  # the parameters are taken from
  # sklearn's CountVectorizer
  # regular expression for tokens
  token_pattern: r'(?u)\b\w\w+\b'
  # remove accents during the preprocessing step
  strip_accents: None # {'ascii', 'unicode', None}
  # list of stop words
  stop_words: None # string {'english'}, list, or None (default)
  # min document frequency of a word to add to vocabulary
  # float - the parameter represents a proportion of documents
  # integer - absolute counts
  min_df: 1 # float in range [0.0, 1.0] or int
  # max document frequency of a word to add to vocabulary
  # float - the parameter represents a proportion of documents
  # integer - absolute counts
  max_df: 1.0 # float in range [0.0, 1.0] or int
  # set ngram range
  min_ngram: 1 # int
  max_ngram: 1 # int
  # limit vocabulary size
  max_features: None # int or None
  # if convert all characters to lowercase
  lowercase: true # bool
  # handling Out-Of-Vacabulary (OOV) words
  # will be converted to lowercase if lowercase is true
  OOV_token: None # string or None
  OOV_words: [] # list of strings
```

---

<div class="post-metadata">

**Author:** ![EliDll](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/elidll/32/4932_2.png) [@EliDll](https://forum.rasa.com/u/EliDll)\
**Post date:** [October 11, 2019, 9:37am UTC](https://forum.rasa.com/t/custom-setting-in-countvectorsfeaturizer-pipeline/19624/2 "2019-10-11T09:37:46Z")

</div>

Hey, I can help you with the first two.

The CountVectorsFeaturizer creates a **bag-of-words vector** of your **training-data sentences** and **intent-labels**

**use\_shared\_vocab** determines whether those two use the same representation, as in are mapped to the same dimensions.

Here’s an example:

- Training data: “Hello I am feeling well”
- Intents: “greet”, “sentiment”

Now, your vocabulary is the **entirity of tokens present in your training data**. For simplicity’s sake let’s use words as our tokens.

1. A **shared vocab** would look like this: (hello, I, am, feeling, well, greet, sentiment) and therefore:

- “Hello I am feeling well” ==\> (1,1,1,1,1,0,0)
- “greet” ==\> (0,0,0,0,0,1,0)
- “sentiment” ==\> (0,0,0,0,0,0,1)

1. **Different vocabs** would mean you have a (hello, I, am, feeling, well) vector for sentences and a (greet, sentiment) vector for intents, meaning

- “Hello I am feeling well” ==\> (1,1,1,1,1)
- “greet” ==\> (1,0)
- “sentiment” ==\> (0,1)

**Lowercase** means that everything will be converted to lowercase, therefore in your vocabulary, “Hello” and “hello” are treated as the same token, so

(Unrelated to the first example)

- lowercase: false “Hello hello there” ==\> (1,1,1)
- lowercase: true “Hello hello there” ==\> “hello hello there” ==\> (2,1)

---

<div class="post-metadata">

**Author:** ![piyush29programmer](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/piyush29programmer/32/4050_2.png) [@piyush29programmer](https://forum.rasa.com/u/piyush29programmer)\
**Post date:** [October 11, 2019, 2:22pm UTC](https://forum.rasa.com/t/custom-setting-in-countvectorsfeaturizer-pipeline/19624/3 "2019-10-11T14:22:58Z")

</div>

@EliDll Thank yor explaining in details with examples . I am beginner in ML/AI can you tell me what are pros and cons of using shared vocab and different vocab in rasa ?