# Have my own language tokenizer and specific classifiers

**URL:** <https://forum.rasa.com/t/have-my-own-language-tokenizer-and-specific-classifiers/4664>\
**Category:** Rasa Open Source\
**Created:** [January 25, 2019, 7:12pm UTC](https://forum.rasa.com/t/have-my-own-language-tokenizer-and-specific-classifiers/4664 "2019-01-25T19:12:52Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![elcolie](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/elcolie/32/1746_2.png) [@elcolie](https://forum.rasa.com/u/elcolie)\
**Post date:** [January 25, 2019, 7:12pm UTC](https://forum.rasa.com/t/have-my-own-language-tokenizer-and-specific-classifiers/4664/1 "2019-01-25T19:12:52Z")

</div>

Hi. Suppose I have [Thai tokenizer](https://github.com/PyThaiNLP/pythainlp) and also I have my own PyTorch trained classifiers. Let says I have 2 classifiers for different purposes. Let them read same sentence and return the prediction as output1 and output2 as follows

```python
output1 = model1(["first_tokenized_word", "second_tokenized_word", ..., "n_th_tokenized_word"]
output2 = model2(["first_tokenized_word", "second_tokenized_word", ..., "n_th_tokenized_word"]

```

1. How can I plug my models in to the Rasa pipeline? I have no intention to train the model in the Rasa. Because the training might imbalanced and I can’t use [ImbalancedDatasetSampler](https://github.com/ufoym/imbalanced-dataset-sampler) to fix it.

I have read thee [document](https://rasa.com/docs/nlu/choosing_pipeline/#section-component-lifecycle), but could not be able to understand.

1. I have seen the [pipeline](https://rasa.com/docs/nlu/components/) with adjustable parameter but in the `Component`'s constructor has no `**kwargs`. Then how to supply the my model configuration in `nlu_config.yml` like this.

```python
model = RNN(input_size=10, hidden_size=256, n_layper=2, dropout=0.2)

```

---

<div class="post-metadata">

**Author:** ![elcolie](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/elcolie/32/1746_2.png) [@elcolie](https://forum.rasa.com/u/elcolie)\
**Post date:** [January 26, 2019, 8:31pm UTC](https://forum.rasa.com/t/have-my-own-language-tokenizer-and-specific-classifiers/4664/2 "2019-01-26T20:31:45Z")

</div>

After trial and error for a while I can figure out some of my questions. I was confused with PyTorch training loop.

To answer first question.

1. create file ` __init__.py` in order to let my local directory becomes a module
2. create file `sentiment.py` and implement your [own](https://rasa.com/docs/nlu/customcomponents/) and set `pipeline` like this.

```auto
    language: "en"

    pipeline:
    - name: "sentiment.MyComponentA"
    - name: "sentiment.MyComponentB"

```

1. instantiate `model = Model()` somewhere in the class(MyCompomentA or MyCompomentB).

2. To complete train(). Access the dataset and follow the PyTorch Dataloader. I have to read training set from `training_data.training_examples`

3. Then save model