How to integrate own Custom tokenizer

hello guys I have transliteration Hindi dateset.With the inbuilt pipeline of RASA,the accuracy of the trained model is not up-to the mark.I have built a custom Tokenizer .How to replace the inbuilt Tokenizer with my own custom Tokenizer.

There’s a discussion in the docs here and you’ll find the source code for several example tokenizers here.

1 Like

After creating a custom tokenizer, what is the next step? I wonder if I can add a custom tokenizer to my current rasa project. I can’t find the folder name rasa in my rasa project. Where should I put the file in my project to override the NLU components?

Is there any solution for this?