hello guys I have transliteration Hindi dateset.With the inbuilt pipeline of RASA,the accuracy of the trained model is not up-to the mark.I have built a custom Tokenizer .How to replace the inbuilt Tokenizer with my own custom Tokenizer.
There’s a discussion in the docs here and you’ll find the source code for several example tokenizers here.
1 Like
After creating a custom tokenizer, what is the next step? I wonder if I can add a custom tokenizer to my current rasa project. I can’t find the folder name rasa in my rasa project. Where should I put the file in my project to override the NLU components?
Is there any solution for this?
You’ll find docs here. Once you’ve created your custom component, such as a tokenizer, include it in the config.yml as shown in this docs example. Do a train and run with --debug and make sure you see the component being used.